Image decoding apparatus and image encoding apparatus

Abstract

The image decoding apparatus (31) includes a transform coefficient decoding section (311) that decodes transform coefficients on a per transform tree basis included in a coding unit, the transform coefficient decoding section decoding transform coefficients related to luminance without splitting a transform unit corresponding to luminance in the transform tree, and decoding transform coefficients related to color difference without splitting a transform unit corresponding to color difference.

Description

[0001] This application is a divisional application of application No. 201880027757.6 entitled "Image Decoding Apparatus and Image Encoding Apparatus" filed with the China National Intellectual Property Administration on April 23, 2018. Technical Field

[0002] Embodiments of the present invention relate to a predictive image generation apparatus, an image decoding apparatus, and an image encoding apparatus. Background Technology

[0003] In order to efficiently transmit or record moving images, a moving image encoding device is used to generate encoded data by encoding the moving images, and a moving image decoding device is used to generate a decoded image by decoding the encoded data.

[0004] Specific examples of motion picture coding methods include those proposed in H.264 / AVC and HEVC (High-Efficiency Video Coding).

[0005] In this motion picture coding method, the image (picture) that constitutes the motion picture is managed by a hierarchical structure including slices obtained by segmenting the image, coding tree units (CTUs) obtained by segmenting the slices, coding units (sometimes also called coding units (CUs)) obtained by segmenting the coding tree units, and blocks obtained by segmenting the coding units, namely prediction units (PUs) and transform units (TUs), and is encoded / decoded on a per CU basis.

[0006] Furthermore, in this motion picture coding method, a prediction image is typically generated based on a locally decoded image obtained by encoding / decoding the input image, and the prediction residual (sometimes called a "difference image" or "residual image") obtained by subtracting the prediction image from the input image (original image) is encoded. Methods for generating the prediction image include inter-frame prediction and intra-frame prediction.

[0007] In addition, non-patent literature 1 can be cited as a technology for motion image encoding and decoding in recent years.

[0008] Furthermore, there exists a technique that independently segments and decodes luma and chromaticity blocks in association with the aforementioned prediction unit (PU) and transform unit (TU), thereby utilizing tree structures that differ in luma and chromaticity within the intra-frame image, i.e., intra-CTB tree structures (QTBT and 444 independent trees, etc.) that differ in luma and chromaticity. When there are different structures in luma and chromaticity, the transform size can be increased in the slowly changing chromaticity, thus concentrating the energy of the prediction residual, enabling encoding or decoding processing. Furthermore, inter-frame prediction techniques (merging mode) that generate prediction images with less code by utilizing surrounding prediction parameters and intra-frame prediction techniques (CCLM, Cross-component Linear Model) that generate chromaticity prediction images from luma images are known. Moreover, techniques that encode luma and chromaticity images with different images possess tree structures independent of luma and chromaticity.

[0009] Existing technical documents

[0010] Non-patent literature

[0011] Non-patent document 1: "Algorithm Description of Joint Exploration Test Model 5 (JEM5)", JVET-E1001-v2, Joint Video Exploration Team (JVET) of ITU-T SG 16WP3 and ISO / IEC JTC 1 / SC 29 / WG 11, 12-20 January 2017 Summary of the Invention

[0012] The problem the invention aims to solve

[0013] However, in techniques that use different tree structures for luminance and chrominance as described above, the amount of code required for the tree structure increases exponentially. Furthermore, when different tree structures are applied to inter-frame images (images using temporal prediction, such as unidirectional or bidirectional prediction), there is a problem that the amount of code increases because motion information (skip flags or merging index motion vector differences, etc.) also requires chrominance. Moreover, when combining intra-frame and inter-frame predictions using tree structures that separate luminance and chrominance, the problem of increased code required for the tree structure as described above persists.

[0014] One aspect of the present invention is made in view of the above-mentioned problems, and its main objective is to provide a technique for reducing the amount of code when applying different tree structures and different prediction methods for brightness and color difference.

[0015] Technical solution

[0016] To address the aforementioned problems, one aspect of the present invention provides an image decoding apparatus that decodes images per encoding unit, comprising a transform coefficient decoding unit that decodes transform coefficients per transform tree included in the encoding unit. The transform coefficient decoding unit decodes transform coefficients related to brightness after segmenting transform units corresponding to brightness in the transform tree, and decodes transform coefficients related to color difference without segmenting transform units corresponding to color difference.

[0017] To address the aforementioned problems, one aspect of the present invention provides an image encoding apparatus that encodes images according to each encoding unit, comprising a transform coefficient encoding unit that encodes transform coefficients according to each transform tree included in the encoding unit. The transform coefficient encoding unit encodes transform coefficients related to brightness after segmenting transform units corresponding to brightness in the transform tree, and encodes transform coefficients related to color difference without segmenting transform units corresponding to color difference.

[0018] To address the aforementioned problems, one aspect of the present invention provides an image decoding apparatus that decodes images per encoding unit, comprising a prediction image generation unit that generates a prediction image per prediction unit included in the encoding unit. The prediction image generation unit generates a prediction image related to the brightness of an object prediction unit using prediction parameters of a reference unit, and generates a prediction image related to the color difference of the object prediction unit by referring to the prediction image related to the brightness of the object prediction unit.

[0019] Beneficial effects

[0020] It can reduce the amount of code by applying different tree structures and different prediction methods for brightness and color difference. Attached Figure Description

[0021] Figure 1 This is a schematic diagram illustrating the configuration of an image transmission system according to one embodiment of the present invention.

[0022] Figure 2 This is a diagram illustrating the hierarchical structure of the encoded stream data in one embodiment of the present invention.

[0023] Figure 3 This is a diagram representing the PU segmentation scheme. Figure 3 (a) to Figure 3 The (h) represents the partition shape when the PU partitioning mode is 2N×2N, 2N×N, 2N×nU, 2N×nD, N×2N, nL×2N, nR×2N and N×N respectively.

[0024] Figure 4This is a concept diagram representing an example of a reference image and a list of reference images.

[0025] Figure 5 This is a block diagram illustrating the configuration of an image encoding apparatus according to one embodiment of the present invention.

[0026] Figure 6 This is a schematic diagram illustrating the configuration of an image decoding apparatus according to one embodiment of the present invention.

[0027] Figure 7 This is a schematic diagram illustrating the configuration of the inter-frame prediction image generation unit of an image coding apparatus according to one embodiment of the present invention.

[0028] Figure 8 This diagram illustrates the configuration of a transmitting device equipped with an image encoding apparatus and a receiving device equipped with an image decoding apparatus, according to one embodiment of the present invention. Figure 8 (a) indicates a transmitting device equipped with an image encoding device. Figure 8 (b) indicates a receiving device equipped with an image decoding device.

[0029] Figure 9 This diagram illustrates the configuration of a recording apparatus equipped with an image encoding device and a reproducing apparatus equipped with an image decoding device, according to one embodiment of the present invention. Figure 9 (a) indicates a recording device equipped with an image encoding device. Figure 9 (b) indicates a reproduction device equipped with an image decoding device.

[0030] Figure 10 This is a block diagram showing the main components of the image decoding apparatus according to Embodiment 1 of the present invention.

[0031] Figure 11 This is a flowchart illustrating an example of a block segmentation method for an image decoding apparatus 31 based on Embodiment 1 of the present invention.

[0032] Figure 12 This is a diagram used to illustrate the block segmentation method of Embodiment 1 of the present invention.

[0033] Figure 13 This is a flowchart illustrating an example of a block segmentation method for an image decoding apparatus 31 based on Embodiment 2 of the present invention.

[0034] Figure 14 This is a diagram used to illustrate the block segmentation method of Embodiment 2 of the present invention.

[0035] Figure 15 This is a diagram used to illustrate the partial division of Embodiment 2 of the present invention.

[0036] Figure 16 This is a diagram used to illustrate the partial division of Embodiment 2 of the present invention.

[0037] Figure 17 This is a diagram used to illustrate a block segmentation method for a modified example of Embodiment 2 of the present invention.

[0038] Figure 18 (a) is a block diagram showing the main components of the image decoding apparatus according to Embodiment 3 of the present invention. Figure 18 (b) means Figure 18 (a) is a block diagram of the main components.

[0039] Figure 19 This is a flowchart illustrating an example of a predictive image derivation method based on the image decoding apparatus 31 according to Embodiment 3 of the present invention.

[0040] Figure 20 This is a flowchart illustrating an example of a prediction mode determination method for an image decoding apparatus 31 based on Embodiment 3 of the present invention.

[0041] Figure 21 This is a diagram used to illustrate the combination of the block segmentation method of Embodiment 1 and the predicted image derivation method of Embodiment 3.

[0042] Figure 22 This is a diagram used to illustrate the combination of the block segmentation method of Embodiment 2 and the predicted image derivation method of Embodiment 3.

[0043] Figure 23 This diagram is used to illustrate a combination of a modified example of the block segmentation method of Embodiment 2 and the predicted image derivation method of Embodiment 3.

[0044] Figure 24 This is a diagram used to illustrate the method for creating a merged candidate list according to Embodiment 3 of the present invention. Detailed Implementation

[0045] (First Implementation)

[0046] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings.

[0047] Figure 1 This is a schematic diagram showing the configuration of the image transmission system 1 of this embodiment.

[0048] Image transmission system 1 is a system that transmits code obtained by encoding an image of an encoded object and displays an image obtained by decoding the transmitted code. Image transmission system 1 is configured to include an image encoding device (moving picture encoding device) 11, a network 21, an image decoding device (moving picture decoding device) 31, and an image display device 41.

[0049] Image encoding apparatus 11 receives an image T representing a single layer or multiple layers of images. A layer is a concept used to distinguish multiple images when there are more than one image constituting a given moment. For example, encoding the same image across multiple layers with different image quality and resolution is called scalable encoding, and encoding images from different viewpoints across multiple layers is called viewpoint-scalable encoding. Encoding efficiency is significantly improved when prediction is performed between images in multiple layers (inter-layer prediction, inter-viewpoint prediction). Furthermore, encoded data can be aggregated even without prediction (simultaneous broadcasting).

[0050] Network 21 transmits the encoded stream Te generated by image encoding device 11 to image decoding device 31. Network 21 is the Internet, a Wide Area Network (WAN), a Local Area Network (LAN), or a combination thereof. Network 21 is not necessarily limited to a two-way communication network; it can also be a one-way communication network transmitting broadcast waves such as terrestrial digital broadcasts or satellite broadcasts. Furthermore, network 21 can also be replaced by storage media such as DVDs (Digital Versatile Discs) or Blu-ray Discs (BDs) that record the encoded stream Te.

[0051] The image decoding device 31 decodes the encoded stream Te transmitted by the network 21 respectively, and generates one or more decoded images Td respectively.

[0052] The image display device 41 displays all or part of one or more decoded images Td generated by the image decoding device 31. The image display device 41 may include, for example, a liquid crystal display (LCD) or an organic EL (electroluminescence) display. Furthermore, in spatially scalable coding and SNR-scalable coding, when the image decoding device 31 and the image display device 41 have high processing capabilities, a high-quality extended layer image is displayed; when they have only low processing capabilities, a basic layer image that does not require the high processing and display capabilities of an extended layer is displayed.

[0053] <operator>

[0054] The operators used in this specification are described below.

[0055] >> shifts to the right, << shifts to the left, & is bitwise AND, | is bitwise OR, and |= is the OR substitution operator.

[0056] x? y: z is a 3-term operator that takes y when x is true (other than 0) and takes z when x is false (0).

[0057] Clip3(a, b, c) is a function that limits c to values ​​greater than a and less than b. It returns a if c < a, returns b if c > b, and returns c otherwise (where a <= b).

[0058] X mod Y is the remainder of X when Y is used as the divisor.

[0059] <Structure of encoded stream Te>

[0060] Before providing a detailed description of the image encoding device 11 and the image decoding device 31 of this embodiment, the data structure of the encoded stream Te generated by the image encoding device 11 and decoded by the image decoding device 31 will be described.

[0061] Figure 2 This is a diagram representing the hierarchical structure of the data in the encoded stream Te. The encoded stream Te exemplarily includes a sequence and multiple images that constitute the sequence. Figure 2 (a) to (f) are respectively diagrams representing the encoded video sequence of the given sequence SEQ, the encoded picture of the specified picture PICT, the encoded slice of the specified slice S, the encoded slice data of the specified slice data, the coding tree unit included in the encoded slice data, and the coding unit (CU) included in the coding tree unit.

[0062] (Encoded video sequence)

[0063] In the encoded video sequence, a set of data is specified that the image decoding device 31 refers to in order to decode the sequence SEQ of the object being processed. The sequence SEQ is as follows: Figure 2 As shown in (a), this includes the Video Parameter Set, Sequence Parameter Set (SPS), Picture Parameter Set (PPS), Picture PICT, and Supplemental Enhancement Information (SEI). Here, the value following the # indicates the layer ID. Figure 2 The example shown is an example of encoded data with #0 and #1, i.e., layer 0 and layer 1, but the type and number of layers do not depend on this.

[0064] In a motion picture composed of multiple layers, the Video Parameter Set (VPS) specifies a set of coding parameters shared by multiple motion pictures, as well as a set of coding parameters associated with the multiple layers included in the motion picture.

[0065] The Sequence Parameter Set (SPS) specifies a set of encoding parameters that the image decoding device 31 references to decode the object sequence. For example, it specifies the width and height of the image. It should be noted that multiple SPSs can exist. In this case, any one of the multiple SPSs is selected from the PPS.

[0066] The image parameter set (PPS) specifies a set of encoding parameters that the image decoding device 31 refers to when decoding each image in the object sequence. These include, for example, a reference value for the quantization width used for image decoding (pic_init_qp_minus26) and a flag indicating the application of weighted prediction (weighted_pred_flag). It should be noted that multiple PPSs can exist. In this case, any one of the multiple PPSs is selected from the images in the object sequence.

[0067] (Encoded image)

[0068] In the encoded image, a set of data is specified that the image decoding device 31 refers to in order to decode the image PICT of the object being processed. The image PICT is as follows: Figure 2 As shown in (b), it includes slices S0 to SNS-1 (NS is the total number of slices included in the image PICT).

[0069] It should be noted that, in the following descriptions, when there is no need to distinguish between the slices S0 to SNS-1, the code suffix may sometimes be omitted. Furthermore, the same applies to other data included in the encoded stream Te that also have suffixes.

[0070] (Encoded slice)

[0071] In the encoded slice, a set of data is specified that the image decoding device 31 refers to in order to decode the slice S of the object being processed. Slice S is as follows: Figure 2 As shown in (c), it includes the slice header SH and the slice data SDATA.

[0072] The slice header SH includes a group of encoded parameters referenced by the image decoding device 31 to determine the decoding method for the object slice. The slice type specification information (slice_type) is an example of the encoded parameters included in the slice header SH.

[0073] Slice types that can be specified by slice type specification information include: (1) I slices that use only intra-frame prediction during encoding, (2) P slices that use unidirectional prediction or intra-frame prediction during encoding, and (3) B slices that use unidirectional prediction, bidirectional prediction or intra-frame prediction during encoding, etc.

[0074] It should be noted that the slice header SH may also include a reference to the picture parameter set PPS (pic_parameter_set_id) included in the above-mentioned encoded video sequence.

[0075] (Encoded slice data)

[0076] In the encoded slice data, a set of data is specified that the image decoding device 31 refers to in order to decode the slice data SDATA of the object being processed. The slice data SDATA is as follows: Figure 2 As shown in (d), this includes Coding Tree Units (CTUs). A CTU is a fixed-size (e.g., 64×64) block that makes up a slice, and is sometimes referred to as the Largest Coding Unit (LCU).

[0077] (Coding Tree Unit)

[0078] like Figure 2 As shown in (e), the image decoding device 31 specifies a set of data referenced for decoding the coding tree unit of the object being processed. The coding tree unit is segmented by recursive quadtree segmentation (QT segmentation) or binary tree segmentation (BT segmentation). The nodes of the tree structure obtained by recursive quadtree segmentation or binary tree segmentation are called coding nodes (CN). The intermediate nodes of the quadtree and binary tree are called coding trees (CT), and the coding tree unit itself is also defined as the top-level coding tree.

[0079] The CTU includes: a QT splitting flag (cu_split_flag) indicating whether QT splitting is performed, and a BT splitting mode (split_bt_mode) indicating the splitting method used for BT splitting. When cu_split_flag is 1, the split is performed into 4 coding nodes (CN). When cu_split_flag is 0, each coding node (CN) has a coding unit (CU) and is not split.

[0080] The coding unit (CU) is the terminal node (leaf node) of the coding node and is not further segmented. The coding unit (CU) is the basic unit of coding processing.

[0081] Furthermore, when the size of the coding tree unit (CTU) is 64×64 pixels, the size of the coding unit can be any one of 64×64 pixels, 64×32 pixels, 32×64 pixels, 32×32 pixels, 64×16 pixels, 16×64 pixels, 32×16 pixels, 16×32 pixels, 16×16 pixels, 64×8 pixels, 8×64 pixels, 32×8 pixels, 8×32 pixels, 16×8 pixels, 8×16 pixels, and 8×8 pixels.

[0082] (Encoding unit)

[0083] like Figure 2 As shown in (f), the image decoding device 31 specifies a set of data referenced by the image decoding device 31 for decoding the encoding unit of the processing object. Specifically, the encoding unit includes a prediction tree, a transform tree, and a CU header CUH. The CU header specifies the prediction mode, segmentation method (PU segmentation mode), etc.

[0084] In the prediction tree, the prediction information (referencing image index, motion vector, etc.) of each prediction unit (PU) obtained by dividing the coding unit into one or more is specified. Alternatively, a prediction unit is one or more non-repeating regions that constitute a coding unit. Furthermore, the prediction tree includes one or more prediction units obtained through the above division. It should be noted that the prediction unit obtained by further dividing the prediction unit is referred to as a "sub-block". A sub-block consists of multiple pixels. When the size of the prediction unit and the sub-block are equal, there is one sub-block in the prediction unit. When the size of the prediction unit is larger than the size of the sub-block, the prediction unit is divided into sub-blocks. For example, if the prediction unit is 8×8 and the sub-block is 4×4, the prediction unit will be divided into four sub-blocks, including two horizontally divided parts and two vertically divided parts.

[0085] Prediction processing can be performed on a per-prediction-unit (sub-block) basis.

[0086] There are two main types of segmentation in prediction trees: intra-frame prediction and inter-frame prediction. Intra-frame prediction is prediction within the same image, while inter-frame prediction refers to prediction processing performed between different images (e.g., between display times or between layer images).

[0087] In the case of intra-frame prediction, the segmentation methods are 2N×2N (same size as the coding unit) and N×N.

[0088] Furthermore, in the case of inter-frame prediction, the segmentation method encodes according to the PU segmentation mode (part_mode) of the encoded data, and includes 2N×2N (same size as the coding unit), 2N×N, 2N×nU, 2N×nD, N×2N, nL×2N, nR×2N, and N×N, etc. It should be noted that 2N×N and N×2N represent 1:1 symmetrical segmentation.

[0089] 2N×nU, 2N×nD, nL×2N, and nR×2N represent asymmetric partitions of 1:3 and 3:1, respectively. The PUs included in CU are denoted as PU0, PU1, PU2, and PU3, respectively.

[0090] Figure 3 Figures (a) through (h) illustrate the shape of the partitions in each PU partitioning mode (the location of the PU partition boundaries). Figure 3 (a) represents a 2N×2N partition. Figure 3 (b) Figure 3 (c) Figure 3 (d) represents the partitions (horizontal partitions) of 2N×N, 2N×nU, and 2N×nD, respectively. Figure 3 of (e) Figure 3 of (f), Figure 3 (g) represents the partitions (vertical partitions) for N×2N, nL×2N, and nR×2N respectively. Figure 3 (h) represents an N×N partition. It should be noted that horizontal and vertical partitions are collectively referred to as rectangular partitions, and 2N×2N and N×N partitions are collectively referred to as square partitions.

[0091] Furthermore, in the Transform Tree (TT), a coding unit is divided into one or more Transform Units (TUs), and the position and size of each Transform Unit are specified. Alternatively, a Transform Unit is one or more non-repeating regions that constitute a coding unit. Additionally, the Transform Tree includes one or more Transform Units obtained through the above division.

[0092] In the segmentation of the transform tree, there are two methods: allocating regions of the same size as the coding unit as the segmentation of the transform unit; and obtaining the segmentation of the transform unit by performing quadtree segmentation (TU segmentation) on the CU, similar to the segmentation of the CU described above. The transformation process is performed on each of these transform units.

[0093] (Prediction parameters)

[0094] The predicted image of the Prediction Unit (PU) is derived from the prediction parameters associated with the PU. These prediction parameters include intra-frame prediction parameters and inter-frame prediction parameters. The following explains the inter-frame prediction parameters. The inter-frame prediction parameters consist of prediction list flags predFlagL0 and predFlagL1, reference image indices refidxL0 and refidxL1, and motion vectors mvL0 and mvL1. The prediction list flags predFlagL0 and predFlagL1 indicate whether to use the respective reference image lists, referred to as the L0 list and L1 list. A value of 1 indicates that the corresponding reference image list is used. It should be noted that in this specification, when a flag is referred to as "a flag indicating whether it is ××", a flag other than 0 (e.g., 1) is considered to be ××, and a flag of 0 is considered not to be ××. In logical NOT, logical product, etc., 1 is considered true, and 0 is considered false (the same applies below). However, in actual devices and methods, other values ​​can also be used as true or false values.

[0095] Syntax elements used to derive inter-frame prediction parameters included in the encoded data, such as part_mode (PU segmentation mode), merge_flag (merge flag), merge_idx (merge index), inter_pred_idc (inter-frame prediction identifier), refIdxLX (reference image index), mvp_LX_idx (prediction vector index), and mvdLX (differential vector).

[0096] (List of reference images)

[0097] The reference image list is a list of reference images stored in the reference image memory 306. Figure 4 This is a concept diagram representing an example of a reference image and a list of reference images. Figure 4 In (a), rectangles represent images, arrows indicate the reference relationships between images, the horizontal axis represents time, and I, P, and B within the rectangles represent intra-frame images, single-prediction images, and double-prediction images, respectively. The numbers within the rectangles indicate the decoding order. As shown in the figure, the decoding order of the images is I0, P1, B2, B3, B4, and the display order is I0, B3, B2, B4, P1. Figure 4Example of a reference image list is shown in (b). A reference image list is a list of candidates for reference images; an image (slice) can have more than one reference image list. In the example of the figure, the object image B3 has two reference image lists: L0 list RefPicList0 and L1 list RefPicList1. The reference images for the object image B3 are I0, P1, and B2, which are used as features. In each prediction cell, the reference image index refIdxLX specifies which image in the reference image list RefPicListX is actually referenced. The figure shows an example of referencing reference images P1 and B2 based on refIdxL0 and refIdxL1.

[0098] (Merge forecast and AMVP forecast)

[0099] The decoding (encoding) methods for prediction parameters include a merge prediction mode and an AMVP (Adaptive Motion Vector Prediction) mode. The merge flag is used to identify them. The merge prediction mode prevents the prediction list from utilizing the predFlagLX (or inter-frame prediction identifier inter_pred_idc), the reference image index refIdxLX, and the motion vector mvLX in the encoded data. The AMVP mode includes the inter-frame prediction identifier inter_pred_idc, the reference image index refIdxLX, and the motion vector mvLX in the encoded data, while using the prediction parameters from the processed nearby PUs. It should be noted that the motion vector mvLX is encoded as the prediction vector index mvp_LX_idx and the difference vector mvdLX, which identify the prediction vector mvpLX.

[0100] The inter-prediction identifier `inter_pred_idc` represents the type and number of reference images, taking any one of `PRED_L0`, `PRED_L1`, or `PRED_BI`. `PRED_L0` and `PRED_L1` indicate the use of reference images managed in the L0 and L1 lists, respectively, and represent the use of one reference image (single prediction). `PRED_BI` indicates the use of two reference images (bipred prediction), using reference images managed in both the L0 and L1 lists. The prediction vector index `mvp_LX_idx` represents the index of the prediction vector, and the reference image index `refIdxLX` represents the index of the reference image managed in the reference image list. It should be noted that `LX` is a notation used when there is no distinction between L0 and L1 predictions. Replacing `LX` with `L0` or `L1` distinguishes between parameters for the L0 list and parameters for the L1 list.

[0101] The merge index merge_idx is an index indicating whether to use any of the prediction parameter candidates (merge candidates) derived from the processed PU as the prediction parameter for the decoded object PU.

[0102] (Motion Vector)

[0103] The motion vector mvLX represents the offset between blocks on two different images. The prediction vector and the difference vector related to the motion vector mvLX are called the prediction vector mvpLX and the difference vector mvdLX, respectively.

[0104] (The inter-frame prediction identifier inter_pred_idc and the prediction list utilize the predFlagLX)

[0105] The inter-frame prediction identifier inter_pred_idc and the prediction list can be transformed into each other using the relationship between the flags predFlagL0 and predFlagL1 as follows.

[0106] inter_pred_idc=(predFlagL1<<1)+predFlagL0

[0107] predFlagL0 = inter_pred_idc & 1

[0108] predFlagL1=inter_pred_idc>>1

[0109] It should be noted that inter-frame prediction parameters can use either the prediction list utilization flag or the inter-frame prediction identifier. Furthermore, a decision using the prediction list utilization flag can be replaced by a decision using the inter-frame prediction identifier, and vice versa.

[0110] (Determination of biPred prediction)

[0111] The flag indicating whether a BiPred is a double-prediction BiPred can be derived from both prediction lists by checking if both flags are 1. For example, it can be derived using the following formula.

[0112] biPred=(predFlagL0==1&&predFlagL1==1)

[0113] The flag biPred can also be derived based on whether the inter-frame prediction identifier indicates the use of two prediction lists (see reference image). For example, it can be derived using the following formula.

[0114] biPred=(inter_pred_idc==PRED_BI)? 1:0

[0115] The above formula can also be expressed by the following formula.

[0116] biPred=(inter_pred_idc==PRED_BI)

[0117] It should be noted that PRED_BI can use a value such as 3.

[0118] (Composition of an image decoding device)

[0119] Next, the configuration of the image decoding device 31 in this embodiment will be described. Figure 5 This is a schematic diagram showing the configuration of the image decoding apparatus 31 of this embodiment. The image decoding apparatus 31 is configured to include: an entropy decoding unit 301, a prediction parameter decoding unit (predictive image decoding apparatus) 302, a loop filter 305, a reference image memory 306, a prediction parameter memory 307, a prediction image generation unit (predictive image generation apparatus) 308, an inverse quantization / inverse transform unit 311, and an adder unit 312.

[0120] Furthermore, the prediction parameter decoding unit 302 is configured to include an inter-frame prediction parameter decoding unit 303 and an intra-frame prediction parameter decoding unit 304. The prediction image generation unit 308 is configured to include an inter-frame prediction image generation unit 309 and an intra-frame prediction image generation unit 310.

[0121] The entropy decoding unit 301 performs entropy decoding on the externally input encoded stream Te, separating and decoding each code (syntactic element). The separated code contains prediction information for generating the predicted image and residual information for generating the difference image.

[0122] The entropy decoding unit 301 outputs a portion of the separated code to the prediction parameter decoding unit 302. This portion of the separated code includes, for example, the prediction mode `predMode`, the PU segmentation mode `part_mode`, the merge flag `merge_flag`, the merge index `merge_idx`, the inter-frame prediction identifier `inter_pred_idc`, the reference image index `refIdxLX`, the prediction vector index `mvp_LX_idx`, and the difference vector `mvdLX`. The unit controls which code to decode based on the instructions from the prediction parameter decoding unit 302. The entropy decoding unit 301 outputs quantization coefficients to the inverse quantization / inverse transform unit 311. These quantization coefficients are obtained by performing frequency transformations such as DCT (Discrete Cosine Transform), DST (Discrete Sine Transform), and KLT (Karyhnen Loeve Transform) on the residual signal during the encoding process and then quantizing it.

[0123] The inter-frame prediction parameter decoding unit 303 decodes the inter-frame prediction parameters based on the code input from the entropy decoding unit 301 and with reference to the prediction parameters stored in the prediction parameter memory 307.

[0124] The inter-frame prediction parameter decoding unit 303 outputs the decoded inter-frame prediction parameters to the prediction image generation unit 308, and stores them in the prediction parameter memory 307. Details of the inter-frame prediction parameter decoding unit 303 will be described below.

[0125] The intra-prediction parameter decoding unit 304 decodes the intra-prediction parameters based on the code input from the entropy decoding unit 301 and with reference to the prediction parameters stored in the prediction parameter memory 307. Intra-prediction parameters refer to the parameters used in the process of predicting the CU within an image, such as the intra-prediction mode (IntraPredMode). The intra-prediction parameter decoding unit 304 outputs the decoded intra-prediction parameters to the prediction image generation unit 308 and stores them in the prediction parameter memory 307.

[0126] The intra-prediction parameter decoding unit 304 can also derive intra-prediction modes that differ in luminance and chrominance. In this case, the intra-prediction parameter decoding unit 304 decodes the luminance prediction mode IntraPredModeY as the luminance prediction parameter and the chrominance prediction mode IntraPredModeC as the chrominance prediction parameter. The luminance prediction mode IntraPredModeY has 35 modes, corresponding to planar prediction (0), DC prediction (1), and directional prediction (2-34). The chrominance prediction mode IntraPredModeC uses any one of planar prediction (0), DC prediction (1), directional prediction (2-34), and LM mode (35). The intra-prediction parameter decoding unit 304 can also decode the flag indicating whether IntraPredModeC is the same mode as the luminance mode. If the flag indicates that it is the same mode as the luminance mode, IntraPredModeY is assigned to IntraPredModeC. If the flag indicates that it is a different mode from the luminance mode, Plane Prediction (0), DC Prediction (1), Direction Prediction (2-34), and LM Mode (35) are decoded as IntraPredModeC.

[0127] The loop filter 305 applies deblocking filtering, sample adaptive offset (SAO), and adaptive loop filtering (ALF) to the decoded image of the CU generated by the adder 312.

[0128] The reference image memory 306 stores the decoded images of the CUs generated by the loop filter 305 in predetermined locations according to each image and CU of the decoded object.

[0129] The prediction parameter memory 307 stores prediction parameters in predetermined locations for each image and prediction unit (or sub-block, fixed-size block, pixel) of the decoded object. Specifically, the prediction parameter memory 307 stores inter-frame prediction parameters decoded by the inter-frame prediction parameter decoding unit 303, intra-frame prediction parameters decoded by the intra-frame prediction parameter decoding unit 304, and prediction modes separated by the entropy decoding unit 301. Among the stored inter-frame prediction parameters are, for example, the prediction list utilization flag predFlagLX (inter-frame prediction identifier inter_pred_idc), the reference image index refIdxLX, and the motion vector mvLX.

[0130] The prediction image generation unit 308 receives the prediction mode `predMode` from the entropy decoding unit 301, and prediction parameters from the prediction parameter decoding unit 302. Furthermore, the prediction image generation unit 308 reads a reference image from the reference image memory 306. Under the prediction mode indicated by `predMode`, the prediction image generation unit 308 uses the input prediction parameters and the read reference image (reference image block) to generate a prediction image for the PU or sub-block.

[0131] Here, when the prediction mode predMode represents the inter-frame prediction mode, the inter-frame prediction image generation unit 309 uses the inter-frame prediction parameters input from the inter-frame prediction parameter decoding unit 303 and the read reference image (reference image block) to generate a prediction image of the PU or sub-block through inter-frame prediction.

[0132] The inter-frame prediction image generation unit 309 uses a list of reference images (L0 list or L1 list) with the flag predFlagLX set to 1 to read reference image blocks from the reference image memory 306 according to the reference image indicated by the reference image index refIdxLX. These reference image blocks are located at positions indicated by the motion vector mvLX based on the decoded object PU. The inter-frame prediction image generation unit 309 performs predictions based on the read reference image blocks to generate a prediction image of the PU. The inter-frame prediction image generation unit 309 outputs the generated prediction image of the PU to the addition unit 312. Here, a reference image block refers to a set of pixels (usually rectangular and therefore called a block) on a reference image, which is a region referenced for generating prediction images of the PU or sub-blocks.

[0133] When the prediction mode (predMode) indicates intra-prediction mode, the intra-prediction image generation unit 310 performs intra-prediction using intra-prediction parameters input from the intra-prediction parameter decoding unit 304 and read reference images. Specifically, the intra-prediction image generation unit 310 reads from the reference image memory 306 the adjacent PUs located within a predetermined range from the decoded PU in the decoded PU of the image of the decoded object. The predetermined range refers to any one of the adjacent PUs, such as left, upper left, upper, or upper right, when the decoded object PUs are moved sequentially in a so-called raster scan order, and varies depending on the intra-prediction mode. The raster scan order is the order in which rows in each image are moved sequentially from left to right from top to bottom.

[0134] The intra-prediction image generation unit 310 predicts the read-out adjacent PUs in the prediction mode shown in IntraPredMode and generates a prediction image of the PU. The intra-prediction image generation unit 310 outputs the generated prediction image of the PU to the adder 312.

[0135] In the intra-prediction parameter decoding unit 304, when different intra-prediction modes are derived for luminance and chrominance, the intra-prediction image generation unit 310 generates a luminance prediction image of the PU by any one of planar prediction (0), DC prediction (1), and directional prediction (2-34) according to the luminance prediction mode IntraPredModeY, and generates a chrominance prediction image of the PU by any one of planar prediction (0), DC prediction (1), directional prediction (2-34), and LM mode (35) according to the chrominance prediction mode IntraPredModeC.

[0136] The inverse quantization / inverse transform unit 311 dequantizes the quantization coefficients input from the entropy decoding unit 301 to obtain the transform coefficients. The inverse quantization / inverse transform unit 311 performs inverse frequency transforms such as inverse DCT, inverse DST, and inverse KLT on the obtained transform coefficients to calculate the residual signal. The inverse quantization / inverse transform unit 311 outputs the calculated residual signal to the adder unit 312.

[0137] The adder 312 adds the predicted image of the PU input from the inter-frame prediction image generation unit 309 or the intra-frame prediction image generation unit 310 to the residual signal input from the inverse quantization / inverse transform unit 311 for each pixel to generate a decoded image of the PU. The loop filter 305 stores the generated decoded image of the PU in the reference image memory 306 and outputs the decoded image Td obtained by integrating the decoded images of the PU generated for each image to the outside.

[0138] (Composition of an image encoding device)

[0139] Next, the configuration of the image encoding device 11 in this embodiment will be described. Figure 6 This is a block diagram illustrating the configuration of the image coding apparatus 11 according to this embodiment. The image coding apparatus 11 is configured to include: a prediction image generation unit 101, a subtraction unit 102, a transform / quantization unit 103, an entropy coding unit 104, an inverse quantization / inverse transform unit 105, an addition unit 106, a loop filter 107, a prediction parameter memory (prediction parameter storage unit, frame memory) 108, a reference image memory (reference image storage unit, frame memory) 109, a coding parameter determination unit 110, and a prediction parameter coding unit 111. The prediction parameter coding unit 111 is configured to include an inter-frame prediction parameter coding unit 112 and an intra-frame prediction parameter coding unit 113.

[0140] The prediction image generation unit 101 generates a prediction image P for each image T, divided into regions (i.e., coding units CU) of that image. Here, the prediction image generation unit 101 reads the decoded blocks from the reference image memory 109 based on prediction parameters input from the prediction parameter encoding unit 111. The prediction parameters input from the prediction parameter encoding unit 111, for example, in the case of inter-frame prediction, refer to motion vectors. The prediction image generation unit 101 reads the blocks located on the reference image at positions indicated by the motion vectors, starting from the object PU. Furthermore, in the case of intra-frame prediction, the prediction parameters refer to, for example, an intra-frame prediction mode. The pixel values ​​of adjacent PUs used in the intra-frame prediction mode are read from the reference image memory 109, and a prediction image P for the PU is generated. The prediction image generation unit 101 generates a prediction image of the PU using one of a plurality of prediction modes for the read-out reference image blocks. The prediction image generation unit 101 outputs the generated prediction image of the PU to the subtraction unit 102.

[0141] It should be noted that the predicted image generation unit 101 operates in the same manner as the previously described predicted image generation unit 308. For example, Figure 7 This is a schematic diagram showing the configuration of the inter-frame prediction image generation unit 1011 included in the prediction image generation unit 101. The inter-frame prediction image generation unit 1011 is configured to include a motion compensation unit 10111 and a weight prediction unit 10112.

[0142] (Motion compensation)

[0143] The motion compensation unit 10111 reads from the reference image memory 109, based on the inter-frame prediction parameters (prediction list using flag predFlagLX, reference image index refIdxLX, and motion vector mvLX) input from the inter-frame prediction parameter encoding unit 112, the block located at the position of the motion vector mvLX offset from the position of the decoded object PU as the starting point, in the reference image specified by the reference image index refIdxLX, thereby generating an interpolated image (motion-compensated image). Here, if the precision of the motion vector mvLX is not integer precision, a filter called motion compensation filtering is performed to generate pixels at fractional positions, thereby generating the motion-compensated image.

[0144] (Weight Prediction)

[0145] The weight prediction unit 10112 generates a predicted image of the PU by multiplying the input motion-compensated image predSamplesLX by weight coefficients. When one of the prediction list utilization flags (predFlagL0 or predFlagL1) is 1 (in the case of single prediction), the following formula is performed to make the input motion-compensated image predSamplesLX (LX is L0 or L1) consistent with the pixel depth bitDepth without using weight prediction.

[0146] predSamples[X][Y]=Clip3(0, (1<<bitDepth)-1, (predSamplesLX[X][Y]+offset1)>>shift1)

[0147] Here, shift1 = 14-bitDepth, offsetl = 1 << (shift1 - 1).

[0148] Furthermore, when both of the reference list flags (predFlagL0 and predFlagL1) are 1 (in the case of bipred prediction), without using weighted prediction, the following formula is used to average the input motion-compensated images predSamplesL0 and predSamplesL1 and make their average consistent with the number of pixels.

[0149] predSamples[X][Y]=C1ip3(0, (1<<bitDepth)-1, (predSamplesL0[X][Y]+predSamplesLl[X][Y]+offset2)>>shift2)

[0150] Here, shift2 = 15-bitDepth, offset2 = 1 << (shift2-1).

[0151] Furthermore, in the case of single prediction, when performing weighted prediction, the weight prediction unit 10112 derives the weight prediction coefficient w0 and the offset value o0 from the encoded data and performs the following calculation.

[0152] predSamples[X][Y]=Clip3(0, (1<<bitDepth)-1, ((predSamplesLX[X][Y]*w0+2^(log2WD-1))>>log2WD)+o0)

[0153] Here, log2WD is a variable representing the specified shift amount.

[0154] Furthermore, in the case of dual prediction BiPred, when performing weight prediction, the weight prediction unit 10112 derives the weight prediction coefficients w0, w1, o0, o1 from the encoded data and performs the following calculation.

[0155] predSamples[X][Y]=Clip3(0, (1<<bitDepth)-1, (predSamplesL0[X][Y]*w0+predSamplesL1[X][Y]*w1+((o0+o1+1)<<log2WD))>>(log2WD+1))

[0156] The prediction image generation unit 101 uses parameters input from the prediction parameter encoding unit to generate a prediction image P of PU based on the pixel values ​​of the reference block read from the reference image memory. The prediction image generated by the prediction image generation unit 101 is output to the subtraction unit 102 and the addition unit 106.

[0157] The subtraction unit 102 subtracts the signal value of the predicted image P of the PU, which is input from the predicted image generation unit 101, from the pixel value of the PU corresponding to the image T to generate a residual signal. The subtraction unit 102 outputs the generated residual signal to the transformation / quantization unit 103.

[0158] The transform / quantization unit 103 transforms the residual signal input from the subtraction unit 102 and calculates the transform coefficients. The transform / quantization unit 103 quantizes the calculated transform coefficients to obtain the quantized coefficients. The transform / quantization unit 103 outputs the obtained quantized coefficients to the entropy encoding unit 104 and the inverse quantization / inverse transform unit 105.

[0159] The entropy coding unit 104 receives quantization coefficients from the transform / quantization unit 103 and coding parameters from the prediction parameter coding unit 111. The input coding parameters include, for example, codes for the reference image index refIdxLX, the prediction vector index mvp_LX_idx, the difference vector mvdLX, the prediction mode predMode, and the merge index merge_idx.

[0160] The entropy coding unit 104 performs entropy coding on the input quantization coefficients and coding parameters to generate a coding stream Te, and outputs the generated coding stream Te to the outside.

[0161] The inverse quantization / inverse transform unit 105 performs inverse quantization on the quantization coefficients input from the transform / quantization unit 103 to obtain the transform coefficients. The inverse quantization / inverse transform unit 105 performs inverse frequency transform on the obtained transform coefficients to calculate the residual signal. The inverse quantization / inverse transform unit 105 outputs the calculated residual signal to the adder unit 106.

[0162] The adder 106 adds the signal value of the predicted image P of the PU input from the predicted image generation unit 101 to the signal value of the residual signal input from the inverse quantization / inverse transform unit 105 for each pixel to generate a decoded image. The adder 106 stores the generated decoded image in the reference image memory 109.

[0163] The loop filter 107 applies deblocking filtering, sample adaptive offset (SAO), and adaptive loop filtering (ALF) to the decoded image generated by the adder 106.

[0164] The prediction parameter memory 108 stores the prediction parameters generated by the encoding parameter determination unit 110 in a predetermined location according to each picture and CU of the encoded object.

[0165] The reference image memory 109 stores the decoded images generated by the loop filter 107 in predetermined locations according to each image and CU of the encoded object.

[0166] The encoding parameter determination unit 110 selects one set from a plurality of sets of encoding parameters. The encoding parameters are the prediction parameters described above, and the parameters generated in association with the prediction parameters as objects to be encoded. The prediction image generation unit 101 uses each set of these encoding parameters to generate a prediction image of the PU.

[0167] The encoding parameter determination unit 110 calculates a cost value representing the amount of information and the encoding error for each of the multiple sets. The cost value is, for example, the sum of the code amount and the squared error multiplied by a coefficient λ. The code amount is the amount of information in the encoded stream Te obtained by entropy encoding the quantization error and the encoding parameters. The squared error is the sum of the squares of the residual values ​​of the residual signal calculated in the subtraction unit 102 among the pixels. The coefficient λ is a real number greater than a predetermined zero. The encoding parameter determination unit 110 selects the set of encoding parameters with the smallest calculated cost value. Therefore, the entropy encoding unit 104 outputs the selected set of encoding parameters as the encoded stream Te to the outside, instead of outputting the set of unselected encoding parameters. The encoding parameter determination unit 110 stores the determined encoding parameters in the prediction parameter memory 108.

[0168] The prediction parameter encoding unit 111 derives the encoding form based on the parameters input from the encoding parameter determination unit 110 and outputs it to the entropy encoding unit 104. The derivation of the encoding form refers, for example, deriving a difference vector from a motion vector and a prediction vector. Furthermore, the prediction parameter encoding unit 111 derives the parameters required to generate the prediction image from the parameters input from the encoding parameter determination unit 110 and outputs them to the prediction image generation unit 101. The parameters required to generate the prediction image refer to, for example, motion vectors at the sub-block unit level.

[0169] The inter-frame prediction parameter encoding unit 112 derives inter-frame prediction parameters, such as difference vectors, based on the prediction parameters input from the encoding parameter determination unit 110. In the inter-frame prediction parameter encoding unit 112, the configuration of parameters required to generate the prediction image output to the prediction image generation unit 101 includes a portion similar to that of the inter-frame prediction parameter decoding unit 303 (see reference 1103). Figure 5 The structure of the inter-frame prediction parameter coding unit 112 is the same as that of the other unit. The structure of the inter-frame prediction parameter coding unit 112 will be described below.

[0170] The intra-prediction parameter coding unit 113 derives the form for encoding (e.g., mpm_idx, rem_intra_luma_pred_mode, etc.) based on the intra-prediction mode IntraPredMode input from the coding parameter determination unit 110.

[0171] (Main components of an image decoding device)

[0172] Figure 10 The diagram shows a block diagram illustrating the main components of the image decoding apparatus according to this embodiment. In this figure, for the sake of simplicity, details are omitted. Figure 10 The diagram shows a subset of the components included in the block diagram. Furthermore, for ease of explanation, [the diagram also shows components with...]. Figure 5 Components that have the same function as those shown in the drawings are labeled with the same reference numerals, and their descriptions are omitted.

[0173] like Figure 10 As shown, the image decoding device 31 includes: a decoding module 9, a CN information decoding unit 10, a prediction image generation unit 308, an inverse quantization / inverse transform unit 311, a reference image memory 306, an adder unit 312, a loop filter 305, a header decoding unit 19, and a CU decoding unit 20. The CU decoding unit 20 also includes a PU information decoding unit 12 and a TT information decoding unit 13 (segmentation information decoding unit, segmentation unit), and the TT information decoding unit 13 also includes a TU decoding unit 22.

[0174] First, the following settings are for each component. Figure 10 The functions of each component shown are the same as those in conventional technology.

[0175] (Decoding module)

[0176] The following is a brief description of the general operations of each module. Decoding module 9 performs decoding processing, decoding the syntax values ​​from binary. More specifically, decoding module 9 decodes the syntax values ​​encoded using CABAC isentropic encoding based on the encoded data supplied from the supply source and the syntax category, and returns the decoded syntax values ​​to the supply source.

[0177] In the example shown below, the sources of the encoded data and syntax categories are the CN information decoding unit 10, the CU decoding unit 20 (PU information decoding unit 12 and TT information decoding unit 13).

[0178] (Header Decoding Department)

[0179] The header decoding unit 19 decodes the VPS (video parameterset), SPS, PPS, and slice header of the encoded data input from the image encoding device 11.

[0180] (CN Information Decoding Department)

[0181] The CN information decoding unit 10 uses the decoding module 9 to perform decoding processing on the encoded data input from the image encoding device 11, including the coding tree unit (CTU) and the coding node (CN). Specifically, the CN information decoding unit 10 decodes the CTU information and CN information from the encoded data in the following order.

[0182] First, the CN information decoding unit 10 uses the decoding module 9 to decode the tree unit header CTUH from the CTU information included in the CTU. Next, the CN information decoding unit 10 decodes the QT segmentation flag from the CN information included in the CN, based on the context of the QT segmentation flag indicating whether QT segmentation is performed on the object CN. Then, the CN information decoding unit 10 recursively segments and decodes the object CN until the QT segmentation flag no longer indicates further segmentation. Finally, the tree unit footnote CTUF is decoded from the CTU information.

[0183] It should be noted that in this embodiment, the example of the decoded CN information being a QT splitting flag (qt_split_flag) is described, but the decoded CN information can also be a BT splitting flag (bt_split_flag) indicating whether the object CN is split by BT (binary tree), or a TT splitting flag (tt_split_flag) indicating whether the object CN is split by TT (triple tree).

[0184] The tree unit header (CTUH) and tree unit footnote (CTUF) include encoding parameters referenced by the image decoding device 31 to determine the decoding method for the object encoded tree unit. Furthermore, in addition to the QT segmentation flag, the CN information may also include the BT direction flag (BT segmentation mode), the object CN, and parameters applied in the lower-level encoded nodes, as described later.

[0185] (CU Decoding Department)

[0186] The CU decoding unit 20 includes a PU information decoding unit 12 and a TT information decoding unit 13, which decode the PUI information and TTI information of the lowest-level encoding node CN (i.e., CU).

[0187] (PU Information Decoding Department)

[0188] In the PU information decoding unit 12, the decoding module 9 is used to decode the PU information of each PU (merge flag, merge index, prediction vector index, reference image index, inter-pred flag, and difference vector, etc.).

[0189] (TT Information Decoding Department)

[0190] The TT information decoding unit 13 uses the decoding module 9 to decode the TT information of the transform tree TT (TU split flag SP_TU (split_transform_flag), TU residual flag CBP_TU (cbf_cb, cbf_cr, cbf_luma), etc., and TU).

[0191] Furthermore, the TT information decoding unit 13 includes a TU decoding unit 22. When the TU includes a residual, the TU decoding unit 22 decodes the QP update information (quantization correction value). It should be noted that the QP update information represents the difference between the quantization parameter QP and the predicted value, i.e., the quantization parameter prediction value qPpred. Additionally, the TU decoding unit 22 decodes the quantization prediction residual (residual_coding).

[0192] (The main components of the image decoding device, which is a feature of Embodiment 1)

[0193] The main components of the image decoding apparatus 31, which are features of this embodiment, will be described below.

[0194] In this embodiment, the CU decoding unit 20 does not segment the CU generated by the CN information decoding unit 10, but sends the PU included in the CU to the PU information decoding unit 12, and sends the TU included in the CU to the TT information decoding unit 13.

[0195] Furthermore, the PU information decoding unit 12 does not segment the PU received from the CU decoding unit 20, but decodes the aforementioned PU information.

[0196] On the other hand, the TT information decoding unit 13 segments the luminance TU included in the TU received from the CU decoding unit 20, but does not segment the chromatic difference TU. Furthermore, the TU decoding unit 22 included in the TT information decoding unit 13 decodes the quantization prediction residual related to the segmented luminance TU and the quantization prediction residual related to the unsegmented chromatic difference TU.

[0197] (Block partitioning method)

[0198] Reference Figure 11 The block segmentation method of the image decoding apparatus 31 based on this embodiment will be described in detail. Figure 11 This is a flowchart illustrating an example of a block segmentation method for the image decoding apparatus 31 based on this embodiment.

[0199] First, in step S0, the CN information decoding unit 10 decodes the QT segmentation flag, which indicates whether to perform QT ​​segmentation on the object CN, from the CN information included in the CN. Then, the CN information decoding unit 10 recursively segments and decodes the object CN until the QT segmentation flag no longer indicates further segmentation. If the QT segmentation flag no longer indicates further segmentation, the CN information decoding unit 10 sends the object CN as a CU (encoding unit in the claim) to the CU decoding unit 20.

[0200] Next, the CU decoding unit 20 sends the PU (prediction unit in the claim) included in the CU received from the CN information decoding unit 10 to the PU information decoding unit 12, and sends the TU (transformation unit in the claim) included in the CU to the TT information decoding unit 13 (step S1).

[0201] Next, the PU information decoding unit 12 decodes the PU information received from the CU decoding unit 20 (step S2). The PU information decoding unit 12 may further decode the information indicating whether to segment the PU and segment the PU.

[0202] Furthermore, the TT information decoding unit 13 divides the luminance TU included in the TU received from the CU decoding unit 20, but does not divide the chromatic difference TU (step S3). As an example of the method by which the TT information decoding unit 13 divides the luminance TU but does not divide the chromatic difference TU, a method using a division flag indicating that the luminance TU is divided but the chromatic difference TU is not divided, or that neither the luminance TU nor the chromatic difference TU is divided, can be cited.

[0203] Next, the TU decoding unit 22 decodes the quantization prediction residual of the luminance TU generated by the TT information decoding unit 13 through segmentation, and decodes the quantization prediction residual of the chrominance TU that is not segmented by the TT information decoding unit 13 (step S4).

[0204] It should be noted that, although Figure 11 Although not shown in the figure, the prediction image generation unit 308 (the prediction image generation unit in the claim) generates a prediction image based on the PU information decoded by the PU information decoding unit 12 in step S2. Furthermore, the inverse quantization / inverse transform unit 311 (the transform coefficient decoding unit in the claim) calculates the transform coefficients corresponding to luminance and chromatic difference based on the quantization prediction residuals decoded by the TU decoding unit 22 in step S4.

[0205] (Specific examples of block partitioning methods)

[0206] The following is for reference Figure 12 Specific examples of the block segmentation method of the image decoding apparatus 31 based on this embodiment will be described in detail using formulas or other methods. Figure 12 This diagram is used to illustrate the block segmentation method of this embodiment. For example... Figure 12 As shown, in the block segmentation method of this embodiment, the CU is not segmented, but only the luminance TU included in the CU is segmented. It should be noted that the following is a specific example using QT segmentation, but BT segmentation or TT segmentation can also be used.

[0207] First, in step S0 above, as shown in formula (1) below, the CN information decoding unit 10 decodes the QT split flag (split_cu_flag) for each encoded node CN. When split_cu_flag indicates further splitting (other than 0 in this case), the object CN is recursively split and decoded. Furthermore, as shown in formula (1) below, if the QT split flag (split_cu_flag) does not indicate further splitting (split_cu_flag = 0 in this case), the CN information decoding unit 10 processes the object CN as a CU through the CU decoding unit 20.

[0208]

[0209] In the above formula (1), coding_tree represents the encoding node CN, x and y represent the top-left coordinates of object CN, and w and h represent the size of object CN. Furthermore, >> indicates a rightward shift. In the example above, split_cu_flag is decoded. If the decoded split_cu_flag is 1, the lower-level CN ​​is further decoded (segmented). If split_cu_flag is 0, coding_unit(CU) is decoded.

[0210] It should be noted that, as shown in the following formula (2), the coding_unit (CU) in the above formula (1) includes prediction_unit (PU) and transformation_unit (TU).

[0211]

[0212] Furthermore, as shown in equation (3) below, the transform_unit(TU) in the above equation (2) includes transform_luma (luminance TU) and transform_chroma (chromaticity TU). Furthermore, as shown in equation (4) below, transform_unit(TU) can also be configured to include three residual_codings (quantization prediction residuals).

[0213]

[0214] In the above formula (4), cIdx represents the color component, cIdx=0 represents the residual of quantization prediction related to luminance (luma), cIdx=1 represents the residual of quantization prediction related to the first color difference component (Cb), and cIdx=2 represents the residual of quantization prediction related to the second color difference component (Cr).

[0215] Furthermore, as shown in equation (5) below, in step S3, the TT information decoding unit 13 refers to the splitting flag (split_luma_flag) indicating whether to split the luminance TU. If the splitting flag indicates splitting, the luminance TU is split, and in step S4, the TU decoding unit 22 decodes the quantization prediction residual corresponding to the luminance. If the splitting flag does not indicate splitting, the TT information decoding unit 13 does not split the luminance TU, and in step S4, the TU decoding unit 22 decodes (inverse quantization, inverse transform) the quantization prediction residual corresponding to the luminance.

[0216]

[0217]

[0218] On the other hand, for the color difference TU, in step S3, the TT information decoding unit 13 does not segment the color difference TU, and the TU decoding unit 22 decodes the quantization prediction residual corresponding to the color difference shown in the following formula (6) (inverse quantization, inverse transformation).

[0219]

[0220] (Summary of Implementation Method 1)

[0221] As described above, the image decoding apparatus 31 of this embodiment decodes the transformation coefficients corresponding to luminance after segmenting the transformation unit (TU) corresponding to luminance in the transformation tree, and decodes the transformation coefficients corresponding to chrominance without segmenting the transformation unit (TU) corresponding to chrominance.

[0222] Based on the above configuration, the transform unit corresponding to the color difference is not segmented, thus maintaining the concentration of residual energy corresponding to the color difference. Therefore, residual decoding can be performed efficiently, reducing the amount of code.

[0223] [Implementation Method 2]

[0224] If Embodiment 2 of the present invention is to be described based on the accompanying drawings, it will be as follows. It should be noted that the image decoding device 31 of Embodiment 1 is also used in this embodiment. Therefore, components that have the same function as the components of the image decoding device 31 described in Embodiment 1 are marked with the same reference numerals, and their descriptions are omitted.

[0225] (Block partitioning method)

[0226] Reference Figure 13 The block segmentation method of the image decoding apparatus 31 based on this embodiment will be described. Figure 13 This is a flowchart illustrating an example of the block segmentation method of the image decoding apparatus 31 based on this embodiment. It should be noted that steps identical to those described in Embodiment 1 are omitted from the description.

[0227] First, in step S10, the CN information decoding unit 10 decodes the segmentation flag indicating whether to segment the object CN from the CN information included in the CN. Then, the CN information decoding unit 10 recursively performs QT segmentation and decoding on the object CN until the segmentation flag no longer indicates further segmentation. For the CN information decoding unit 10, if the segmentation flag no longer indicates further segmentation, the CN information decoding unit 10 sends the partially segmented object CN as a CU to the CU decoding unit 20.

[0228] It should be noted that the segmentation in Implementation Method 2 includes both ordinary segmentation and partial segmentation. Ordinary segmentation is the same as the segmentation described in Implementation Method 1, and its description is omitted. Partial segmentation involves encoding / decoding the PU information and luminance quantization prediction residuals in each of the multiple segmented CUs. However, for the chrominance quantization prediction residuals, the multiple segmented CUs are aggregated and transformed, quantized, or inversely quantized and inversely transformed, and the encoded data is configured in the segmentation of the first CU. It should be noted that ordinary segmentation and partial segmentation are switched according to the conditions described later.

[0229] As the next step after step S10, the CU decoding unit 20 sends the PU included in the CU received from the CN information decoding unit 10 to the PU information decoding unit 12, and sends the TU included in the CU to the TU decoding unit 22 (step S11).

[0230] Next, the PU information decoding unit 12 decodes the PU information of each PU received from the CU decoding unit 20 (step S12).

[0231] Next, the TU decoding unit 22 decodes the quantization prediction residual of the luminance TU for each CU received from the CU decoding unit 20, and decodes the quantization prediction residual of the chromatic difference TU stored in the first CU received from the CU decoding unit 20 (step S13).

[0232] (Specific examples of block partitioning methods)

[0233] The following is for reference Figure 14 Specific examples of the block segmentation method of the image decoding apparatus 31 based on this embodiment will be described in detail using formulas or other methods. Figure 14 This diagram is used to illustrate the partial segmentation method of this embodiment. For example... Figure 14 As shown, in the partial segmentation method of this embodiment, PU and luminance TU are obtained by using the partially segmented CU, and color difference TU is obtained in the first CU.

[0234] First, in step S10 above, as shown in formula (7) below, the CN information decoding unit 10 decodes the split flag (split_cu_flag) in the object CN (coding_tree), recursively splits and decodes the object CN until the split flag split_cu_flag no longer signals further splitting (split_cu_flag = 1). If the split flag split_cu_flag no longer signals further splitting (split_cu_flag = 0), the CN information decoding unit 10 decodes the CU (coding_unit) of the object CN in the CU decoding unit 20.

[0235]

[0236] In Implementation 2, when the segmentation flag indicates a segmentation (split_cu_flag = 1), the process switches between normal segmentation and partial segmentation based on the condition described later (referred to as "normal CU segmentation determination" in formula (7)). It should be noted that the processing of normal segmentation is the same as in Implementation 1, and the explanation is omitted.

[0237] In the above formula (7), within the (...) of coding_tree(...), from left to right, the top left coordinates (x, y) and size (w, h) of CN, and the top left coordinates (xc, yc) and size (wc, hc) of the color difference quantization prediction residual block are represented.

[0238] It should be noted that the coordinates or dimensions of the color difference differ depending on the color difference format (4:2:0, 4:2:2, 4:4:4, etc.). However, for the sake of simplicity, the coordinates and dimensions of the color difference below will use values ​​converted to brightness. That is, values ​​transformed into the same coordinate system as brightness will be used. For example, when the coordinates of the color difference corresponding to the luminance coordinates (x, y) and the size (w, h) are set as (xk, yk) and (wk, hk), respectively, in the case of 4:2:0, (xk, yk) = (x>>1, y>>1) and (wk, hk) = (w>>1, h>>1); in the case of 4:2:2, (xk, yk) = (x>>1, y) and (wk, hk) = (w>>1, h); and in the case of 4:4:4, (xk, yk) = (x, y) and (wk, hk) = (w, h). However, the color difference coordinates (xk, yk) and color difference size (wk, hk) are not used in the following explanation. That is, the coordinates and size of the color difference are considered to be (xc, yc) = (x, y) and (wc, hc) = (w, h) regardless of the color difference format. Therefore, in order to obtain the actual coordinates (xk, yk) and dimensions (wk, hk) of the color difference block at the actual processing time, it is necessary to perform...

[0239] xk = xc >> 1, yk = yc >> 1, wk = wc >> 1, hk = hc >> 1 (in the case of 4:2:0)

[0240] xk = xc >> 1, wk = wc >> 1 (in the case of 4:2:2)

[0241] This transformation.

[0242] As shown in the above formula (7), the CN information decoding unit 10 performs partial segmentation when the segmentation flag (split_cu_flag) notifies further segmentation and the normal segmentation conditions are not met.

[0243] More specifically, in the above formula (7), when the CN information decoding unit 10 performs decoding of the object CN (coding_tree), it uses the same values ​​as the upper left coordinates (xc, yc) and the size (wc, hc) of the upper-level CN ​​(as the values ​​passed as independent variables) to process the upper left coordinates (xc, yc) and the size (wc, hc) of the color difference quantization prediction residual block.

[0244] Furthermore, as shown in the following formula (8), the coding_unit(x, y, w, h, xc, yc, wc, hc) as CN in the above formula (7) includes the prediction_unit(x, y, w, h) as PU and the transform_unit(x, y, w, h, xc, yc, wc, hc) as TU.

[0245]

[0246]

[0247] Then, as shown in the following formula (9), in step S13, the TU decoding unit 22 decodes (inverse quantization, inverse transformation) the quantization prediction residual (residual_coding(x, y, w, h, cIdx=0)) related to luminance, and decodes (inverse quantization, inverse transformation) the quantization prediction residual (residual_coding(xc, yc, wc, hc, cIdx=1) and residual_coding(xc, yc, wc, hc, cIdx=2)) related to chrominance (step S13).

[0248]

[0249] In the above formula (9), the TU decoding unit 22 executes the decision formula (if(x==xc&&y==yc)). According to this decision formula, when the coordinates (x, y) of the object CU are the same as the coordinates of the color difference object TU, the TU decoding unit 22 performs the decoding processing residual_coding of the quantization prediction residual of the color difference Cb and Cr of the CU located in the upper left.

[0250] As shown in the normal segmentation of equation (7), in the case of normal segmentation, the coordinates (x, y) of object CU and the coordinates (xc, yc) of object TU of color difference are substituted with the same value, and the decoding processing of the quantization prediction residuals of color difference Cb and Cr is performed. However, as shown in the partial segmentation of equation (7), in the case of partial segmentation, only in the first CU of the initial segmentation of the four CUs, the coordinates (x, y) of object CU and the coordinates (xc, yc) of object TU of color difference are substituted with the same value, and the decoding processing of the quantization prediction residuals of color difference Cb and Cr is performed only in the first CU.

[0251] Reference Figure 15 and Figure 16The decision expression (if(x==xc&&y==yc)) in the above formula (9) will be explained. Both of these are examples of partial partitioning starting in block B. That is, examples of partial partitioning in the hierarchy below block B. Figure 15 This represents CN (block B) that is not partially partitioned and CN (blocks PB1 to PB4) that further partially partition block B. It should be noted that... Figure 15 Blocks PB1 to PB4 are not further divided.

[0252] Figure 16 Is with Figure 15 Different examples, representing in Figure 15 The beginning CN of the partially divided blocks PB1 to PB4 shown in the diagram, i.e., block PB1, contains blocks PB1 that are further partially divided to form blocks PB. 11 ~PB 14 An example. It should be noted that... Figure 16 PB block 11 ~PB 14 They will not be further divided. Furthermore, PB2 through PB4 will also not be further divided.

[0253] like Figure 15 As shown, the coordinates and dimensions of the unpartitioned block B (non-shared block) are (x, y, w, h, xc, yc, wc, hc = 0, 0, 8, 8, 0, 0, 8, 8), satisfying x = xc && y = yc. Furthermore, the partially partitioned starting block PB1 (shared block) is (x, y, w, h, xc, yc, wc, hc = 0, 0, 4, 4, 0, 0, 8, 8), satisfying x = xc && y = yc. Therefore, in step S13, the TU decoding unit 22 decodes the quantization prediction residual related to the overall color difference of block B to obtain the result of the determination based on the determination formula in the above formula (9). That is, in the initial block PB1 of block B, the quantization prediction residuals residual_coding(xc, yc, wc, hc) = (0, 0, 8, 8) of block B are decoded using the following methods: residual_coding(xc, yc, wc, hc, cIdx = 1) and residual_coding(xc, yc, wc, hc, cIdx = 2). However, in blocks PB2 to PB4, which are outside the partially segmented initial block, the condition x == xc && y == yc is not satisfied, therefore the quantization prediction residuals related to chromatic aberration are not decoded.

[0254] In addition, such as Figure 16 As shown, block PB 11 ~PB 14 The beginning of CU (leaf) is block PB. 11The coordinates and dimensions are (x, y, w, h, xc, yc, wc, hc = 0, 0, 2, 2, 0, 0, 8, 8), satisfying x = xc && y = yc. Therefore, in step S13, the TU decoding unit 22, in block PB... 11 The quantization prediction residual related to the overall color difference of the block is decoded to serve as the result of the judgment based on the above formula (9). However, in the block PB other than the first block after partial segmentation... 12 ~PB 14 In PB2 to PB4, x==xc&&y==yc does not satisfy, therefore the quantization prediction residuals related to color difference are not decoded.

[0255] (Modification 1 of Implementation Method 2)

[0256] The following is for reference Figure 17 A variation of embodiment 2, example 1, will be described. Figure 17 This diagram is used to illustrate the block segmentation method of this modified example. For example... Figure 17 As shown, in the block segmentation method of this variant, the luminance TU is obtained by using the partially segmented CU, and the PU and chromatic difference TU are obtained in the first CU.

[0257] In step S12 of the block segmentation method described above, the PU information decoding unit 12 decodes the PU information related to each PU for each CU. In step S13, the TU decoding unit 22 decodes the quantization prediction residual of the luminance TU and decodes the quantization prediction residual of the chrominance TU for the first CU. However, in this modified example, the PU information decoding unit 12 decodes the PU information included in a specific CU. Therefore, the PU information related to the PU is decoded for the first CU, the TU decoding unit 22 decodes the quantization prediction residual of the chrominance TU, and the quantization prediction residual of the luminance TU is decoded for each CU.

[0258] For example, as shown in the following formula (10), the PU information decoding unit 12 determines whether the CU is the starting CU and only decodes the PU information included in the starting CU.

[0259]

[0260] As shown in formula (10) above, the PU information decoding unit 12 decodes the PU information included in the CU (starting CU) that satisfies the condition (x==xc&&y==yc). It should be noted that the method for TU is the same as that described in the above item (specific example of block segmentation method), so the description is omitted. Thus, the PU information decoding unit 12 or the TU decoding unit 22 can decode the PU and chromatic difference TU in the starting CU, and can decode the luminance TU for each CU.

[0261] (Modification 2 of Implementation Method 2)

[0262] Hereinafter, a detailed description of a variation of embodiment 2 will be provided. In this variation, the CN information decoding unit 10 uses a flag (signaling_flag) representing a temporary variable indicating whether to implement (encoding or decoding the quantization prediction residual of the color difference per CU) partial segmentation in step S10 as shown in the following formula (11).

[0263]

[0264]

[0265] Then, the CN information decoding unit 10 sends the value of signaling_flag and CU to the CU decoding unit 20. Then, the coding_unit(x, y, w, h, xc, yc, wc, hc, signaling_flag) in the above formula (11) is as shown in the following formula (12), including prediction_unit(x, y, w, h) and transform_unit(x, y, w, h, xc, yc, wc, hc, signaling_flag).

[0266]

[0267] Then, as shown in the following formula (13), in step S13, the TU decoding unit 22 decodes the quantization prediction residual (residual_coding(x, y, w, h, cIdx=0)) related to luminance and the quantization prediction residual (residual_coding(xc, yc, we, hc, cIdx=1) and residual_coding(xc, yc, wc, hc, c=2)) related to chromatic difference (step S13).

[0268]

[0269]

[0270] In the above formula (13), the TU decoding unit 22 executes the decision formula (if(signaling_flag)). According to this decision formula, if signaling_flag represents 1, the TU decoding unit 22 decodes the quantization prediction residual related to color difference; if signaling_flag represents 0, it does not decode the quantization prediction residual related to color difference.

[0271] Furthermore, the signaling_flag of this variant example can also be applied to the variant example 1 described above. In this case, in step S12, the PU information decoding unit 12 executes the determination formula (if(signaling_flag)) shown in the following formula (14). If signaling_flag represents 1, the PU information is decoded; if signaling_flag represents 0, the PU information is not decoded. It should be noted that the TU decoding unit 22 in step S13 performs the same operation as in the variant example 1 described above, and the explanation is omitted.

[0272]

[0273] As mentioned above, by using signaling_flag, it is possible to determine whether to decode the quantization prediction residual or PU information without determining the coordinate position.

[0274] (Modification 3 of Implementation Method 2)

[0275] Hereinafter, a detailed description of variation 3 of embodiment 2 will be provided. In this variation, the CN information decoding unit 10 determines whether to perform normal segmentation or partial segmentation on the object CN in step S10 described above using the determination method described below. This determination method involves the conditions expressed as "normal CU segmentation determination" in formulas (7) and (11). It should be noted that in step S10, the CN information decoding unit 10 performs only partial segmentation after performing one partial segmentation. Furthermore, the object block that the CN information decoding unit 10 performs partial segmentation on is preferably a relatively small block. Hereinafter, the determination methods based on at least one of these conditions (1) to (3) described below will be described.

[0276] Hereinafter, the flag indicating whether to perform normal splitting will be called `split_mode_flag` (luma_chroma_common_split_mode_flag). A `split_mode_flag` of 1 indicates normal splitting, while a `split_mode_flag` of 0 indicates partial splitting. Alternatively, `partial_split_mode_flag`, which has the opposite meaning, can be used. When `partial_split_mode_flag` is 0, normal splitting is performed on the object CN; when the decoded `partial_split_mode_flag` is 1, partial splitting is performed. It should be noted that in normal splitting, there is a common structure in both luminance and chrominance, so `split_mode_flag` can also be called `luma_chroma_common_split_mode_flag`. Furthermore, in partial splitting, there are different structures in luminance and chrominance, and blocks are further merged in the chrominance section; therefore, `partial_split_mode_flag` can also be called `chroma_shared_split_mode_flag`.

[0277] After a partial split, at least one of {x, y, w, h} and {xc, yc, wc, hc} is different. Therefore, if only the partial split is performed after the partial split, the following formula (15) holds. That is, the value of split_mode_flag(partial_split_mode_flag) can be set as shown in the following formula (15).

[0278]

[0279] It should be noted that w! = we is equivalent to w < wc, and h! = hc is equivalent to h < hc.

[0280] (1) Branch formation P1: Size dependency determination

[0281] In this determination method, the CN information decoding unit 10 determines whether the block size of the object CN is large in step S10. The size of the block size determined by the CN information decoding unit 10 can be the sum of width w and height h, the product of width w and height h, or the sum of the logarithm of width log2(w) and the logarithm of height log2(h), etc., but in the following explanation, the determination of whether w+h>M is satisfied will be used. Specifically, it is determined whether w+h>M is satisfied (M is a threshold representing the size of the object CN). Then, if the object CN is large (if w+h>M is satisfied), the CN information decoding unit 10 performs normal CU segmentation, and if not (if the object CN does not satisfy w+h>M), it performs partial segmentation. Examples of M are M=16 or M=24, etc. That is, partial_split_mode_flag can be derived from the following formula (16).

[0282]

[0283] (2) Branch formation P2: Flag determination

[0284] In this determination method, in step S10, the CN information decoding unit 10 decodes the split_mode_flag (luma_chroma_common_split_mode_flag) indicating whether to perform normal splitting or partial splitting from the encoded data. Then, the CN information decoding unit 10 performs normal splitting on the object CN when the decoded split_mode_flag indicates 1, and performs partial splitting when the decoded split_mode_flag indicates 0. Alternatively, the partial_split_mode_flag (chroma_shared_split_mode_flag), which has the opposite meaning to split_mode_flag, can also be used. In this case, in step S10, the CN information decoding unit 10 performs normal splitting on the object CN when the decoded partial_split_mode_flag indicates 0, and performs partial splitting when the decoded partial_split_mode_flag indicates 1. Since there is no need to decode the flag when a partial splitting has been performed, the flag is decoded according to the conditions of the following formula (17).

[0285]

[0286] (3) Branching P3: Size-dependent flag determination

[0287] In this determination method, in step S10, the CN information decoding unit 10 first determines whether the block size of the object CN is large. The block size determined by the CN information decoding unit 10 can be the sum of width w and height h, the product of width w and height h, or the sum of the logarithm of width log2(w) and the logarithm of height log2(h), etc. However, in the following explanation, the sum of width w and height h, i.e., whether w+h>M is satisfied, will be used. Specifically, when the object CN is large (w+h>M is satisfied), the CN information decoding unit 10 performs normal segmentation. When the object CN is small (w+h>M is not satisfied) and x! = xc||y! = yc||w<wc||h<hc, partial segmentation has already been performed, therefore partial segmentation is performed without decoding the flag. In other cases, the CN information decoding unit 10 decodes the aforementioned split_mode_flag or partial_split_mode_flag and performs normal splitting or partial splitting based on the value of the flag. For example, M = 32, etc.

[0288] That is, to perform the following actions.

[0289]

[0290] It should be noted that, as a result, the CN information decoding unit 10 decodes the flag only without performing partial segmentation, therefore

[0291]

[0292] The above formula (19) decodes the flag. If the flag is not decoded, the flag can be derived by the following formula (20).

[0293]

[0294] The following formula (21) represents a specific example of the determination method of (3) above.

[0295]

[0296] Without decoding the partial_split_mode_flag, the partial_split_mode_flag is derived using formula (20). As shown in formula (21) above, the CN information decoding unit 10 determines whether the block size of the target CN is large (w+h>M) until the split flag (split_cu_flag) does not indicate further normal splitting. Then, if the block size of the target CN is large (w+h>M), the CN information decoding unit 10 performs normal splitting. If the CN information decoding unit 10 is partially split (x!=xc||y!=yc||w<wc||h<hc), it performs partial splitting without decoding the flag. Otherwise, it decodes the split_mode_flag or partial_split_mode_flag and performs normal splitting or partial splitting based on the value of the flag.

[0297] (Modification 4 of Implementation Method 2)

[0298] The above variations illustrate QT segmentation of object CN. This variation illustrates BT segmentation of object CN.

[0299] The CN information decoding unit 10 executes the process shown in formula (21) of the above-described variant 3 as shown in formula (22) below. Then, when split_cu_flag represents 0 and no partial splitting is performed in QT splitting, the CN information decoding unit 10 decodes the flag indicating whether to perform BT splitting on the object CN, namely split_bt_flag.

[0300]

[0301]

[0302]

[0303] As shown in equation (22) above, after decoding split_bt_flag, the CN information decoding unit 10 decodes split_dir_flag, which serves as the BT splitting direction flag, when split_bt_flag represents 1. Furthermore, when the block size of the object CN is large (satisfying (w+h)>M), the CN information decoding unit 10 performs normal splitting with partial_bt_split_mode_flag=0. In other cases where the CN information decoding unit 10 is partially split (x!=xc||y!=yc||w<wc||h<hc), it performs partial splitting with partial_bt_split_mode_flag=1. In all other cases, the CN information decoding unit 10 decodes the aforementioned partial_bt_split_mode_flag. Then, when partial_bt_split_mode_flag is 0 and split_dir_flag is 1, the CN information decoding unit 10 performs normal segmentation of the object CN with the two segmented CNs arranged vertically. Furthermore, when partial_bt_split_mode_flag is 0 and split_dir_flag is 0, the CN information decoding unit 10 performs normal segmentation of the object CN with the two segmented CNs arranged horizontally. Furthermore, when partial_bt_split_mode_flag is 1 and split_dir_flag is 1, the CN information decoding unit 10 performs partial segmentation of the object CN with the two segmented CNs arranged vertically. Furthermore, when partial_bt_split_mode_flag is 1 and split_dir_flag is 0, the CN information decoding unit 10 performs partial segmentation of the object CN with the two segmented CNs arranged horizontally.

[0304] It should be noted that the QT tree coding_tree and the BT tree coding_bt_tree can also be separated and constructed using the following formula (23).

[0305]

[0306]

[0307] (Summary of Implementation Method 2)

[0308] As described above, the image decoding apparatus 31 of this embodiment is the same as that of Embodiment 1. In the transform tree included in the encoding unit (CU), the transform coefficients related to luminance are decoded after segmenting the transform units (TU) corresponding to luminance, but the transform coefficients related to chrominance are decoded without segmenting the transform units (TU) corresponding to chrominance. Furthermore, in the prediction tree included in the encoding unit (CU), the image decoding apparatus 31 generates a prediction image related to luminance without segmenting the prediction units (PU) corresponding to luminance, and generates a prediction image related to chrominance without segmenting the prediction units (PU) corresponding to chrominance in the prediction tree.

[0309] Based on the above configuration, only the transformation unit corresponding to luminance is segmented, thus maintaining the concentration of residual energy corresponding to chromatic difference and reducing code overhead.

[0310] Furthermore, the image decoding apparatus 31 of this embodiment can generate a prediction image in the prediction tree related to the prediction unit (PU) corresponding to luminance by means of the segmentation coding unit (CU), and generate a prediction image in the prediction tree related to the prediction unit (PU) corresponding to chromatic difference by means of the segmentation coding unit (CU).

[0311] Based on the above configuration, the transform unit (TU) corresponding to luminance can be uniformly segmented by dividing the coding unit (CU). Therefore, the block segmentation process can be simplified.

[0312] Furthermore, the image decoding apparatus 31 of this embodiment can generate a luminance-related prediction image without segmenting the prediction unit corresponding to luminance in the prediction tree included in the encoding unit (CU), and generate a chromatic difference-related prediction image without segmenting the prediction unit corresponding to chromatic difference.

[0313] Based on the above structure, the prediction unit is not split, thus reducing the amount of code.

[0314] [Implementation Method 3]

[0315] If Embodiment 3 of the present invention is to be described based on the accompanying drawings, it will be as follows. It should be noted that the image decoding device 31 of Embodiment 1 is also used in this embodiment. Therefore, components that have the same function as the components of the image decoding device 31 described in Embodiment 1 are marked with the same reference numerals, and their descriptions are omitted.

[0316] In Embodiments 1 and 2 described above, the processing related to the PU is performed as follows: First, the PU information decoding unit 12 decodes the PU information of each PU. Next, the inter-frame prediction parameter decoding unit 303 decodes the inter-frame prediction parameters based on the PU information and with reference to the prediction parameters stored in the prediction parameter memory 307, and outputs the decoded inter-frame prediction parameters to the prediction image generation unit 308. On the other hand, the intra-frame prediction parameter decoding unit 304 decodes the intra-frame prediction parameters based on the aforementioned PU information and with reference to the prediction parameters stored in the prediction parameter memory 307. Then, the prediction image generation unit 308 generates a prediction image of the PU or sub-block using the prediction parameters and read-out reference images (reference image blocks) under the prediction mode indicated by the prediction mode predMode included in the PU information. More specifically, when the prediction mode predMode indicates an inter-frame prediction mode, the inter-frame prediction image generation unit 309 generates a prediction image of the PU or sub-block by inter-frame prediction using the inter-frame prediction parameters and read-out reference images (reference image blocks). When the prediction mode predMode indicates the intra-prediction mode, the intra-prediction image generation unit 310 uses the intra-prediction parameters and the read-out reference image to generate the prediction image of the PU through intra-prediction.

[0317] In the aforementioned PU-related processing, the following problems exist. First, in the generation of the predicted image, sometimes the same prediction for chromatic difference as the prediction for luminance cannot function. Furthermore, as in Embodiments 1 and 2, when techniques using different trees for luminance and chromatic difference are applied to inter-frame prediction, not only does the amount of code for the tree (block construction) double, but the amount of code for motion information also increases (generally, motion information is mostly the same for luminance and chromatic difference, so a single motion information is sufficient). Moreover, there is a problem that skipping, representing the absence of residuals, cannot be used for either luminance or chromatic difference. To solve these problems, the image decoding apparatus 31 of this embodiment uses CCLM (Cross-component Linear Model) prediction, as detailed below, to generate a chromatic difference-related predicted image.

[0318] (The main components of the image decoding device, which is a feature of Embodiment 3)

[0319] The following is for reference Figure 18 The main components of the image decoding apparatus 31, which is a feature of this embodiment, will be described in detail. Figure 18 (a) is a block diagram showing an example of the configuration of the CCLM prediction unit 3104 included in the intra-frame prediction image generation unit 310. The CCLM prediction unit 3104 includes a CCLM parameter derivation unit 31041 and a CCLM prediction filter unit 31042. Figure 18(b) is a block diagram showing an example of the configuration of the CCLM predictive filter unit 31042.

[0320] The CCLM parameter derivation unit 31041 derives CCLM prediction parameters by taking the pixel values ​​(x[i], y[i], i = 0, ..., N) of the processed image adjacent to the target block as input signals. The CCLM parameter derivation unit 31041 outputs the derived CCLM prediction parameters (a, b) to the CCLM prediction filter unit 31042. It should be noted that the pixel value x[i] represents the value of the luminance pixel (Luma) in the region adjacent to the target block, and the pixel value y[i] represents the value of the chrominance pixel (Cb, Cr) in the region adjacent to the target block.

[0321] CCLM predictive filter unit 31042 performs CCLM prediction on the luminance decoding image refSamplesX[][] of the object block, which is the input signal, to generate a chromatic difference prediction image predSamplesY[][] of the object block (where Y represents Cb and Cr). That is, CCLM predictive filter unit 31042 generates the prediction image predSamplesY[][] from the reference image refSamplesX[][], which is the input signal.

[0322] Figure 18 (b) indicates the configuration of the CCLM prediction filter unit 31042. The CCLM prediction filter unit 31042 includes a linear prediction unit 310421. The linear prediction unit 310421 takes the reference image refSamplesX[][] as the input signal and outputs the prediction image predSamplesY[][] using CCLM prediction parameters (a, b).

[0323] More specifically, the linear prediction unit 310421 takes the luminance-related reference image refSamplesLuma (the luminance-related prediction image in the claim) as an input signal, and derives the prediction images of the color difference signals Cb and Cr by using the following formulas (24) and (25) with prediction parameters (a, b) and (a', b') respectively, and outputs them as the color difference-related prediction images predSamplesCb and predSamplesCr.

[0324] predSamplesCb=(a*refSamplesLuma)>>shift+b...Equation (24)

[0325] predSamplesCb=(a*refSamplesLuma)>>shift+b...Equation (24)

[0326] Here, (a, b) are the CCLM prediction parameters for the color difference Cb (where y[i] is Cb), and (a', b') are the CCLM prediction parameters for the color difference Cr (where y[i] is Cr). (a, b) and (a', b') will be denoted as (a, b) in the following text.

[0327] (Predictive Image Export Method)

[0328] Reference Figure 19 The predicted image derivation method of the image decoding apparatus 31 based on this embodiment will be described in detail. Figure 19 This is a flowchart illustrating an example of the predicted image derivation method of the image decoding apparatus 31 based on this embodiment. It should be noted that the predicted image derivation method in this embodiment is a step following the block segmentation method in Embodiment 1 or the block segmentation method in Embodiment 2. Therefore, the description of steps S0 to S5 and steps S10 to S13 described above is omitted.

[0329] First, the inter-frame prediction image generation unit 309 uses the inter-frame prediction parameters decoded by the inter-frame prediction parameter decoding unit 303 and the reference image (reference image block) read from the reference image memory 306 to generate a brightness-related prediction image through inter-frame prediction. Then, the residual images of the decoded brightness blocks are added together to generate a decoded image of the brightness blocks (step S20).

[0330] Next, the CCLM parameter derivation unit 31041 uses the pixel values ​​(x[i], y[i], i = 0, ..., N) of the processed image adjacent to the object block as input signals to derive CCLM prediction parameters (a, b) (step S21). Here, x[i] represents the luminance pixel value of the region adjacent to the object block, and y[i] represents the color difference pixel value of the region adjacent to the object block.

[0331] Next, the linear prediction unit 310421 takes the luminance-related reference image refSamplesLuma (luminance prediction image or luminance decoding image of the object block) as the input signal, and derives the prediction images of the color difference signals Cb and Cr by using the above formulas (24) and (25) of the prediction parameters (a, b), and outputs them as the color difference-related prediction images predSamplesCb and predSamplesCr (step S22).

[0332] Typically, CCLM prediction is applied in intra-frame prediction when the chromatic aberration image of an object block has a higher correlation with the luminance image of an object block located at the same position compared to adjacent chromatic aberration images. In this embodiment, by combining CCLM prediction and inter-frame prediction (combined prediction) (referred to as CCLM combined mode), a more accurate predicted image can be generated under inter-frame prediction even when the chromatic aberration image and luminance image of the object block have a high correlation, thereby improving coding efficiency by reducing prediction residuals.

[0333] This section describes a CCLM merging mode, one of the merging candidates described in Implementation 1, which uses the prediction parameters of adjacent blocks for luminance prediction and CCLM prediction for chromatic difference prediction. The list of merging candidates for this case, mergeCand[], is shown in the following formula. A1CCLM represents a CCLM merging candidate. A1, B1, B0, A0, and B2 in formula (26) represent... Figure 24 The prediction pattern for the positions of adjacent blocks of the object block shown.

[0334] mergeCand[]={A1, B1, B0, A0, B2, Col, A1CCLM, Combined, Zero}... Equation (26)

[0335] To explain the above situation in more detail, as a method for creating the candidate list for merging, candidates are first added in the order shown in the following formula (27). Here, availableFlagX is a flag that is true (TRUE) if the adjacent block X is available, and false (FALSE) if it is not available. Figure 24 The adjacent block X is shown. Next, candidates are added to the merge candidate list in the order of A1CCLM, Combined merge, and Zero merge. Here, A1, B1, B0, A0, B2, and Col represent the modes that use the prediction parameters of blocks A1, B1, B0, A0, B2, or Col for luminance prediction and chromatic difference prediction, respectively.

[0336]

[0337]

[0338] When A1CCLM is selected as the merging candidate (merge_idx indicates A1CCLM), for luminance, the prediction parameters of A1 at the beginning of the merging candidate list are used to generate a prediction image, and the separately decoded prediction residual images are added to generate the decoded image. For chromatic aberration, either the luminance prediction image or the decoded image is used to generate a prediction image through CCLM prediction, and the separately decoded prediction residual images are added to generate the decoded image. Further deblocking filtering can be applied to the decoded image, but it is preferable to use the decoded image before deblocking filtering when CCLM prediction uses the decoded image.

[0339] It should be noted that the merging CCLM mode is not limited to A1CCLM, which uses the prediction parameters of A1 in the generation of the luminance prediction image. It can also be B1CCLM, B0CCLM, A0CCLM, B2CCLM, ColCCLM, etc., which use the prediction parameters of B1, B0, A0, B2, and Col in the generation of the luminance prediction image. For example, the luminance prediction image can be generated by using the initially available merging candidates in the order of A1, B1, B0, A0, B2, Col, etc. For chromatic aberration, the predicted image can be generated by using the decoded image of that luminance through CCLM prediction.

[0340] (Additional configuration in this embodiment)

[0341] Hereinafter, as a variation of embodiment 3, refer to Figure 20 An example is provided to illustrate the CCLM merge mode notification using a flag different from the merge index. Figure 20 This is a flowchart illustrating the prediction mode determination method in this embodiment.

[0342] First, the prediction parameter decoding unit 302 instructs the PU information decoding unit 12 to decode the merge flag merge_flag, and extracts the merge flag merge_flag (step S30).

[0343] Next, the prediction parameter decoding unit 302 determines whether the extracted merge flag represents 1 (step S31). If the merge flag represents 1 (yes in step S31), proceed to step S32; if the merge flag represents 0 (no in step S31), proceed to step S33.

[0344] In step S32, the prediction parameter decoding unit 302 instructs the PU information decoding unit 12 to decode the merge index merge_idx and extracts the merge index merge_idx as the prediction parameter for merge prediction. In addition, the PU information decoding unit 12 is instructed to decode merge_cclm_flag and extract merge_cclm_flag, where merge_cclm_flag indicates whether to export the prediction image related to color difference through CCLM mode.

[0345] In step S33, the prediction parameter decoding unit 302 instructs the PU information decoding unit 12 to decode the AMVP prediction parameters and extract the AMVP prediction parameters (the following steps are equivalent to the AMVP prediction mode).

[0346] As the next step after step S32, the prediction parameter decoding unit 302 determines whether the extracted merge_cclm_flag represents 1 (step S34). If merge_cclm_flag represents 1 (yes in step S34), proceed to step S35; if merge_cclm_flag represents 0 (yes in step S34), proceed to step S36.

[0347] In step S35, the prediction parameter decoding unit 302 selects the merge candidate shown by the extracted merge index merge_idx as the inter-frame prediction parameter of the object PU. The merge candidate here is any one of the merge candidates mergeCand[] shown in the following formula (28).

[0348] mergeCand[]={A1, B1, BO, A0, B2, Col, Combined, Zero}... Equation (28)

[0349] The next step after step S35 is steps S20, S21, and S22 as described above. Specifically, based on the selected merging candidates (merging modes), a brightness-related prediction image is generated through inter-frame prediction (step S20), and CCLM prediction parameters (a, b) are derived (step S21). The color difference signals Cb and Cr of the prediction image using prediction parameters (a, b) are then derived (step S22).

[0350] Furthermore, in step S36, the prediction parameter decoding unit 302 selects the merge candidate shown by the extracted merge index merge_idx as the inter-frame prediction parameter of the object PU. Here, the merge candidate is any one of the merge candidates mergeCand[] shown in the above formula (28).

[0351] It should be noted that the process after step S36 (not shown) is equivalent to the usual merging mode. The inter-frame prediction image generation unit 309 uses the inter-frame prediction parameters derived by the inter-frame prediction parameter decoding unit 303 and the reference image (reference image block) read from the reference image memory 306 to generate a brightness-related prediction image and a color difference-related prediction image through inter-frame prediction.

[0352] In this variation, unlike the merge index, a flag (merge_CCLM_flag) is used to indicate whether CCLM prediction is used for color difference. In Implementation 3, when CCLM prediction is used for color difference, only block A1 (a limited merge candidate) can be selected for luminance. However, in this variation, even when CCLM prediction is used for color difference, all candidates available in normal merge prediction can be selected for luminance. That is, it has the advantage of improving the prediction efficiency of color difference without sacrificing the prediction efficiency of luminance.

[0353] (Combined with Implementation Method 1 or Implementation Method 2)

[0354] The following is for reference. Figures 21-23 The combination of the predicted image export method of this embodiment with the block segmentation method in embodiment 1 or embodiment 2 will be described. Figure 21 This diagram is used to illustrate the combination of the block segmentation method in Embodiment 1 and the predicted image derivation method in Embodiment 3. Figure 22 This diagram is used to illustrate the combination of the block segmentation method in Embodiment 2 and the predicted image derivation method in Embodiment 3. Figure 23 This diagram is used to illustrate a combination of a modified example of the block segmentation method in Embodiment 2 and a predicted image derivation method in Embodiment 3.

[0355] like Figure 21 As shown, the addition unit 312 adds (1) and (2) below to generate a decoded image of the luminance block.

[0356] (1) The prediction image generation unit 308 generates a prediction image of the brightness block based on the PU information of the PU decoded by the PU information decoding unit 12 in step S2.

[0357] (2) The inverse quantization / inverse transform unit 311 generates a residual image of the brightness block based on the quantization prediction residual of the TU decoded by the TU decoding unit 22 in step S4.

[0358] Then, in step S22, the linear prediction unit 310421 uses the predicted or decoded image of the luminance block as an input signal to derive the predicted image of the chrominance block through CCLM prediction. Figure 21The image shown is an example of using a decoded image as an input signal to predict the chromatic aberration block using CCLM.

[0359] Then, the addition unit 312 adds the predicted image of the color difference block and the residual image of the color difference block generated by the inverse quantization / inverse transform unit 311 based on the quantization prediction residual decoded by the TU decoding unit 22 in step S4, to generate the decoded image of the color difference block.

[0360] In addition, such as Figure 22 As shown, the addition unit 312 adds (3) and (4) below to generate a decoded image of the luminance block.

[0361] (3) The prediction image generation unit 308 generates a prediction image of the brightness block based on the PU information of the PU included in the CU that is partially segmented by the PU information decoding unit 12 in step S12.

[0362] (4) The inverse quantization / inverse transform unit 311 generates a residual image of the brightness block based on the quantization prediction residual decoded by the TU decoding unit 22 in step S13 with reference to the TU included in the partially segmented CU.

[0363] Then, in step S22, the linear prediction unit 310421 uses the predicted or decoded image of the luminance block as an input signal to derive the predicted image of the chrominance block per PU through CCLM prediction. Figure 22 The example shown illustrates using the decoded image as an input signal to derive a predicted image of the color difference block via CCLM prediction. Then, the adder 312 adds the predicted image of the color difference block to the residual image of the color difference block generated by the inverse quantization / inverse transform unit 311 based on the quantization prediction residual decoded by the TU decoding unit 22 in step S13 with reference to the unsegmented TU, thereby generating a decoded image of the color difference block.

[0364] In addition, such as Figure 23 As shown, the adder 312 adds (5) and (6) together to generate a decoded image of the luminance block.

[0365] (5) As described in Variation 1 of Embodiment 2, the prediction image generation unit 308 generates a prediction image of the luminance block based on the PU information of the PU that has not been segmented by the PU information decoding unit 12.

[0366] (6) The inverse quantization / inverse transform unit 311 generates a residual image of the luminance block based on the quantization prediction residual decoded by the TU decoding unit 22 in step S13 with reference to the TU included in the partially segmented CU.

[0367] Then, in step S22, the linear prediction unit 310421 uses the predicted or decoded image of the luminance block as an input signal and derives the predicted image of the chrominance block through CCLM prediction. Then, the addition unit 312 adds the predicted image of the chrominance block and the residual image of the chrominance block generated by the inverse quantization / inverse transform unit 311 based on the quantization prediction residual decoded by the TU decoding unit 22 in step S13 with reference to the unsegmented TU, to generate the decoded image of the chrominance block.

[0368] [Summary of Implementation Method 3]

[0369] As described above, the image decoding apparatus 31 of this embodiment generates a prediction image related to the brightness of the object prediction unit (object PU) using the prediction parameters of the reference unit (processed PU located near the object PU), and generates a prediction image related to the color difference of the object prediction unit by referring to the prediction image or the decoded image of the brightness of the object prediction unit (object PU).

[0370] Furthermore, based on the above configuration, both the luminance tree structure and the chrominance tree structure can be shared, while different predictions (intra-frame prediction and inter-frame prediction) can be used for luminance and chrominance, thus without increasing the amount of code related to the tree structure. When the above prediction is selected as one of the merging mode candidates, the amount of code required for intra-frame prediction and the prediction methods related to inter-frame prediction for switching between luminance and chrominance can be minimized.

[0371] Based on the above configuration, when generating a prediction image related to chromatic aberration, the generated luminance prediction image or decoded image can be used. Therefore, in cases where the motion information (motion vector) is suitable for luminance but not for chromatic aberration, a more accurate prediction image can be generated, and the amount of code can be reduced.

[0372] Furthermore, the image decoding device 31 of this embodiment can also determine a mode (e.g., merge mode or CCLM mode) related to the generation method of the predicted image with respect to brightness and the predicted image with respect to color difference based on a flag (e.g., merge_cclm_flag).

[0373] Based on the above configuration, CCLM prediction can be applied to color differences without limiting luminance merging candidates, further reducing the amount of code. Furthermore, by limiting the selection to a merging mode, the amount of code required for the flags when selecting the above prediction can be minimized.

[0374] [Image encoding device]

[0375] An image encoding device 11 having the same configuration as the image decoding device 31 in embodiments 1 to 3 described above is also included in one embodiment of the present invention.

[0376] For example, the image decoding device 31 of Embodiment 1 and the image encoding device 11 having the same configuration encode the transformation coefficients related to brightness after dividing the transformation units corresponding to brightness in the transformation tree, and encode the transformation coefficients related to color difference without dividing the transformation units corresponding to color difference.

[0377] Based on the above configuration, the transform unit corresponding to the color difference is not segmented, thus maintaining the concentration of the residual energy corresponding to the color difference. Therefore, residual encoding can be performed efficiently, reducing the amount of code.

[0378] Furthermore, in Embodiment 2, the image decoding apparatus 31 and the image encoding apparatus 11, which have the same configuration, encode the luminance-related transformation coefficients in the transformation tree included in the encoding unit (CU) after segmenting the luminance-related transformation units (TU), but encode the chrominance-related transformation coefficients without segmenting the chrominance-related transformation units (TU). Additionally, in the prediction tree included in the encoding unit (CU), the image decoding apparatus 31 encodes the PU information related to the luminance-related prediction units (PU) after segmenting them, and encodes the PU information related to the chrominance-related prediction units (PU) after segmenting them in the prediction tree.

[0379] Based on the above configuration, only the transformation unit corresponding to luminance is segmented, thus maintaining the concentration of residual energy corresponding to color difference, and the prediction unit can be segmented.

[0380] Furthermore, the image decoding device 31 of Embodiment 3 and the image encoding device 11 having the same configuration generate a prediction image related to the brightness of the object prediction unit (object PU) using the prediction parameters of the reference unit (processed PU located near the object PU), and generate a prediction image related to the color difference of the object prediction unit by referring to the prediction image or the decoded image of the brightness of the object prediction unit (object PU).

[0381] Based on the above configuration, when generating a prediction image related to color difference, the generated brightness prediction image or decoded image can be used, thus generating a more accurate prediction image and reducing the amount of code.

[0382] [Software-based implementation example]

[0383] It should be noted that a portion of the image encoding device 11 and image decoding device 31 described in the foregoing embodiments can be implemented by a computer. For example, this includes the entropy decoding unit 301, the prediction parameter decoding unit 302, the loop filter 305, the prediction image generation unit 308, the inverse quantization / inverse transform unit 311, the addition unit 312, the prediction image generation unit 101, the subtraction unit 102, the transform / quantization unit 103, the entropy encoding unit 104, the inverse quantization / inverse transform unit 105, the loop filter 107, the encoding parameter determination unit 110, and the prediction parameter encoding unit 111. In this case, the program for implementing this control function can be recorded on a computer-readable recording medium, and the computer system can read and execute the program recorded on the recording medium. It should be noted that the "computer system" mentioned here refers to a computer system built into either the image encoding device 11 or the image decoding device 31, employing hardware including an operating system and peripheral devices. Furthermore, "computer-readable recording medium" refers to removable media such as floppy disks, magneto-optical disks, ROMs, and CD-ROMs, as well as storage devices such as hard disks built into computer systems. Moreover, "computer-readable recording medium" can also include: recording media that dynamically stores programs for a short period of time, such as communication lines used to transmit programs via networks like the Internet or telephone lines; and recording media that store programs for a fixed period of time, such as volatile memory within a computer system serving as a server or client in such cases. Furthermore, the aforementioned program can be a program used to implement the above-mentioned functions, or it can be a program that can implement the above-mentioned functions by combining with programs already recorded in the computer system.

[0384] Furthermore, the image encoding device 11 and image decoding device 31 in the above embodiments can also be implemented as integrated circuits such as LSI (Large Scale Integration). Each functional block of the image encoding device 11 and image decoding device 31 can be processorized individually, or some or all can be integrated for processorization. Moreover, the method of integrated circuit implementation is not limited to LSI; it can also be implemented using dedicated circuits or general-purpose processors. Furthermore, if an integrated circuit technology emerges to replace LSI as semiconductor technology advances, integrated circuits based on this technology can also be used.

[0385] The above description, with reference to the accompanying drawings, details one embodiment of the invention. However, the specific configuration is not limited to the above embodiment, and various design changes can be made without departing from the spirit of the invention.

[0386] [Application Example]

[0387] The image encoding device 11 and image decoding device 31 described above can be mounted on various devices for transmitting, receiving, recording, and reproducing moving images. It should be noted that the moving images can be natural moving images captured by a camera or the like, or artificial moving images (including CG and GUI) generated by a computer or the like.

[0388] First, refer to Figure 8 The following describes the case in which the image encoding device 11 and the image decoding device 31 described above can be used for the transmission and reception of moving images.

[0389] Figure 8 (a) is a block diagram showing the configuration of the transmitting device PROD_A equipped with the image encoding device 11. For example... Figure 8 As shown in (a), the transmitting device PROD_A includes: an encoding unit PROD_A1 that obtains encoded data by encoding a moving image; a modulation unit PROD_A2 that obtains a modulated signal by modulating a carrier wave using the encoded data obtained by the encoding unit PROD_A1; and a transmitting unit PROD_A3 that transmits the modulated signal obtained by the modulation unit PROD_A2. The image encoding device 11 described above is used as the encoding unit PROD_A1.

[0390] As a source of motion images input to the encoding unit PROD_A1, the transmitting device PROD_A may further include: a camera PROD_A4 for capturing motion images, a recording medium PROD_A5 for recording motion images, an input terminal PROD_A6 for inputting motion images from an external source, and an image processing unit A7 for generating or processing images. Figure 8 Example (a) shows that the transmitting device PROD_A has all of these components, but some can be omitted.

[0391] It should be noted that the recording medium PROD_A5 can be a medium that records unencoded motion images, or a medium that records motion images encoded using a recording encoding method different from the encoding method used for transmission. In the latter case, it is preferable that the decoding unit (not shown) that decodes the encoded data read from the recording medium PROD_A5 according to the recording encoding method is located between the recording medium PROD_A5 and the encoding unit PROD_A1.

[0392] Figure 8 (b) is a block diagram showing the configuration of the receiving device PROD_B equipped with the image decoding device 31. Figure 8As shown in (b), the receiving device PROD_B includes: a receiving unit PROD_B1 for receiving a modulated signal, a demodulation unit PROD_B2 for obtaining coded data by demodulating the modulated signal received by the receiving unit PROD_B1, and a decoding unit PROD_B3 for obtaining a moving image by decoding the coded data obtained by the demodulation unit PROD_B2. The image decoding device 31 described above is used as the decoding unit PROD_B3.

[0393] The receiving device PROD_B, serving as the destination for the moving images output by the decoding unit PROD_B3, may further include a display PROD_B4 for displaying the moving images, a recording medium PROD_B5 for recording the moving images, and an output terminal PROD_B6 for outputting the moving images to an external device. Figure 8 Example (b) shows the receiving device PROD_B having all of these components, but some may be omitted.

[0394] It should be noted that the recording medium PROD_B5 can be a medium for recording unencoded motion images, or it can be a medium encoded with a recording encoding method different from the encoding method used for transmission. In the latter case, it is preferable that the encoding unit (not shown) that encodes the motion images acquired from the decoding unit PROD_B3 according to the recording encoding method is located between the decoding unit PROD_B3 and the recording medium PROD_B5.

[0395] It should be noted that the transmission medium for modulated signals can be wireless or wired. Furthermore, the transmission scheme for modulated signals can be broadcast (here, a transmission scheme where the destination is not predetermined) or communication (here, a transmission scheme where the destination is predetermined). That is, the transmission of modulated signals can be achieved through any of the following: wireless broadcasting, wired broadcasting, wireless communication, and wired communication.

[0396] For example, a terrestrial digital broadcasting station (broadcasting equipment, etc.) / receiving station (television receiver, etc.) is an example of a transmitting device PROD_A / receiving device PROD_B that transmits and receives modulated signals via wireless broadcasting. Similarly, a cable television broadcasting station (broadcasting equipment, etc.) / receiving station (television receiver, etc.) is an example of a transmitting device PROD_A / receiving device PROD_B that transmits and receives modulated signals via cable broadcasting.

[0397] Furthermore, servers (workstations, etc.) and clients (TV receivers, personal computers, smartphones, etc.) using internet-based VOD (Video On Demand) services, moving image sharing services, etc., are examples of transmitting devices PROD_A and receiving devices PROD_B that transmit and receive modulated signals via communication (typically, either wireless or wired is used as the transmission medium in a LAN, and wired is used in a WAN). Here, personal computers include desktop PCs, laptop PCs, and tablet PCs. Additionally, smartphones also include multi-functional portable telephone terminals.

[0398] It should be noted that, in addition to decoding the encoded data downloaded from the server and displaying it on the screen, the client of the motion picture sharing service also has the function of encoding motion pictures captured by a camera and uploading them to the server. That is, the client of the motion picture sharing service performs the functions of both the sending device PROD_A and the receiving device PROD_B.

[0399] Next, refer to Figure 9 The following describes the case in which the image encoding device 11 and the image decoding device 31 described above can be used for recording and reproducing moving images.

[0400] Figure 9 (a) is a block diagram showing the configuration of the recording device PROD_C equipped with the aforementioned image encoding device 11. Figure 9 As shown in (a), the recording device PROD_C includes an encoding unit PROD_C1 that obtains encoded data by encoding a moving image and a writing unit PROD_C2 that writes the encoded data obtained by the encoding unit PROD_C1 to the recording medium PROD_M. The image encoding device 11 described above is used as the encoding unit PROD_C1.

[0401] It should be noted that the recording medium PROD_M can be (1) a type of recording medium built into the recording device PROD_C, such as HDD (Hard Disk Drive) or SSD (Solid State Drive), or (2) a type of recording medium connected to the recording device PROD_C, such as SD memory card or USB (Universal Serial Bus) flash memory, or (3) a recording medium loaded into a drive (not shown) built into the recording device PROD_C, such as DVD (Digital Versatile Disc) or BD (Blu-ray Disc).

[0402] Furthermore, as a source of motion images input to the encoding unit PROD_C1, the recording device PROD_C may further include: a camera PROD_C3 for capturing motion images, an input terminal PROD_C4 for inputting motion images from the outside, a receiving unit PROD_C5 for receiving motion images, and an image processing unit PROD_C6 for generating or processing images. Figure 9 Example (a) shows that the recording device PROD_C has all of these components, but some may be omitted.

[0403] It should be noted that the receiving unit PROD_C5 can receive unencoded motion images, or it can receive encoded data encoded using a transmission encoding method different from the encoding method used for recording. In the latter case, it is preferable to place the transmission decoding unit (not shown) that decodes the encoded data encoded using the transmission encoding method between the receiving unit PROD_C5 and the encoding unit PROD_C1.

[0404] Examples of such recording devices PROD_C include DVD recorders, BD recorders, and HDD (Hard Disk Drive) recorders (in which case the input terminal PROD_C4 or the receiving unit PROD_C5 is the main source of moving images). Furthermore, portable camcorders (in which case the camera PROD_C3 is the main source of moving images), personal computers (in which case the receiving unit PROD_C5 or the image processing unit C6 is the main source of moving images), and smartphones (in which case the camera PROD_C3 or the receiving unit PROD_C5 is the main source of moving images) are also examples of such recording devices PROD_C.

[0405] Figure 9 (b) is a block diagram showing the configuration of the playback device PROD_D equipped with the aforementioned image decoding device 31. Figure 9 As shown in (b), the playback device PROD_D includes a readout unit PROD_D1 that reads out encoded data written to the recording medium PROD_M, and a decoding unit PROD_D2 that obtains a moving image by decoding the encoded data read out by the readout unit PROD_D1. The image decoding device 31 described above is used as the decoding unit PROD_D2.

[0406] It should be noted that the recording medium PROD_M can be (1) a recording medium built into the playback device PROD_D, such as HDD or SSD, or (2) a recording medium connected to the playback device PROD_D, such as SD memory card or USB flash drive, or (3) a recording medium loaded into a drive device (not shown) built into the playback device PROD_D, such as DVD or BD.

[0407] Furthermore, as the destination for the motion images output by the decoding unit PROD_D2, the playback device PROD_D may further include: a display PROD_D3 for displaying motion images, an output terminal PROD_D4 for outputting motion images to the outside, and a transmitting unit PROD_D5 for transmitting motion images. Figure 9 Example (b) shows the reproduction device PROD_D having all of these components, but some can be omitted.

[0408] It should be noted that the transmitting unit PROD_D5 can transmit unencoded motion images, or it can transmit encoded data encoded using a transmission encoding method different from the encoding method used for recording. In the latter case, it is preferable to place the encoding unit (not shown) that encodes the motion images using the transmission encoding method between the decoding unit PROD_D2 and the transmitting unit PROD_D5.

[0409] Examples of such playback devices PROD_D include DVD players, BD players, HDD players, etc. (in which case, the output terminal PROD_D4 connected to a television receiver, etc., is the main destination for the moving images). Other examples include television receivers (in which case the display PROD_D3 is the main destination for the moving images), digital signage (also called electronic billboards, electronic bulletin boards, etc., where the display PROD_D3 or the transmitter PROD_D5 is the main destination for the moving images), desktop PCs (in which case the output terminal PROD_D4 or the transmitter PROD_D5 is the main destination for the moving images), laptop or tablet PCs (in which case the display PROD_D3 or the transmitter PROD_D5 is the main destination for the moving images), and smartphones (in which case the display PROD_D3 or the transmitter PROD_D5 is the main destination for the moving images).

[0410] (Hardware implementation and software implementation)

[0411] Furthermore, each of the aforementioned image decoding device 31 and image encoding device 11 can be implemented in hardware using logic circuits formed on an integrated circuit (IC chip), or in software using a CPU (Central Processing Unit).

[0412] In the latter case, the aforementioned devices include: a CPU that executes commands for programs that perform various functions; a ROM (Read Only Memory) that stores the programs; a RAM (Random Access Memory) that expands the programs; and a memory that stores the programs and various data, etc., such as a storage device (recording medium). Furthermore, the objective of embodiments of the present invention is to achieve this by supplying a recording medium containing program code (executable form program, intermediate code program, source program) of the software implementing the aforementioned functions, i.e., the control program of the aforementioned devices, in a computer-readable manner to the aforementioned devices, wherein the computer (or CPU, MPU) reads the program code recorded on the recording medium and executes it.

[0413] As recording media, the following can be used: tapes, cassette tapes, etc.; disks including floppy disks (registered trademark) / hard disks, CD-ROMs (Compact Disc Read-Only Memory), MO disks (Magneto-Optical Disc), MD disks (Mini Disc), DVDs (Digital Versatile Disc), CD-Rs (CD Recordable), Blu-ray discs (registered trademark), etc.; cards (including memory cards) / optical cards, etc.; semiconductor memory types such as mask ROMs / EPROMs (Erasable Programmable Read-Only Memory), EEPROMs (Electrically Erasable and Programmable Read-Only Memory), and flash memory ROMs; or logic circuits such as PLDs (Programmable Logic Devices) and FPGAs (Field Programmable Gate Arrays), etc.

[0414] Furthermore, the aforementioned devices can be configured to connect to a communication network and supply the program code via the communication network. This communication network need only be capable of transmitting program code and is not particularly limited. For example, the Internet, intranet, extranet, LAN (Local Area Network), ISDN (Integrated Services Digital Network), VAN (Value-Added Network), CATV (Community Antenna television / Cable Television) communication network, Virtual Private Network, telephone line network, mobile communication network, satellite communication network, etc., can be used. Furthermore, the transmission medium constituting this communication network need only be a medium capable of transmitting program code and is not limited to a specific configuration or type. For example, it can be used in wired networks such as IEEE (Institute of Electrical and Electronic Engineers) 1394, USB, power line transmission, cable TV lines, telephone lines, and ADSL (Asymmetric Digital Subscriber Line) lines, as well as in wireless networks such as IrDA (Infrared Data Association), infrared (like remote controls), Bluetooth (registered trademark), IEEE 802.11 wireless, HDR (High Data Rate), NFC (Near Field Communication), DLNA (Digital Living Network Alliance, registered trademark), mobile phone networks, satellite lines, and terrestrial digital broadcasting networks. It should be noted that embodiments of the present invention can also be implemented as computer data signals with embedded carriers that embody the above-mentioned program code via electronic transmission.

[0415] The embodiments of the present invention are not limited to the embodiments described above, and various modifications can be made within the scope of the claims. That is, embodiments obtained by combining technical solutions with appropriate modifications within the scope of the claims are also included within the technical scope of the present invention.

[0416] (Cross-reference to related applications)

[0417] This application claims priority to Japanese Patent Application No. 2017-090481, filed on April 28, 2017, and incorporates the entire contents of which hereby refer to.

[0418] Industrial availability

[0419] The embodiments of the present invention are preferably applied to an image decoding apparatus for decoding encoded data obtained by encoding image data and an image encoding apparatus for generating encoded data obtained by encoding image data. Furthermore, they are preferably applied to a data structure of encoded data generated by the image encoding apparatus and referenced by the image decoding apparatus.

[0420] Explanation of reference numerals in the attached figures

[0421] 11 Image encoding device

[0422] 31 Image Decoding Device

[0423] 301 Entropy Decoding Department

[0424] 302 Prediction Parameter Decoding Unit

[0425] 303 Inter-frame Prediction Parameter Decoding Unit

[0426] 304 Intra-frame Prediction Parameter Decoding Unit

[0427] 308 Predictive Image Generation Unit

[0428] 309 Inter-frame Prediction Image Generation Unit

[0429] 310 Intra-frame Prediction Image Generation Unit

[0430] 311 Inverse quantization / inverse transform unit

[0431] 31041 CCLM Parameter Derivation Section

[0432] 31042 CCLM Predictive Filtering Unit

[0433] 310421 Linear Prediction Unit

Claims

1. An image decoding apparatus for decoding an image for an encoding node, the image decoding apparatus comprising: The CN information decoding circuit for the encoded node is configured to: decode a first flag in the encoding tree, wherein the first flag indicates whether the encoded node is segmented, and the encoding tree is defined by the top-left coordinate, the width of the object encoded node, and the height of the object encoded node; and segment the encoding tree when the first flag is true, wherein the segmentation of the encoding tree is defined by the top-left coordinate, the half-width of the object encoded node, and the half-height of the object encoded node. as well as The encoding unit (CU) decoding circuit is configured to: decode the encoding unit when the first flag is false, wherein the encoding unit is defined by the upper left coordinate, the width of the object encoding node, and the height of the object encoding node, and the encoding unit includes a transformation unit, wherein the transformation unit includes a brightness transformation unit and a color difference transformation unit; decode a second flag when the first flag is false, wherein the second flag indicates whether the brightness transformation unit is segmented; if the second flag indicates that the brightness transformation unit is segmented, segment the brightness transformation unit into several brightness transformation units and do not segment the color difference transformation unit; and if the second flag indicates that the brightness transformation unit is not segmented, do not segment the brightness transformation unit and do not segment the color difference transformation unit.

2. An image encoding apparatus for encoding images for encoding nodes, the image encoding apparatus comprising: The encoding node CN information encoding circuit is configured to: encode a first flag in the encoding tree, wherein the first flag indicates whether the encoding node is segmented, and the encoding tree is defined by the top-left coordinate, the width of the object encoding node, and the height of the object encoding node; and segment the encoding tree when the first flag is true, wherein the segmentation of the encoding tree is defined by the top-left coordinate, the half-width of the object encoding node, and the half-height of the object encoding node. as well as The encoding unit (CU) encoding circuit is configured to: encode the encoding unit when the first flag is false, wherein the encoding unit is defined by the upper left coordinate, the width of the object encoding node, and the height of the object encoding node, and the encoding unit includes a transformation unit, wherein the transformation unit includes a brightness transformation unit and a color difference transformation unit; encode a second flag when the first flag is false, wherein the second flag indicates whether the brightness transformation unit is segmented; if the second flag indicates that the brightness transformation unit is segmented, the brightness transformation unit is segmented into several brightness transformation units and the color difference transformation unit is not segmented; and if the second flag indicates that the brightness transformation unit is not segmented, the brightness transformation unit is not segmented and the color difference transformation unit is not segmented.

Images