Prediction image generation device, moving image decoding device, moving image encoding device, and prediction image generation method

Abstract

The present invention reduces the storage amount required for CCLM prediction. A CCLM prediction parameter derivation section (310442) derives a scale shift value corresponding to a difference value of luminance, shifts a value obtained by multiplying a table value by a difference value of color difference using the scale shift value, and thereby derives a CCLM prediction parameter, wherein the table value is referred to as an index with a value obtained by right shifting the difference value of luminance by the scale shift value. Furthermore, in deriving a prediction image, the bit width is reduced by adaptively deriving a shift amount of a linear prediction parameter from a difference value of color difference.

Description

[0001] This application is a divisional application of the invention patent application filed on December 17, 2019, with application number 201980083015.X, entitled "Predictive image generation apparatus, motion picture decoding apparatus, motion picture encoding apparatus and predictive image generation method". Technical Field

[0002] The embodiments of the present invention relate to a predictive image generation apparatus, a motion picture decoding apparatus, a motion picture encoding apparatus, and a predictive image generation method. Background Technology

[0003] In order to efficiently transmit or record moving images, a moving image encoding device is used to generate encoded data by encoding the moving images, and a moving image decoding device is used to generate decoded images by decoding the encoded data.

[0004] Specific motion picture coding methods include H.264 / AVC and HEVC (High-Efficiency Video Coding).

[0005] In this motion picture coding method, the images (pictures) that constitute the motion picture are managed through a hierarchical structure and encoded / decoded by each CU. The hierarchical structure includes slices obtained by segmenting the image, coding tree units (CTUs) obtained by segmenting the slices, coding units (sometimes also called coding units (CUs)) obtained by segmenting the coding tree units, and transformation units (TUs) obtained by segmenting the coding units.

[0006] Furthermore, in such moving image coding methods, a prediction image is typically generated based on a locally decoded image obtained by encoding / decoding the input image, and the prediction error (sometimes called a "differential image" or "residual image") obtained by subtracting this prediction image from the input image (original image) is encoded. Methods for generating the prediction image include inter-frame prediction and intra-frame prediction. Non-patent literature 1 cites recent techniques for moving image coding and decoding.

[0007] Furthermore, in recent motion picture coding and decoding techniques, CCLM (Cross-Component Linear Model) prediction, which generates a chromatic difference image from a luminance image, is known. In CCLM prediction, linear prediction parameters are derived using decoded images adjacent to the object block, and the chromatic difference of the object block is predicted based on this linear prediction model (CCLM model) (Non-Patent Document 2).

[0008] Existing technical documents

[0009] Non-patent literature

[0010] Non-patent document 1: "Versatile Video Coding (Draft 3)", JVET-L1001, Joint Video Exploration Team (JVET) of ITU-T SG 16WP 3 and ISO / IEC JTC I / SC 29 / WG 11, 2018-11-08 17:06:06

[0011] Non-patent document 2: "CE3-5.1: On cross-component linear model simplification", JVET-L0191, Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC I / SC 29 / WG 11, 2018-10-03 Summary of the Invention

[0012] The problem the invention aims to solve

[0013] As mentioned above, in CCLM processing, linear prediction parameters are derived, and a linear prediction model is used to generate predicted images. Integer arithmetic and table references are used in the derivation of the linear prediction parameters, but this results in a large memory usage for the table.

[0014] Furthermore, when deriving the predicted value using the product of the gradient term of the linear prediction parameter (CCLM prediction parameter a) and the pixel value, the bit width of the CCLM prediction parameter a increases in the method of Non-Patent Document 1, resulting in high complexity of the product. Additionally, Non-Patent Document 1 also uses the product in the derivation of the gradient term of the linear prediction parameter (CCLM prediction parameter a) and the offset term of the linear prediction parameter (CCLM prediction parameter b), but this product is also a product of values ​​with large bit widths, thus increasing complexity. It should be noted that the product of values ​​with large bit widths increases the hardware size.

[0015] Technical solution

[0016] To address the aforementioned problems, one aspect of the present invention is a CCLM prediction unit that generates a prediction image through CCLM prediction. It is characterized by comprising: a CCLM prediction parameter derivation unit that derives CCLM prediction parameters (a, b) using luminance difference values, chrominance difference values, and a table; and a CCLM prediction filtering unit that generates a chrominance prediction image using a luminance reference image and the CCLM prediction parameters (a, b). The CCLM prediction parameter derivation unit right-shifts the value obtained by multiplying the value of the reciprocal table referenced to the luminance difference value with the chrominance difference value using a first shift value shift_a, thereby deriving the CCLM prediction parameter a. The CCLM prediction filtering unit right-shifts the product of parameter a and luminance using a second predetermined shift value shiftA, thereby deriving the chrominance prediction image.

[0017] Beneficial effects

[0018] According to one aspect of the present invention, there is an effect of simplifying the multiplication with linear prediction parameters in CCLM prediction. Attached Figure Description

[0019] Figure 1 This is a schematic diagram showing the configuration of the image transmission system of this embodiment.

[0020] Figure 2 This diagram illustrates the configuration of a transmitting device equipped with a motion image encoding apparatus according to this embodiment and a receiving device equipped with a motion image decoding apparatus. Figure 2 (a) indicates a transmitting device equipped with a motion image encoding device. Figure 2 (b) indicates a receiving device equipped with a motion picture decoding device.

[0021] Figure 3 This diagram illustrates the configuration of a recording apparatus equipped with a motion image encoding device according to this embodiment and a playback apparatus equipped with a motion image decoding device. Figure 3 (a) indicates a recording device equipped with a motion picture encoding device. Figure 3 (b) indicates a playback device equipped with a motion picture decoding device.

[0022] Figure 4 It is a diagram representing the hierarchical structure of the encoded stream data.

[0023] Figure 5 This is a diagram representing a segmentation example of CTU.

[0024] Figure 6 It is a schematic diagram representing the types (mode numbers) of intra-frame prediction modes.

[0025] Figure 7 This is a schematic diagram showing the configuration of a motion picture decoding device.

[0026] Figure 8 This is a schematic diagram showing the structure of the intra-frame prediction parameter decoding unit.

[0027] Figure 9 It is a map representing the reference region used for intra-frame prediction.

[0028] Figure 10 This is a diagram showing the structure of the intra-frame prediction image generation unit.

[0029] Figure 11 (a) is a block diagram illustrating an example configuration of the CCLM prediction unit according to an embodiment of the present invention. Figure 11 (b) is a diagram showing the derivation method of IntraPredModeC.

[0030] Figure 12 This is a block diagram illustrating an example configuration of a CCLM predictive filter unit according to an embodiment of the present invention.

[0031] Figure 13 (a) to (e) are diagrams illustrating the pixels referenced in the derivation of the CCLM prediction parameters in one embodiment of the present invention.

[0032] Figure 14 This is a diagram illustrating an example of the combination of (brightness, color difference) used in CCLM prediction in this embodiment.

[0033] Figure 15 This is a block diagram illustrating the structure of a motion picture encoding device.

[0034] Figure 16 This is a schematic diagram showing the structure of the intra-frame prediction parameter coding unit.

[0035] Figure 17 This is a diagram used to illustrate an example of the values ​​of elements that are not retained in a calculation table.

[0036] Figure 18 This is a diagram used to illustrate an example of the values ​​of elements that are not retained in a calculation table. Detailed Implementation

[0037] (First Implementation)

[0038] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings.

[0039] Figure 1 This is a schematic diagram showing the configuration of the image transmission system 1 of this embodiment.

[0040] Image transmission system 1 is a system for transmitting an encoded stream obtained by encoding an image of an encoded object, decoding the transmitted encoded stream, and displaying the image. Image transmission system 1 is configured to include: a moving image encoding device (image encoding device) 11, a network 21, a moving image decoding device (image decoding device) 31, and a moving image display device (image display device) 41.

[0041] The motion picture encoding device 11 is input to the image T.

[0042] Network 21 transmits the encoded stream Te generated by the motion picture encoding device 11 to the motion picture decoding device 31. Network 21 is the Internet, a wide area network (WAN), a local area network (LAN), or a combination thereof. Network 21 is not necessarily limited to a two-way communication network; it can also be a one-way communication network transmitting broadcast waves such as terrestrial digital broadcasting or satellite broadcasting. Furthermore, network 21 can also be replaced by a storage medium containing the encoded stream Te, such as a DVD (Digital Versatile Disc) or a Blu-ray Disc (Blu-ray Disc).

[0043] The motion picture decoding device 31 decodes the encoded stream Te transmitted by the network 21 to generate one or more decoded images Td.

[0044] The moving image display device 41 displays all or part of one or more decoded images Td generated by the moving image decoding device 31. The moving image display device 41 may include, for example, a liquid crystal display (LCD) or an organic EL (electroluminescence) display. Examples of display types include fixed, mobile, and HMD (Head-Down Display). Furthermore, when the moving image decoding device 31 has high processing power, it displays high-quality images; when it has only low processing power, it displays images that do not require high processing power or high display capabilities.

[0045] <operator>

[0046] The operators used in this specification are described below.

[0047] >> shifts to the right, << shifts to the left, & is bitwise AND, | is bitwise OR, |= is the OR substitution operator, and || represents logical AND.

[0048] x? y: z is a 3-term operator that takes y when x is true (other than 0) and takes z when x is false (0).

[0049] Clip3(a, b, c) is a function that clips c to a value that is greater than a and less than b. It returns a if c < a, returns b if c > b, and returns c otherwise (where a <= b).

[0050] abs(a) is a function that returns the absolute value of a.

[0051] Int(a) is a function that returns the integer value of a.

[0052] floor(a) is a function that returns the largest integer less than or equal to a.

[0053] ceil(a) is a function that returns the smallest integer greater than or equal to a.

[0054] a / d means a divided by d (with the decimal point discarded).

[0055] a^b means a raised to the power of b.

[0056] <Structure of the encoded stream Te>

[0057] Before providing a detailed description of the motion picture encoding device 11 and the motion picture decoding device 31 of this embodiment, the data structure of the encoded stream Te generated by the motion picture encoding device 11 and decoded by the motion picture decoding device 31 will be described.

[0058] Figure 4 This is a diagram representing the hierarchical structure of the data in the encoded stream Te. The encoded stream Te exemplarily includes a sequence and multiple images constituting the sequence. Figure 4 (a) to (f) are respectively diagrams representing the encoded video sequence of the given sequence SEQ, the encoded picture of the specified picture PICT, the encoded slice of the specified slice S, the encoded slice data of the specified slice data, the coding tree unit included in the encoded slice data, and the coding unit included in the coding tree unit.

[0059] (Encoded video sequence)

[0060] In the encoded video sequence, a set of data is specified for the motion picture decoding device 31 to refer to in order to decode the sequence SEQ of the object being processed. The sequence SEQ is as follows: Figure 4 As shown in (a), it includes the Video Parameter Set, the Sequence Parameter Set (SPS), the Picture Parameter Set (PPS), the Picture PICT, and the Supplemental Enhancement Information (SEI).

[0061] The Video Parameter Set (VPS) defines a set of common coding parameters for multiple motion pictures in a motion picture composed of multiple layers, as well as a set of coding parameters for the multiple layers included in the motion picture and associated with each layer.

[0062] The Sequence Parameter Set (SPS) specifies a set of encoding parameters that the moving image decoding device 31 refers to in order to decode the object sequence. For example, it specifies the width and height of the image. It should be noted that multiple SPSs can exist. In this case, any one of the multiple SPSs is selected from the PPS.

[0063] The Picture Parameter Set (PPS) specifies a set of encoding parameters that the moving image decoding device 31 refers to for decoding each picture in the object sequence. These parameters include, for example, a reference value for the quantization width used for picture decoding (pic_init_qp_minus26) and a flag indicating the application of weighted prediction (weighted_pred_flag). It should be noted that multiple PPSs can exist. In this case, any one of the multiple PPSs is selected from the pictures in the object sequence.

[0064] (Encoded image)

[0065] In the encoded image, a set of data is specified for the motion picture decoding device 31 to refer to in order to decode the image PICT of the object being processed. The image PICT is as follows: Figure 4 As shown in (b), it includes slice 0 to slice NS-1 (NS is the total number of slices included in the image PICT).

[0066] It should be noted that, in the following descriptions, where there is no need to distinguish between slices 0 to NS-1, the subscripts of the figure labels may sometimes be omitted. Furthermore, the same applies to other data with subscripts included in the encoded stream Te described below.

[0067] (Encoded slice)

[0068] In the encoded slice, a set of data is specified for the motion picture decoding device 31 to refer to in order to decode the slice S of the object being processed. The slice is as follows: Figure 4 (c) shows the slice header and slice data.

[0069] The slice header includes a set of encoded parameters for the moving image decoding device 31 to refer to in order to determine the decoding method for the object slice. The slice type specification information (slice_type) is an example of the encoded parameters included in the slice header.

[0070] As slice types that can be specified by slice type specification information, the following can be listed: (1) I slices that use only intra-frame prediction during encoding, (2) P slices that use unidirectional prediction or intra-frame prediction during encoding, and (3) B slices that use unidirectional prediction, bidirectional prediction, or intra-frame prediction during encoding. It should be noted that inter-frame prediction is not limited to unidirectional or bidirectional prediction, and more reference images can be used to generate the prediction image. Hereinafter, the cases referred to as P and B slices refer to slices that include blocks that can use inter-frame prediction.

[0071] It should be noted that the slice header may also include a reference to the image parameter set (PPS) (pic_parameter_set_id).

[0072] (Encoded slice data)

[0073] In the encoded slice data, a set of data is specified for the motion picture decoding device 31 to refer to in order to decode the slice data of the object being processed. Slice data such as... Figure 4 (d) shows the CTU. A CTU is a fixed-size (e.g., 64×64) block that makes up a slice, also known as the Largest Coding Unit (LCU).

[0074] (Coding Tree Unit)

[0075] exist Figure 4 In section (e), a set of data is specified for the motion picture decoding device 31 to refer to in order to decode the CTU of the processing object. The CTU is divided into coding units (CUs) as the basic unit of coding processing through recursive quadtree partitioning (QT), binary tree partitioning (BT), or ternary tree partitioning (TT). BT partitioning and TT partitioning are collectively referred to as multitree partitioning (MT). The nodes of the tree structure obtained by recursive quadtree partitioning are called coding nodes. The intermediate nodes of quadtrees, binary trees, and ternary trees are coding nodes, and the CTU itself is also defined as the top-level coding node.

[0076] CT includes the following information as CT information: a QT splitting flag (qt_split_cu_flag) indicating whether QT splitting is performed, an MT splitting flag (mtt_split_cu_flag) indicating whether MT splitting is performed, an MT splitting direction flag (mtt_split_cu_vertical_flag) indicating the MT splitting direction, and an MT splitting type flag (mtt_split_cu_binary_flag) indicating the MT splitting type. qt_split_cu_flag, mtt_split_cu_flag, mtt_split_cu_vertical_flag, and mtt_split_cu_binary_flag are transmitted per coding node.

[0077] Figure 5 This is a diagram representing a CTU split example. With qt_split_cu_flag set to 1, the encoding node is split into 4 encoding nodes. Figure 5 (b)

[0078] When qt_split_cu_flag is 0, and mtt_split_cu_flag is 0, the encoding nodes are not split, and one CU is retained as a node. Figure 5 (a)). CU is the terminal node of the encoding node and is not further segmented. CU is the basic unit of encoding processing.

[0079] When mtt_split_cu_flag is 1, the encoding node is split into two MT nodes as described below. When mtt_split_cu_vertical_flag is 0 and mtt_split_cu_binary_flag is 1, the encoding node is horizontally split into two encoding nodes. Figure 5 (d) means that when mtt_split_cu_vertical_flag is 1 and mtt_split_cu_binary_flag is 1, the encoding node is vertically split into 2 encoding nodes. Figure 5 (c)). Furthermore, when mtt_split_cu_vertical_flag is 0 and mtt_split_cu_binary_flag is 0, the encoding node is horizontally split into 3 encoding nodes ( Figure 5 (f) means that when mtt_split_cu_vertical_flag is 1 and mtt_split_cu_binary_flag is 0, the encoding node is vertically split into 3 encoding nodes. Figure 5 (e)). They will be in Figure 5As shown in (g).

[0080] Furthermore, when the size of the CTU is 64×64 pixels, the size of the CU can be any of the following: 64×64 pixels, 64×32 pixels, 32×64 pixels, 32×32 pixels, 64×16 pixels, 16×64 pixels, 32×16 pixels, 16×32 pixels, 16×16 pixels, 64×8 pixels, 8×64 pixels, 32×8 pixels, 8×32 pixels, 16×8 pixels, 8×16 pixels, 8×8 pixels, 64×4 pixels, 4×64 pixels, 32×4 pixels, 4×32 pixels, 16×4 pixels, 4×16 pixels, 8×4 pixels, 4×8 pixels, and 4×4 pixels.

[0081] (Encoding unit)

[0082] like Figure 4 As shown in (f), a set of data is specified for the motion picture decoding device 31 to refer to in order to decode the encoding unit of the object being processed. Specifically, the CU consists of a CU header CUH, prediction parameters, transform parameters, quantization transform coefficients, etc. The prediction mode, etc., are specified in the CU header.

[0083] Prediction processing can be performed on a per-unit basis (CU) or on a per-unit basis, based on further subdivisions of the CU into sub-CUs. When the size of the CU and its sub-CUs are equal, there is one sub-CU within the CU. When the size of the CU is larger than the size of its sub-CUs, the CU is divided into sub-CUs. For example, if the CU is 8×8 and the sub-CUs are 4×4, the CU is divided into four sub-CUs, comprising two horizontally divided parts and two vertically divided parts.

[0084] There are two types of prediction (prediction modes): intra-frame prediction and inter-frame prediction. Intra-frame prediction is prediction within the same image, while inter-frame prediction refers to prediction processing performed between different images (e.g., between display times).

[0085] Transform / quantization processing is performed on a unit basis (CU), but quantization transform coefficients can also be entropy encoded on a unit basis (4×4 sub-blocks).

[0086] (Prediction parameters)

[0087] The predicted image is derived from the prediction parameters appended to the block. These prediction parameters include those for intra-frame prediction and those for inter-frame prediction.

[0088] The following explains the prediction parameters for intra-frame prediction. The intra-frame prediction parameters consist of the luminance prediction mode (IntraPredModeY) and the chrominance prediction mode (IntraPredModeC). Figure 6 This is a schematic diagram representing the types (mode numbers) of intra-frame prediction modes. For example... Figure 6 As shown, there are 67 intra-frame prediction modes (0-66). For example, there are planar prediction (0), DC prediction (1), and Angular prediction (2-66). Furthermore, CCLM modes (81-83) can be added to the chromatic aberration.

[0089] Syntax elements used to derive intra-prediction parameters include, for example, intra_luma_mpm_flag, mpm_idx, and mpm_remainder.

[0090] (MPM)

[0091] `intra_luma_mpm_flag` is a flag indicating whether the object block's lumen prediction mode, `IntraPredModeY`, matches the `Most Probable Mode` (MPM). `MPM` refers to the prediction modes included in the `mpmCandList[]` of MPM candidates. The `MPM` candidate list stores candidates with high probabilities of being applied to the object block based on the intra prediction modes of adjacent blocks and the specified intra prediction mode. When `intra_luma_mpm_flag` is 1, the lumen prediction mode `IntraPredModeY` of the object block is derived using the MPM candidate list and the index `mpm_idx`.

[0092] IntraPredModeY=mpmCandList[mpm_idx]

[0093] (REM)

[0094] With intra_luma_mpm_flag set to 0, the luma prediction mode IntraPredModeY is derived using mpm_remainder. Specifically, the intra prediction mode is selected from RemIntraPredMode, the remaining mode after removing the intra prediction modes included in the MPM candidate list from all intra prediction modes.

[0095] (Composition of a motion picture decoding device)

[0096] The motion picture decoding device 31 of this embodiment ( Figure 7 The composition of ) will be explained.

[0097] The motion picture decoding device 31 is configured to include: an entropy decoding unit 301, a parameter decoding unit (prediction image decoding device) 302, a loop filter 305, a reference image memory 306, a prediction parameter memory 307, a prediction image generation unit 308, an inverse quantization / inverse transform unit 311, and an adder unit 312. It should be noted that, according to the motion picture encoding device 11 described later, there is also a configuration where the motion picture decoding device 31 does not include the loop filter 305.

[0098] The parameter decoding unit 302 also includes a header decoding unit 3020, a CT information decoding unit 3021, and a CU decoding unit 3022 (prediction mode decoding unit). The CU decoding unit 3022 also includes a TU decoding unit 3024. These can be collectively referred to as decoding modules. The header decoding unit 3020 decodes parameter set information such as VPS, SPS, and PPS, and the slice header (slice information) from the encoded data. The CT information decoding unit 3021 decodes the CT from the encoded data. The CU decoding unit 3022 decodes the CU from the encoded data. In the case where the TU includes prediction errors, the TU decoding unit 3024 decodes the QP update information (quantization correction value) and the quantization prediction error (residual_coding) from the encoded data.

[0099] Furthermore, the parameter decoding unit 302 is configured to include an inter-frame prediction parameter decoding unit 303 and an intra-frame prediction parameter decoding unit 304. The prediction image generation unit 308 is configured to include an inter-frame prediction image generation unit 309 and an intra-frame prediction image generation unit 310.

[0100] Furthermore, examples of using CTU and CU as processing units are described below, but this is not the only approach; processing can also be performed on a sub-CU basis. Alternatively, processing can be performed on a block or sub-block basis by replacing CTU and CU with blocks and sub-CUs with sub-blocks.

[0101] The entropy decoding unit 301 performs entropy decoding on the externally input encoded stream Te, separating and decoding each code (syntactic element). The separated code contains prediction information for generating the predicted image and prediction errors for generating the difference image. The entropy decoding unit 301 outputs the separated code to the parameter decoding unit 302.

[0102] (Configuration of the intra-frame prediction parameter decoding unit 304)

[0103] The intra-prediction parameter decoding unit 304 decodes the intra-prediction parameters based on the code input from the entropy decoding unit 301 and with reference to the prediction parameters stored in the prediction parameter memory 307, for example, decoding the intra-prediction mode IntraPredMode. The intra-prediction parameter decoding unit 304 outputs the decoded intra-prediction parameters to the prediction image generation unit 308 and then stores them in the prediction parameter memory 307. The intra-prediction parameter decoding unit 304 can also derive intra-prediction modes that differ in brightness and chromatic difference.

[0104] Figure 8 This is a schematic diagram showing the configuration of the intra-frame prediction parameter decoding unit 304 of the parameter decoding unit 302. (See diagram below.) Figure 8 As shown, the intra-frame prediction parameter decoding unit 304 is configured to include: a parameter decoding control unit 3041, a luminance intra-frame prediction parameter decoding unit 3042, and a chrominance intra-frame prediction parameter decoding unit 3043.

[0105] The parameter decoding control unit 3041 instructs the entropy decoding unit 301 to decode the syntax elements and receives the syntax elements from the entropy decoding unit 301. When `intra_luma_mpm_flag` is 1, the parameter decoding control unit 3041 outputs `mpm_idx` to the MPM parameter decoding unit 30422 within the luma intra-frame prediction parameter decoding unit 3042. Furthermore, when `intra_luma_mpm_flag` is 0, the parameter decoding control unit 3041 outputs `mpm_remainder` to the non-MPM parameter decoding unit 30423 of the luma intra-frame prediction parameter decoding unit 3042. Additionally, the parameter decoding control unit 3041 outputs the intra-frame prediction parameter `intra_chroma_pred_mode` for chroma to the chroma intra-frame prediction parameter decoding unit 3043.

[0106] The luminance intra-frame prediction parameter decoding unit 3042 is configured to include: an MPM candidate list derivation unit 30421, an MPM parameter decoding unit 30422, and a non-MPM parameter decoding unit 30423 (decoding unit and derivation unit).

[0107] The MPM parameter decoding unit 30422 derives the brightness prediction mode IntraPredModeY by referring to the MPM candidate list mpmCandList[] and mpm_idx derived by the MPM candidate list derivation unit 30421, and outputs it to the intra-frame prediction image generation unit 310.

[0108] The non-MPM parameter decoding unit 30423 derives IntraPredModeY from the MPM candidate list mpmCandList[] and mpm_remainder, and outputs it to the intra-prediction image generation unit 310.

[0109] The chroma intra-prediction parameter decoding unit 3043 derives the chroma prediction mode IntraPredModeC from intra_chroma_pred_mode and outputs it to the intra-prediction image generation unit 310.

[0110] The loop filter 305 is a filter located within the encoding loop, used to remove block distortion and ringing distortion to improve image quality. The loop filter 305 performs deblocking filtering, sample adaptive offset (SAO), and adaptive loop filtering (ALF) on the decoded image of the CU generated by the adder 312.

[0111] The image memory 306 stores the CU decoded image generated by the addition unit 312 in a predetermined location according to each object image and object CU.

[0112] The prediction parameter memory 307 stores the prediction parameters in predetermined locations according to the CTU or CU of each decoded object. Specifically, the prediction parameter memory 307 stores the parameters decoded by the parameter decoding unit 302 and the prediction mode predMode separated by the entropy decoding unit 301.

[0113] The prediction image generation unit 308 is input with a prediction mode (predMode), prediction parameters, etc. Furthermore, the prediction image generation unit 308 reads a reference image from the reference image memory 306. Under the prediction mode indicated by the prediction mode (predMode), the prediction image generation unit 308 uses the prediction parameters and the read-out reference image (reference image block) to generate a prediction image of a block or sub-block. Here, a reference image block refers to a set of pixels (usually rectangular, hence called a block) on the reference image, which is a region referenced for generating the prediction image.

[0114] (Intra-frame prediction image generation unit 310)

[0115] When the prediction mode predMode indicates the intra-prediction mode, the intra-prediction image generation unit 310 uses the intra-prediction parameters input from the intra-prediction parameter decoding unit 304 and the reference pixels read from the reference image memory 306 to perform intra-prediction.

[0116] Specifically, the intra-frame prediction image generation unit 310 reads adjacent blocks on the object image from the reference image memory 306, which are within a predetermined range from the object block. The predetermined range consists of adjacent blocks to the left, upper left, upper, and upper right of the object block, and varies depending on the region referenced by the intra-frame prediction mode.

[0117] The intra-prediction image generation unit 310 generates a prediction image of the target block by referring to the read-out decoded pixel values ​​and the prediction mode indicated by IntraPredMode. The intra-prediction image generation unit 310 outputs the generated prediction image of the block to the addition unit 312.

[0118] The following describes the generation of predicted images based on intra-frame prediction modes. In Planar prediction, DC prediction, and Angular prediction, the decoded surrounding region adjacent to (close to) the predicted object block is set as the reference region R. Then, the predicted image is generated by extrapolating the pixels on the reference region R to a specific direction. For example, the reference region R can be set as an L-shaped region including the left and top (or further top-left, top-right, and bottom-left) of the predicted object block (e.g., by...). Figure 9 (a) The area represented by the circular marker with a diagonal line.

[0119] (Details of the predictive image generation department)

[0120] Next, use Figure 10 The detailed configuration of the intra-frame prediction image generation unit 310 will be described below. The intra-frame prediction image generation unit 310 includes: a prediction target block setting unit 3101, an unfiltered reference image setting unit 3102 (first reference image setting unit), a filtered reference image setting unit 3103 (second reference image setting unit), a prediction unit 3104, and a prediction image correction unit 3105 (prediction image correction unit, filter switching unit, weighting coefficient changing unit).

[0121] The prediction unit 3104 generates a temporary prediction image (pre-correction prediction image) of the prediction target block based on each reference pixel (unfiltered reference image) on the application reference region R, the filtered reference image generated by the reference pixel filter (first filter), and the intra-frame prediction mode, and outputs it to the prediction image correction unit 3105. The prediction image correction unit 3105 corrects the temporary prediction image according to the intra-frame prediction mode, generates a prediction image (corrected prediction image), and outputs it.

[0122] The components of the intra-frame prediction image generation unit 310 will be described below.

[0123] (Prediction Object Block Setting Unit 3101)

[0124] The prediction object block setting unit 3101 sets the object CU as the prediction object block and outputs information related to the prediction object block (prediction object block information). The prediction object block information includes at least the size, position, and index representing the brightness or color difference of the prediction object block.

[0125] (Unfiltered reference image setting unit 3102)

[0126] The unfiltered reference image setting unit 3102 sets the adjacent surrounding area of ​​the predicted object block as a reference region R based on the size and position of the predicted object block. Then, it sets the corresponding decoded pixel values ​​(unfiltered reference image, boundary pixels) in the reference image memory 306 for each pixel value in the reference region R. Figure 9 The row r[x][-1] of the decoded pixels adjacent to the top edge of the predicted object block and the column r[-1][y] of the decoded pixels adjacent to the left edge of the predicted object block shown in (a) are unfiltered reference images.

[0127] (Filtered reference image setting unit 3103)

[0128] The filtered reference image setting unit 3103 applies a reference pixel filter (first filter) to the unfiltered reference image according to the intra-frame prediction mode, and derives the filtered reference image s[x][y] for each position (x, y) on the reference region R. Specifically, a low-pass filter is applied to the unfiltered reference image at position (x, y) and its surroundings to derive the filtered reference image (s[x][y]). Figure 9 (b) It should be noted that a low-pass filter may not be applied to all intra-frame prediction modes, but may be applied to a portion of the intra-frame prediction modes. It should be noted that the filter applied to the unfiltered reference image on the reference region R in the filtered reference pixel setting unit 3103 is called the "reference pixel filter (first filter)", while the filter used to correct the temporary prediction image in the prediction image correction unit 3105 described later is called the "boundary filter (second filter)".

[0129] (Configuration of the intra-frame prediction unit 3104)

[0130] The intra-frame prediction unit 3104 generates a temporary prediction image (temporary prediction pixel values, pre-correction prediction image) of the prediction target block based on the intra-frame prediction mode, the unfiltered reference image, and the filtered reference pixel values, and outputs it to the prediction image correction unit 3105. The prediction unit 3104 internally includes: a Planar prediction unit 31041, a DC prediction unit 31042, an Angular prediction unit 31043, and a CCLM prediction unit (prediction image generation device) 31044. The prediction unit 3104 selects a specific prediction unit according to the intra-frame prediction mode and inputs the unfiltered reference image and the filtered reference image. The relationship between the intra-frame prediction mode and the corresponding prediction unit is shown below.

[0131] •Planar Prediction•··Planar Prediction Department 31041

[0132] DC Forecasting... DC Forecasting Department 31042

[0133] ·Angular Prediction···Angular Prediction Department 31043

[0134] ·CCLM Forecast···CCLM Forecast Department 31044

[0135] (Planar prediction)

[0136] The Planar prediction unit 31041 generates a temporary prediction image by linearly adding multiple filtered reference images based on the distance between the predicted object pixel position and the reference pixel position, and outputs it to the prediction image correction unit 3105.

[0137] (DC forecast)

[0138] The DC prediction unit 31042 derives a DC prediction value equivalent to the average value of the filtered reference image s[x][y], and outputs a temporary prediction image q[x][y] with the DC prediction value as the pixel value.

[0139] (Angular prediction)

[0140] The Angular prediction unit 31043 generates a temporary prediction image q[x][y] using the filtered reference image s[x][y] in the prediction direction (reference direction) indicated by the intra-frame prediction mode, and outputs it to the prediction image correction unit 3105.

[0141] (CCLM (Cross-Component Linear Model) Prediction)

[0142] The CCLM prediction unit 31044 predicts the pixel values ​​of color difference based on the pixel values ​​of luminance. Specifically, it generates a predicted image of the color difference (Cb, Cr) using a linear model based on the decoded luminance image.

[0143] (Configuration of the predictive image correction unit 3105)

[0144] The prediction image correction unit 3105 corrects the temporary prediction image output from the prediction unit 3104 according to the intra-prediction mode. Specifically, for each pixel of the temporary prediction image, the prediction image correction unit 3105 performs a weighted sum (weighted average) of the unfiltered reference image and the temporary prediction image based on the distance between the reference region R and the target prediction pixel, thereby deriving the corrected prediction image (corrected prediction image) Pred. It should be noted that in some intra-prediction modes (e.g., Planar prediction, DC prediction, etc.), the temporary prediction image can be directly used as the prediction image without correcting it through the prediction image correction unit 3105.

[0145] The inverse quantization / inverse transform unit 311 dequantizes the quantization transform coefficients input from the entropy decoding unit 301 to obtain the transform coefficients. These quantization transform coefficients are obtained by performing frequency transformations such as DCT (Discrete Cosine Transform) and DST (Discrete Sine Transform) on the prediction error during the encoding process and then quantizing it. The inverse quantization / inverse transform unit 311 performs inverse DCT, inverse DST, and other inverse frequency transformations on the obtained transform coefficients to calculate the prediction error. The inverse quantization / inverse transform unit 311 outputs the prediction error to the adder unit 312.

[0146] The adder 312 adds the predicted image of the block input from the predicted image generation unit 308 to the prediction error input from the inverse quantization / inverse transform unit 311 for each pixel to generate the decoded image of the block. The adder 312 stores the decoded image of the block in the reference image memory 306 and outputs it to the loop filter 305.

[0147] (Composition of a motion picture encoding device)

[0148] Next, the configuration of the motion image encoding device 11 in this embodiment will be described. Figure 14 This is a block diagram illustrating the configuration of the motion picture encoding apparatus 11 according to this embodiment. The motion picture encoding apparatus 11 is configured to include: a prediction image generation unit 101, a subtraction unit 102, a transform / quantization unit 103, an inverse quantization / inverse transform unit 105, an addition unit 106, a loop filter 107, a prediction parameter memory (prediction parameter storage unit, frame memory) 108, a reference image memory (reference image storage unit, frame memory) 109, an encoding parameter determination unit 110, a parameter encoding unit 111, and an entropy encoding unit 104.

[0149] The prediction image generation unit 101 generates a prediction image based on the regions (CU) formed by dividing each image T into its constituent parts. The prediction image generation unit 101 performs the same operation as the prediction image generation unit 308 described previously, and its description is omitted here.

[0150] The subtraction unit 102 subtracts the pixel values ​​of the predicted image of the block input from the prediction image generation unit 101 from the pixel values ​​of the image T to generate a prediction error. The subtraction unit 102 outputs the prediction error to the transform / quantization unit 103.

[0151] The transform / quantization unit 103 calculates the transform coefficients from the prediction error input from the subtraction unit 102 through frequency transformation, and derives the quantized transform coefficients through quantization. The transform / quantization unit 103 outputs the quantized transform coefficients to the entropy encoding unit 104 and the inverse quantization / inverse transform unit 105.

[0152] Inverse quantization / inverse transform unit 105 and inverse quantization / inverse transform unit 311 in motion image decoding device 31 Figure 7 The same applies, so its explanation is omitted here. The calculated prediction error is input into the adder 106.

[0153] In the entropy coding unit 104, quantization transformation coefficients are input from the transform / quantization unit 103, and coding parameters are input from the parameter coding unit 111. The coding parameters include, for example, codes for the reference image index refIdxLX, the prediction vector index mvp_LX_idx, the difference vector mvdLX, the motion vector precision mode amvr_mode, the prediction mode predMode, and the merge index merge_idx.

[0154] The entropy coding unit 104 entropy codes the segmentation information, prediction parameters, quantization transformation coefficients, etc. to generate a coded stream Te and outputs it.

[0155] The parameter coding unit 111 includes: a header coding unit 1110 (not shown), a CT information coding unit 1111, a CU coding unit 1112 (prediction mode coding unit), an inter-frame prediction parameter coding unit 112, and an intra-frame prediction parameter coding unit 113. The CU coding unit 1112 also includes a TU coding unit 1114.

[0156] (The structure of the intra-frame prediction parameter coding unit 113)

[0157] The intra-prediction parameter encoding unit 113 derives the encoding format (e.g., mpm_idx, mpm_remainder, etc.) based on the intra-prediction mode IntraPredMode input from the encoding parameter determination unit 110. The intra-prediction parameter encoding unit 113 includes the same configuration as the configuration portion derived by the intra-prediction parameter decoding unit 304.

[0158] Figure 15 This is a schematic diagram showing the configuration of the intra-frame prediction parameter coding unit 113 of the parameter coding unit 111. The intra-frame prediction parameter coding unit 113 is configured to include: a parameter coding control unit 1131, a luma intra-frame prediction parameter derivation unit 1132, and a chromatic difference intra-frame prediction parameter derivation unit 1133.

[0159] The encoding parameter determination unit 110 inputs the luminance prediction mode IntraPredModeY and the chrominance prediction mode IntraPredModeC to the parameter encoding control unit 1131. The parameter encoding control unit 1131 determines intra_luma_mpm_flag by referring to the MPM candidate list mpmCandList[] of the reference candidate list derivation unit 30421. Then, intra_luma_mpm_flag and IntraPredModeY are output to the luminance intra-frame prediction parameter derivation unit 1132. In addition, IntraPredModeC is output to the chrominance intra-frame prediction parameter derivation unit 1133.

[0160] The luminance intra-frame prediction parameter derivation unit 1132 is configured to include: an MPM candidate list derivation unit 30421 (candidate list derivation unit), an MPM parameter derivation unit 11322 (parameter derivation unit), and a non-MPM parameter derivation unit 11323 (coding unit, derivation unit).

[0161] The MPM candidate list derivation unit 30421 derives the MPM candidate list `mpmCandList[]` by referring to the intra-prediction modes of adjacent blocks stored in the prediction parameter memory 108. The MPM parameter derivation unit 11322 derives `mpm_idx` from `IntraPredModeY` and `mpmCandList[]` when `intra_luma_mpm_flag` is 1, and outputs it to the entropy coding unit 104. The non-MPM parameter derivation unit 11323 derives `mpm_remainder` from `IntraPredModeY` and `mpmCandList[]` when `intra_luma_mpm_flag` is 0, and outputs it to the entropy coding unit 104.

[0162] The intra-chroma prediction parameter derivation unit 1133 derives and outputs intra_chroma_pred_mode from IntraPredModeY and IntraPredModeC.

[0163] The addition unit 106 adds the pixel values ​​of the block prediction image input from the prediction image generation unit 101 and the prediction error input from the inverse quantization / inverse transform unit 105 to generate a decoded image by adding each pixel. The addition unit 106 stores the generated decoded image in the reference image memory 109.

[0164] The loop filter 107 performs deblocking filtering, SAO, and ALF on the decoded image generated by the adder 106. It should be noted that the loop filter 107 does not necessarily include the above three filters; for example, it may only include a deblocking filter.

[0165] The prediction parameter memory 108 stores the prediction parameters generated by the encoding parameter determination unit 110 in a predetermined location for each object image and CU.

[0166] The image memory 109 stores the decoded images generated by the loop filter 107 in predetermined locations for each object image and each CU.

[0167] The encoding parameter determination unit 110 selects one set from a plurality of sets of encoding parameters. The encoding parameters refer to the QT, BT, or TT segmentation information, prediction parameters, or parameters generated in association with them as encoding objects, as described above. The prediction image generation unit 101 uses these encoding parameters to generate a prediction image.

[0168] The encoding parameter determination unit 110 calculates the RD cost value, representing the amount of information and the encoding error, for each of the multiple sets. The RD cost value is, for example, the sum of the code size and the squared error multiplied by a coefficient λ. The encoding parameter determination unit 110 selects the set of encoding parameters with the smallest calculated cost value. Therefore, the entropy encoding unit 104 outputs the selected set of encoding parameters as a code stream Te. The encoding parameter determination unit 110 stores the determined encoding parameters in the prediction parameter memory 108.

[0169] It should be noted that a portion of the motion picture encoding device 11 and motion picture decoding device 31 described above can be implemented using a computer. For example, this includes the entropy decoding unit 301, parameter decoding unit 302, loop filter 305, prediction image generation unit 308, inverse quantization / inverse transform unit 311, addition unit 312, prediction image generation unit 101, subtraction unit 102, transform / quantization unit 103, entropy encoding unit 104, inverse quantization / inverse transform unit 105, loop filter 107, encoding parameter determination unit 110, and parameter encoding unit 111. In this case, the program for implementing this control function can be recorded on a computer-readable recording medium, and the computer system can read and execute the program recorded on the recording medium. It should be noted that the "computer system" mentioned here refers to a computer system built into either the motion picture encoding device 11 or the motion picture decoding device 31, employing hardware including an operating system and peripheral devices. Furthermore, "computer-readable recording medium" refers to removable media such as floppy disks, magneto-optical disks, ROMs, and CD-ROMs, as well as storage devices such as hard disks built into computer systems. Moreover, "computer-readable recording medium" can also include: recording media that dynamically stores programs for a short period of time, such as communication lines used to transmit programs via networks like the Internet or telephone lines; and recording media that store programs for a fixed period of time, such as volatile memory within a computer system serving as a server or client in such cases. Furthermore, the aforementioned program can be a program used to implement the above-mentioned functions, or a program that can implement the above-mentioned functions by combining with programs already recorded in the computer system.

[0170] Furthermore, some or all of the motion picture encoding device 11 and motion picture decoding device 31 in the above embodiments can be implemented as integrated circuits such as LSI (Large Scale Integration). Each functional block of the motion picture encoding device 11 and motion picture decoding device 31 can be processorized individually, or some or all can be integrated for processorization. Moreover, the method of integrated circuit implementation is not limited to LSI; it can also be implemented using dedicated circuits or general-purpose processors. Furthermore, in the event that an integrated circuit technology replacing LSI emerges with advancements in semiconductor technology, integrated circuits based on this technology can also be used.

[0171] The above description, with reference to the accompanying drawings, details one embodiment of the invention. However, the specific configuration is not limited to the above embodiment, and various design changes can be made without departing from the spirit of the invention.

[0172] [Application Example]

[0173] The aforementioned moving image encoding device 11 and moving image decoding device 31 can be mounted on various devices for transmitting, receiving, recording, and reproducing moving images. It should be noted that the moving images can be natural moving images captured by a camera or the like, or artificial moving images (including CG and GUI) generated by a computer or the like.

[0174] First, refer to Figure 2 The following describes the situation where the above-described motion picture encoding device 11 and motion picture decoding device 31 can be used for the transmission and reception of motion pictures.

[0175] Figure 2 (a) is a block diagram showing the configuration of the transmitting device PROD_A equipped with the motion picture encoding device 11. For example... Figure 2 As shown in (a), the transmitting device PROD_A includes: an encoding unit PROD_A1 that obtains encoded data by encoding a moving image; a modulation unit PROD_A2 that obtains a modulated signal by modulating a carrier wave using the encoded data obtained by the encoding unit PROD_A1; and a transmitting unit PROD_A3 that transmits the modulated signal obtained by the modulation unit PROD_A2. The moving image encoding device 11 described above is used as the encoding unit PROD_A1.

[0176] As a source of motion images input to the encoding unit PROD_A1, the transmitting device PROD_A may further include: a camera PROD_A4 for capturing motion images, a recording medium PROD_A5 for recording motion images, an input terminal PROD_A6 for inputting motion images from an external source, and an image processing unit A7 for generating or processing images. Figure 2 Example (a) shows that the transmitting device PROD_A has all of these components, but some may be omitted.

[0177] It should be noted that the recording medium PROD_A5 can be a medium that records unencoded motion images, or a medium that records motion images encoded using a recording encoding method different from the encoding method used for transmission. In the latter case, it is preferable that the decoding unit (not shown) that decodes the encoded data read from the recording medium PROD_A5 according to the recording encoding method is located between the recording medium PROD_A5 and the encoding unit PROD_A1.

[0178] Figure 2 (b) is a block diagram showing the configuration of the receiving device PROD_B equipped with the motion picture decoding device 31. Figure 2As shown in (b), the receiving device PROD_B includes: a receiving unit PROD_B1 for receiving a modulated signal, a demodulation unit PROD_B2 for obtaining coded data by demodulating the modulated signal received by the receiving unit PROD_B1, and a decoding unit PROD_B3 for obtaining a moving image by decoding the coded data obtained by the demodulation unit PROD_B2. The aforementioned moving image decoding device 31 is used as the decoding unit PROD_B3.

[0179] The receiving device PROD_B, serving as the destination for the moving images output by the decoding unit PROD_B3, may further include a display PROD_B4 for displaying the moving images, a recording medium PROD_B5 for recording the moving images, and an output terminal PROD_B6 for outputting the moving images to an external device. Figure 2 Example (b) shows the receiving device PROD_B having all of these components, but some may be omitted.

[0180] It should be noted that the recording medium PROD_B5 can be a medium for recording unencoded motion images, or it can be a medium encoded with a recording encoding method different from the encoding method used for transmission. In the latter case, it is preferable that the encoding unit (not shown) that encodes the motion images acquired from the decoding unit PROD_B3 according to the recording encoding method is located between the decoding unit PROD_B3 and the recording medium PROD_B5.

[0181] It should be noted that the transmission medium for modulated signals can be wireless or wired. Furthermore, the transmission scheme for modulated signals can be broadcast (here, a transmission scheme where the destination is not predetermined) or communication (here, a transmission scheme where the destination is predetermined). That is, the transmission of modulated signals can be achieved through any of the following: wireless broadcasting, wired broadcasting, wireless communication, and wired communication.

[0182] For example, a terrestrial digital broadcasting station (broadcasting equipment, etc.) / receiving station (television receiver, etc.) is an example of a transmitting device PROD_A / receiving device PROD_B that transmits and receives modulated signals via wireless broadcasting. Similarly, a cable television broadcasting station (broadcasting equipment, etc.) / receiving station (television receiver, etc.) is an example of a transmitting device PROD_A / receiving device PROD_B that transmits and receives modulated signals via cable broadcasting.

[0183] Furthermore, servers (workstations, etc.) and clients (TV receivers, personal computers, smartphones, etc.) using internet-based VOD (Video On Demand) services, moving image sharing services, etc., are examples of transmitting devices PROD_A and receiving devices PROD_B that transmit and receive modulated signals via communication (typically, either wireless or wired is used as the transmission medium in a LAN, and wired is used in a WAN). Here, personal computers include desktop PCs, laptop PCs, and tablet PCs. Additionally, smartphones also include multi-functional portable telephone terminals.

[0184] It should be noted that, in addition to decoding the encoded data downloaded from the server and displaying it on the screen, the client of the motion picture sharing service also has the function of encoding motion pictures captured by a camera and uploading them to the server. That is, the client of the motion picture sharing service performs the functions of both the sending device PROD_A and the receiving device PROD_B.

[0185] Next, refer to Figure 3 The following describes the situation where the above-mentioned motion picture encoding device 11 and motion picture decoding device 31 can be used for recording and reproducing motion pictures.

[0186] Figure 3 (a) is a block diagram showing the configuration of the recording device PROD_C equipped with the aforementioned motion picture encoding device 11. Figure 3 As shown in (a), the recording device PROD_C includes an encoding unit PROD_C1 that obtains encoded data by encoding a moving image and a writing unit PROD_C2 that writes the encoded data obtained by the encoding unit PROD_C1 to the recording medium PROD_M. The moving image encoding device 11 described above is used as the encoding unit PROD_C1.

[0187] It should be noted that the recording medium PROD_M can be (1) a type of recording medium built into the recording device PROD_C, such as HDD (Hard Disk Drive) or SSD (Solid State Drive), or (2) a type of recording medium connected to the recording device PROD_C, such as SD memory card or USB (Universal Serial Bus) flash memory, or (3) a recording medium loaded into a drive (not shown) built into the recording device PROD_C, such as DVD (Digital Versatile Disc) or BD (Blu-ray Disc).

[0188] Furthermore, as a source of motion images input to the encoding unit PROD_C1, the recording device PROD_C may further include: a camera PROD_C3 for capturing motion images, an input terminal PROD_C4 for inputting motion images from the outside, a receiving unit PROD_C5 for receiving motion images, and an image processing unit PROD_C6 for generating or processing images. Figure 3 Example (a) shows that the recording device PROD_C has all of these components, but some may be omitted.

[0189] It should be noted that the receiving unit PROD_C5 can receive unencoded motion images, or it can receive encoded data encoded using a transmission encoding method different from the encoding method used for recording. In the latter case, it is preferable to place the transmission decoding unit (not shown) that decodes the encoded data encoded using the transmission encoding method between the receiving unit PROD_C5 and the encoding unit PROD_C1.

[0190] Examples of such recording devices PROD_C include DVD recorders, BD recorders, and HDD (Hard Disk Drive) recorders (in which case the input terminal PROD_C4 or the receiving unit PROD_C5 is the main source of moving images). Furthermore, portable camcorders (in which case the camera PROD_C3 is the main source of moving images), personal computers (in which case the receiving unit PROD_C5 or the image processing unit C6 is the main source of moving images), and smartphones (in which case the camera PROD_C3 or the receiving unit PROD_C5 is the main source of moving images) are also examples of such recording devices PROD_C.

[0191] Figure 3 (b) is a block diagram showing the configuration of the playback device PROD_D equipped with the aforementioned motion picture decoding device 31. Figure 3 As shown in (b), the playback device PROD_D includes a readout unit PROD_D1 that reads encoded data written to the recording medium PROD_M and a decoding unit PROD_D2 that obtains a motion image by decoding the encoded data read out by the readout unit PROD_D1. The motion image decoding device 31 described above is used as the decoding unit PROD_D2.

[0192] It should be noted that the recording medium PROD_M can be (1) a recording medium built into the playback device PROD_D, such as HDD or SSD, or (2) a recording medium connected to the playback device PROD_D, such as SD memory card or USB flash drive, or (3) a recording medium loaded into a drive device (not shown) built into the playback device PROD_D, such as DVD or BD.

[0193] Furthermore, as the destination for the motion images output by the decoding unit PROD_D2, the playback device PROD_D may further include: a display PROD_D3 for displaying motion images, an output terminal PROD_D4 for outputting motion images to the outside, and a transmitting unit PROD_D5 for transmitting motion images. Figure 3 Example (b) shows the reproduction device PROD_D having all of these components, but some can be omitted.

[0194] It should be noted that the transmitting unit PROD_D5 can transmit unencoded motion images, or it can transmit encoded data encoded using a transmission encoding method different from the encoding method used for recording. In the latter case, it is preferable to place the encoding unit (not shown) that encodes the motion images using the transmission encoding method between the decoding unit PROD_D2 and the transmitting unit PROD_D5.

[0195] Examples of such playback devices PROD_D include DVD players, BD players, HDD players, etc. (in which case, the output terminal PROD_D4 connected to a television receiver, etc., is the main destination for the moving images). Other examples include television receivers (in which case the display PROD_D3 is the main destination for the moving images), digital signage (also called electronic billboards, electronic bulletin boards, etc., where the display PROD_D3 or the transmitter PROD_D5 is the main destination for the moving images), desktop PCs (in which case the output terminal PROD_D4 or the transmitter PROD_D5 is the main destination for the moving images), laptop or tablet PCs (in which case the display PROD_D3 or the transmitter PROD_D5 is the main destination for the moving images), and smartphones (in which case the display PROD_D3 or the transmitter PROD_D5 is the main destination for the moving images).

[0196] (Intra-frame prediction mode for chromatic aberration)

[0197] Next, refer to Figures 11-14 Explanation of CCLM prediction.

[0198] When deriving the chroma prediction mode IntraPredModeC, the intra-frame prediction parameter decoding unit 304 refers to the luminance prediction mode IntraPredModeY, intra_chroma_pred_mode, and Figure 11 The table of (b). Figure 11 (b) illustrates the derivation method of IntraPredModeC. IntraPredModeC is derived based on the value of IntraPredModeY when intra_chroma_pred_mode is 0–3 and 7. For example, when intra_chroma_pred_mode is 0 and IntraPredModeY is 0, IntraPredModeC is 66. Furthermore, when intra_chroma_pred_mode is 3 and IntraPredModeY is 50, IntraPredModeC is 1. It should be noted that the values ​​of IntraPredModeY and IntraPredModeC represent… Figure 6 The intra-frame prediction mode. With intra_chroma_pred__mode set to 4–6, IntraPredModeC is derived independently of the value of IntraPredModeY. intra_chroma_pred_mode = 81 (INTRA_LT_CCLM), 82 (INTRA_L_CCLM), and 83 (INTRA_T_CCLM) represent the prediction modes for generating chromatic aberration images based on the luminance images of the upper and left adjacent blocks, the prediction modes for generating chromatic aberration images based on the luminance images of the left adjacent block, and the prediction modes for generating chromatic aberration images based on the luminance images of the upper adjacent block, respectively.

[0199] The following explains CCLM prediction. In the figure, pY[][] and pRefY[][] represent the object block and adjacent block of the brightness image. The width of the object block is bW and the height is bH.

[0200] CCLM prediction unit 31044 (unfiltered reference image setting unit 3102) will Figure 13 The brightness adjacency images pRefY[][] of (a) to (c) and Figure 13 The color difference adjacency image pRefC[][] of (e) is used as a reference region to derive CCLM prediction parameters. The CCLM prediction unit 31044 uses the luminance object image pRef[] to derive the color difference prediction image.

[0201] CCLM prediction unit 31044, when intra_chroma_pred_mode is 81 (INTRA_LT_CCLM), such as Figure 13 As shown in (a), the CCLM prediction parameters are derived using the pixel values ​​of the top and left adjacent blocks of the object block, with intra_chroma_pred_mode set to 82 (INTRA_L_CCLM). Figure 13 As shown in (b), the CCLM prediction parameters are derived using the pixel values ​​of the left neighboring block. With intra_chroma_pred_mode set to 83 (INTRA_T_CCLM), the following results are obtained: Figure 13 As shown in (c), the CCLM prediction parameters are derived using the pixel values ​​of the upper adjacent blocks. The sizes of each region can be as follows. Figure 13 In (a), the top of the object block has a width of bW and a height of refH (refH > 1), and the left side of the object block has a height of bH and a width of refW (refW > 1). Figure 13 In (b), the height is 2*bH and the width is refW. Figure 13 In (c), the width is 2*bW and the height is refH. To implement downsampling, refW and refH can be set to values ​​greater than 1, matching the number of taps in the downsampling filter. Furthermore, in Figure 13 In (e), pC[][] and pRefC[][] represent the object block and adjacent block of the color difference image (Cb, Cr). The object block has a width of bWC and a height of bHC.

[0202] (CCLM Forecasting Department)

[0203] based on Figure 11 The CCLM prediction unit 31044 will be explained. Figure 11 This is a block diagram illustrating an example of the configuration of the CCLM prediction unit 31044. The CCLM prediction unit 31044 includes: a downsampling unit 310441, a CCLM prediction parameter derivation unit (parameter derivation unit) 310442, and a CCLM prediction filtering unit 310443.

[0204] The downsampling unit 310441 downsamples pRefY[][] and pY[][] to match the size of the color difference image. When the color difference format is 4:2:0, the horizontal and vertical pixel counts of pRefY[][] and pY[][] are sampled at a 2:1 ratio, and the results are stored in […]. Figure 13pRefDsY[][] and pDsY[][] of (d). It should be noted that bW / 2 and bH / 2 are equal to bWC and bHC, respectively. In the case of a color difference format of 4:2:2, the number of pixels in the horizontal direction of pRefY[][] and pY[][] is sampled at a ratio of 2:1, and the result is stored in pRefDsY[][] and pDsY[][]. In the case of a color difference format of 4:4:4, no sampling is performed, and pRefY[][] and pY[][] are stored in pRefDsY[][] and pDsY[][]. An example of sampling is represented by the following formula.

[0205] pDsY[x][y]=

[0206] (pY[2*x-1][2*y]+pY[2*x-1][2*y+1]+2*pY[2*x][2*y]+2*pY[2*x][2*y+1]+

[0207] pY[2*x+1][2*y]+pY[2*x+1][2*y+1]+4)>>3

[0208] pRefDsY[x][y]=

[0209] (pRefY[2*x-1][2*y]+pRefY[2*x-1][2*y+1]+2*pRefY[2*x][2*y]+2*pRefY

[0210] [2*x][2*y+1]+pRefY[2*x+1][2*y]+pRefY[2*x+1][2*y+1]+4)>>3

[0211] The CCLM prediction filter unit 310443 takes the reference image refSamples[][] as the input signal and outputs the prediction image predSamples[][] using CCLM prediction parameters (a, b).

[0212] predSamples[][]=((a*refSamples[][])>>shiftA)+b(CCLM-1)

[0213] Here, refSamples is Figure 13 pDsY of (d), (a, b) are CCLM prediction parameters derived by CCLM prediction parameter derivation unit 310442, and predSamples[][] are color difference prediction images ( Figure 13(e) of pC). It should be noted that (a, b) are derived for Cb and Cr respectively. In addition, shiftA is the normalized shift number representing the precision of the value of a, where a = af << shiftA when the slope of the decimal precision is set to af. For example, shiftA = 16.

[0214] Figure 12 This describes the configuration of the CCLM predictive filter unit 310443, which predicts color difference based on luminance. For example... Figure 12 As shown, the CCLM predictive filter unit 310443 includes a linear prediction unit 310444. The linear prediction unit 310444 takes refSamples[][] as input signal and outputs predSamples[][] using CCLM prediction parameters (a, b).

[0215] More specifically, the linear prediction unit 310444 uses the following formula of CCLM prediction parameters (a, b) to derive the color difference Cb or Cr based on the luminance Y, and outputs predSamples using the color difference Cb or Cr[][].

[0216] Cb (or Cr) = aY + b

[0217] CCLM prediction parameter derivation unit 310442 calculates the brightness of the adjacent block pRefY ( Figure 13 (d) of pRefDsY[][]) and the adjacent block of color difference pRefC[][]( Figure 13 The CCLM prediction parameters are derived by taking pRefC[][]) as the input signal. The CCLM prediction parameter derivation unit 310442 outputs the derived CCLM prediction parameters (a, b) to the CCLM prediction filter unit 310443.

[0218] (Derivation of CCLM Prediction Parameters)

[0219] CCLM prediction parameter derivation section 310442 derives the CCLM prediction parameters (a, b) for the case of linear prediction of the prediction block predSamples[][] of the target block based on the reference block refSamples[][].

[0220] In the derivation of CCLM prediction parameters (a, b), the CCLM prediction parameter derivation unit 310442 derives the point (x1, y1) with the maximum (Y_MAX) luminance value Y and the point (x2, y2) with the minimum (Y_MIN) luminance value Y from the group of adjacent blocks (luminance value Y, color difference value C). Next, the pixel values ​​of (x1, y1) and (x2, y2) on pRefC corresponding to (x1, y1) and (x2, y2) on pRefDsY are set to C_MAX (or C_Y_MAX) and C_MIN (or C_Y_MIN), respectively. Then, as... Figure 14 As shown, find the straight line connecting (Y_MAX, C_MAX) and (Y_MIN, C_MIN) on the graph with Y and C as the x and y axes, respectively. The CCLM prediction parameters (a, b) of this line can be derived using the following formula.

[0221] a=(C_MAX-C_MIN) / (Y_MAX-Y_MIN)

[0222] b = C_MIN - (a * Y_MIN)

[0223] When using (a, b), shiftA = 0 for the expression (CCLM-1).

[0224] Here, when the color difference is Cb, (C_MAX, C_MIN) are the pixel values ​​(x1, y1) and (x2, y2) of the adjacent block pRefCb[][] of Cb, and when the color difference is Cr, (C_MAX, C_MIN) are the pixel values ​​(x1, y1) and (x2, y2) of the adjacent block pRefCr[][] of Cr.

[0225] It should be noted that division is computationally expensive; therefore, integer arithmetic and table references are used instead of division to derive the CCLM prediction parameters (a, b). Specifically, they are calculated using the following formula. It should be noted that in the following implementation, a table excluding the case of d = 0 (a table with d = diff - 1 set as the independent variable) is used in the reciprocal table required for division of 1 / d, but of course, a table with d = diff set as the independent variable can also be used.

[0226] ChromaDelta = C_MAX - C_MIN

[0227] low=(ChrmaDelta*LMDivTableLow[diff-1]+2^15)>>16

[0228] a=(ChromaDelta*LMDivTable[diff-1]+low+add)>>shiftB

[0229] b=C_MIN-((a*Y_MIN)>>shiftA)

[0230] diff=(Y_MAX-Y_MIN+add)>>shiftB

[0231] shiftB = (BitDepthC > 8)?

[0232] (BitDepthC-9): 0

[0233] add=(shiftB>0)? 1<<(shiftB-1):0

[0234] In this case, when diff = 0, a = 0 is set. Here, LMDivTableLow[] and LMDivTable[] are tables (reciprocal, reciprocal table) used to perform division by reference, which are derived beforehand using the following formula. In other words, the values ​​held in the table are the derived values ​​(the values ​​corresponding to the reciprocal of the divisor). That is, the difference values ​​and derived values ​​are maintained in a corresponding manner in the table. Furthermore, shiftB is a shift value used to quantize the range of diff values, which varies depending on the bit depth, to a value less than 2^9 = 512. It should be noted that when the bit depth C of the color difference image is 10 bits or more, quantization is performed beforehand to make diff within the range of 0 to 512. shiftB is not limited to the above. For example, a specified constant Q (e.g., 2, 6, etc.) can be used as shown below. The same applies in other embodiments.

[0235] shiftB = BitDepthC-Q

[0236] LMDivTable[diff-1]=floor(2^16 / diff)

[0237] LMDivTableLow[diff-1]=floor((2^16*2^16) / diff)-floor(2^16 / diff)*2^16

[0238] LMDivTable[diff-1] represents the integer part of (1 / diff*2^16). LMDivTableLow[diff-1] represents the fractional part of (1 / diff*2^16) multiplied by 2^16. Furthermore, a and b are 2^16 times the value of the above expression (C=a*Y+b).

[0239] For example, if diff = 7, it will look like this.

[0240] LMDivTable[7-1]=floor(2^16 / 7)=9362

[0241] LMDivTableLow[7-1]=floor((2^16*2^16) / 7)-floor(2^16 / 7)*2^16=18724

[0242] When using (a, b) derived from the above formula, the shiftA of the formula (CCLM-1) is 16.

[0243] (Specific processing in the CCLM forecasting department)

[0244] (Processing Example 1)

[0245] In the above embodiment, the CCLM prediction parameter derivation unit 310442 uses a table LMDivTable representing the integer part of 1 / diff and a table LMDivTableLow representing the fractional part of 1 / diff to derive CCLM prediction parameters. Here, the maximum value of table LMDivTable is 65536 (17 bits), and the maximum value of table LMDivTableLow is 65140 (16 bits). Since each table has 512 elements, a very large memory size of 17*512 + 16*512 = 16896 (bits) is required to store tables LMDivTable and LMDivTableLow.

[0246] In this processing example, the CCLM prediction parameter derivation unit 310442 derives the CCLM prediction parameters without using the LMDivTableLow, which represents the decimal part of 1 / diff in the two tables. That is, the CCLM prediction parameter derivation unit 310442 derives the CCLM prediction parameters (a, b) using the following formula.

[0247] a=(ChromaDelta*LMDivTable[diff-1]+add)>>shiftB

[0248] b=C_MIN-((a*Y_MIN)>>shiftA)

[0249] add=(shiftB>0)? 1<<(shiftB-1):0

[0250] The inventors experimentally confirmed that CCLM prediction has sufficient performance using only a single table, LMDivTable.

[0251] Therefore, since there is no need to store the table LMDivTableLow, the storage required for storing the table can be roughly halved. Furthermore, since there are no calculations needed to derive the div, the processing load is also reduced.

[0252] As described above, the CCLM prediction unit 31044 of this process generates a prediction image through CCLM prediction. It has a CCLM prediction parameter derivation unit 310442, which uses the difference value with a plurality of brightness reference pixels and the derivation value for deriving CCLM prediction parameters based on the difference value to establish a corresponding table LMDivTable to derive CCLM prediction parameters. The table LMDivTable retains the integer part of the value obtained by multiplying the reciprocal of the difference value by a constant.

[0253] This reduces the amount of table storage required for deriving CCLM prediction parameters.

[0254] (Processing Example 2)

[0255] In this processing example, the number of bits (bit width) of the LMDivTable used by the CCLM prediction parameter derivation unit 310442 is reduced.

[0256] In the above implementation, the value of table LMDivTable is the integer part of (1 / diff)*65536, as shown below. 65536, 32768, 21845, 16384, 13107, 10922, 9362, 8192, 7281, 6553, 5957, 5461, 5041, 4681, 4369, 4096, 3855, 3640, 3449, 3276, 3120, 2978, 2849, 2730, 2621, 2520, 2427, 2340, 2259, 2184, 2114, 2048......

[0258] In this processing example, the mantissa (m) of each of the above values ​​is approximated in the exponential representation (m*2^exp) using P bits, and only the mantissa is kept in the table DivTableM. For example, with P=5, the values ​​of the reciprocal table are as follows.

[0259] 16*2^12, 16*2^11, 21*2^10, 16*2^10, 26*2^9, 21*2^9, 18*2^9, 16*2^9, 28*2^8, 26*2^8, 23*2^8, 21*2^8, 20*2^8, 18*2^8, 17*2^8, 16*2^8, 30*2^7, 28*2^7, 27*2^7, 26*2^7, 24*2^7, 23*2^7, 22*2^7, 21*2^7, 20*2^7, 20*2^7, 19*2^7, 18*2^7, 18*2^7, 17*2^7, 17*2^7, 16*2^7...

[0260] In the DivTableM table, only the mantissa portion of these values ​​is retained. That is,

[0261] DivTableM[]={16,16,21,16,26,21,18,16,28,26,23,21,20,18,17,16,30,28,27,26,24,23,22,21,20,20,19,18,18,17,17,16...}

[0262] Therefore, in the above implementation, each value that requires a maximum of 17 bits can be represented by 5 bits, which can reduce the storage amount required for the DivTableM storage table.

[0263] It should be noted that when the number of elements to be retained is set to 2^N from the beginning of the table, the minimum value of the table DivTableM is 2^(P-1). Therefore, the values ​​obtained by subtracting 2^(P-1) from each value can be retained in the table DivTableM, and the above value can be derived by adding 2^(P-1) to the values ​​obtained from this table. In this case, the memory required for each value can be further reduced by 1 bit. Hereinafter, the offset value of the table DivTableM when the number of elements to be retained is set to 2^N from the beginning of the table will be called offsetM. When using a table with the offset subtracted, offsetM = 2^(P-1). In other cases, offsetM = 0.

[0264] Furthermore, when the integer part of (1 / diff)*(2^16) is expressed through the exponent, and only the last part of the reciprocal table is retained, it is necessary to derive the value of the exponent part. In this processing example, the CCLM prediction parameter derivation unit 310442 derives the value of the exponent part according to the following formula.

[0265] exp=clz(d,N)+(16-N-(P-1))

[0266] Here, d = diff - 1 (the difference in brightness), exp represents the exponent, and N represents the number of elements retained in the table. For example, if N = 9, 512 elements are retained; if N = 5, 32 elements are retained. Furthermore, "16" is the precision of 1 / diff, which is the number of bits used as the multiplier to convert 1 / diff into an integer representation. In the above implementation, multiplication by 65536 (= 2^16) is performed to derive the value of 1 / diff with integer precision. It should be noted that the precision of 1 / diff is arbitrary; if other values ​​are used, the "16" must be changed accordingly.

[0267] The `clz(count leading zeros)` function is a function consisting of two arguments (d, mw). It returns the number of consecutive zeros starting from the most significant bit (MSB) of the first argument, `d`, represented in binary. The second argument, `mw`, represents the maximum number of bits. For example, if P = 5, then in the case of `d = 1` (0b00001, diff = 2) (where `0b` is the prefix representing the binary number), `clz(1, mw) = 4`, and in the case of `d = 2` (0b00010, diff = 3), `clz(2, mw) = 3`. Furthermore, in the case of `d = 16` (0b10000, diff = 17), `clz(16, mw) = 0`. It should be noted that when the first argument `d` is 0, the function returns the second argument `mw`. That is, in the case of `d = 0` (0b00000, diff = 1), `clz(0, mw) = mw`.

[0268] It should be noted that the clz function has dedicated instructions on various CPUs. In these dedicated instructions, the maximum bit depth is sometimes limited to values ​​such as 8, 16, or 32. However, for example, when mw <= 8, clz(d, mw) = clz(d, 8) - (8 - mw). Furthermore, these dedicated instructions are not mandatory; for example, in the case of clz(d, 4), the following formula can be used instead.

[0269] clz(d,4)=(d&0x08)? 1: (d&0x04)? 2: (d&0x02)? 3: (d&0x01)? 4:5

[0270] The CCLM prediction unit 31044 uses the exp derived from d to shift the value obtained by multiplying the DivTableM[d] referenced by the luminance difference d (=diff-1) with the chromatic difference ChromaDelta, thereby deriving the CCLM prediction parameter a.

[0271] a=(ChromaDelta*DivTableM[d]<<exp)+add)>>shiftB

[0272] b=C_MIN-((a*Y_MIN)>>shiftA)

[0273] Here, exp = clz(d, N) + (16 - N - (P - 1)) = 16 - (P - 1) - (N - clz(d, N))

[0274] For LMDivTable[d], DivTableM[d], and exp in (Example 1), the following relationship holds.

[0275] LMDivTable[d]=DivTableM[d]<<exp

[0276] It should be noted that the CCLM prediction parameter a can be derived after deriving the shift value (shiftB-exp) using the exponent part exp as described below. However, for simplicity, it is assumed that the sign and direction of the shift value are reversed when the shift values ​​of the right and left bit shift operations are negative. The same applies to other examples.

[0277] a=(ChromaDelta*DivTableM[d]+add)>>(shiftB-exp)

[0278] Here, add = (shiftB - exp > 0) ? 1 << (shiftB - exp - 1) : 0

[0279] Alternatively, the CCLM prediction parameter derivation unit 310442 can derive the value of the exponent part according to the following formula.

[0280] exp=16-(P-1)-ceil(log2(diff))

[0281] Alternatively, only the exponent portion can be kept as the table ShiftTableE. For example, with P=5, the table ShiftTableE[] would look like this:

[0282] ShiftTableE[] = {12, 11, 10, 10, 9, 9, 9, 9, 8, 8, 8, 8, 8, 8, 8, 7 ...

[0283] ShiftTableE'[]={0,1,2,2,3,3,3,3,4,4,4,4,4,4,4,4,5,5,5,5,5,5,5,5,5,5,5,5,5,5,5,5,5...}

[0284] The exponent part is calculated as follows:

[0285] exp=16-(P-1)-ShiftTableE'[d].

[0286] If the maximum number of bits mw is less than N, then tables ShiftTableE and ShiftTableE' can be used instead of the clz function.

[0287] clz(d,mw)=mw-ShiftTableE'[d]=mw-(16-(P-1)-ShiftTableE[d]))

[0288] CCLM prediction unit 31044 uses table DivTableM and shift table ShiftTableE to derive CCLM parameters through the following formula.

[0289] a=(ChromaDelta*DivTableM[d]<<ShiftTableE[d])+add)>>shiftBb=C_MIN-((a*Y_MIN)>>shiftA)

[0290] It should be noted that the shift factor (shiftB-exp) can be derived using the exponential part exp, and then the CCLM prediction parameter a can be derived.

[0291] a=(ChromaDelta*DivTableM[d]+add)>>(shiftB-exp)

[0292] Here, add = (shiftB - exp > 0) ? 1 << (shiftB - exp - 1) : 0, exp = ShiftTableE[d]

[0293] As described above, the CCLM prediction unit 31044 of this processing example generates a prediction image through CCLM prediction. It includes a CCLM prediction parameter derivation unit 310442, which derives the CCLM prediction parameter a using the difference value of luminance (d), the difference value of chromatic difference (ChromaDelta), and the reciprocal table DivTableM. The CCLM prediction parameter derivation unit 310442 derives the exponent part exp corresponding to the reciprocal of the difference value of luminance (d), multiplies it by the element of the table DivTableM and the difference value of chromatic difference, and derives the CCLM prediction parameter by shifting the number of shifts derived from the exponent part exp.

[0294] By reducing the number of bits of values ​​held in the reciprocal table required for deriving the CCLM prediction parameters as described above, the required storage can be reduced. It should be noted that, as explained in Processing Example 1, the table LMDivTableLow can be omitted, but while retaining LMDivTableLow, a table can still be generated that divides the elements of LMDivTableLow into mantissa and exponent portions.

[0295] (Supplementing the number of bits in the product)

[0296] As mentioned above, in CCLM, the derivation of CCLM prediction parameters a and b and the generation of prediction images using a and b require the product of ChromaDelta*LMDivTable[diff-1], a*Y_MIN, and a*refSamples[][].

[0297] a=(ChromaDelta*LMDivTable[diff-1]+low+add)>>shiftB / / 11bit*17bit=28

[0298] b=C_MIN-((a*Y_MIN)>>shifiA) / / 27bit*10bit=37

[0299] predSamples[][]=((a*refSamples[][])>>shiftA)+b / / 27bit*10bit=37

[0300] (Bit width of parameter a)

[0301] For example, in the calculation prior to Example 1 above, in a 10-bit image, the bit widths of ChromaDelta, a, Y_MIN, and refSamples[][] are 11 bits, 10 bits, 10 bits, and 10 bits, respectively. When shiftA = 16, the bit widths of LMDivTable[diff-1] and a are 16 bits and 27 bits (= 11 bits * 16 bits), respectively. As a result, the derivation of a, the derivation of b, and the product of the prediction using a are 11 bits * 17 bits, 27 bits * 10 bits, and 27 bits * 10 bits, respectively, and the product of the larger bit widths, making the hardware more complex.

[0302] In Example 2, the product of ChromaDelta*DivTableM[d] in the derivation of a is reduced to a bit width of exp (=ShiftTableE[d]) smaller than that in Example 1, thus simplifying the product.

[0303] (Processing Example 3)

[0304] In the above implementation, the tables LMDivTable (and LMDivTableLow) required for CCLM prediction store the values ​​of 512 features, ranging from 1 to 512, which are the possible values ​​for the brightness difference value diff. In this processing example, the number of features stored in the tables is reduced, and the features that are not maintained are derived by calculation, thereby reducing the required memory.

[0305] For example, the CCLM prediction parameter derivation unit 310442 derives the CCLM prediction parameters (a, b) using a table LMDivTable_2N with 2^N features. Then, the CCLM prediction parameter derivation unit 310442 calculates the values ​​of features not stored in the table LMDivTable_2N using 1 / k of the stored values. LMDivTable_2N[] is a table that stores the first 2^N features of LMDivTable[].

[0306] Specifically, refer to Figure 17 Please provide an explanation. Figure 17 This is a diagram used to illustrate an example of the values ​​of elements not retained in a calculation table. In this example, N = 3. (The diagram is incomplete and requires further context.) Figure 17As shown, the CCLM prediction parameter derivation unit 310442 directly uses the values ​​of table LMDivTable_2N for the interval D0[0..2^N-1] (eg.0..7) where the values ​​of table LMDivTable_2N are held. For the next interval D1[2^N..2^(N+1)-1] (eg.8..15), it uses the second half of the interval D0'[2^N / 2..2^(N+1)-1] (eg.4) of table LMDivTable_2N. The value of ..7) is half of the value of the next interval D2[2^(N+1)..2^(N+2)-1](eg.16..31), and the value of the next interval D3[2^(N+2)..2^(N+3)-1](eg.32..63) is also 1 / 8 of the value of the next interval D0'(eg.4..7) in the LMDivTable_2N. Furthermore, interval D1 is twice the width of interval D0', interval D2 is four times the width of interval D0', and interval D3 is eight times the width of interval D0'. In other words, the values ​​in the interval Dsc[2^(N+sc-1)..2^(N+sc)-1] are obtained by multiplying the values ​​in the interval D0'[2^N / 2..2^(N+1)-1] by 1 / k (where K = 2^sc). Every k values ​​starting from the beginning of interval Dsc are stored as the same value. Here, 1 <= sc <= 6. 6 is derived from 9-3, where 9 represents the precision of the diff, and 3 is determined based on D1 starting from 8 (= 2^3).

[0307] For example, if N = 3, the values ​​after d(=diff-1) = 8 are calculated by multiplying the values ​​of interval D0'[4..7] by 1 / k as described below.

[0308] Interval [8..15] → 1 / 2

[0309] Interval [16..31] → 1 / 4

[0310] Interval [32..63] → 1 / 8

[0311] Interval [64..127] → 1 / 16

[0312] Interval [128..255] → 1 / 32

[0313] Interval [256..511] → 1 / 64

[0314] [Table 1]

[0315] D1 [8..15] 1 / 2 1 D2 [16..31] 1 / 4 2 D3 [32..63] 1 / 8 3 D4 [64..127] 1 / 16 4 D5 [128..255] 1 / 32 5 D6 [256..511] 1 / 64 6

[0316] More specifically, such as Figure 18 As shown, d=8, 9 are half of d=4; d=10, 11 are half of d=5; d=12, 13 are half of d=6; d=14, 15 are half of d=7; d=16, 17, 18, 19 are quarter of d=4; d=20, 21, 22, 23 are quarter of d=5… (and so on). Thus, within the range of 1 / k values, calculations are performed by referring to LMDivTable_2N with k consecutive identical values. That is, d / k is used to refer to LMDivTable_2N. Hereinafter, k will be called the scale, and sc = log2(k) will be called the scale shift value.

[0317] Specifically, if expressed as a formula, then by using the derived k through the following formula, refer to the reciprocal table LMDivTable_2N in LMDivTable_2N[d / k] / k(=LMDivTable_2N[d>>sc]>>sc).

[0318] sc=(9-N)-clz(d>>N, 9-N)

[0319] k = 2^(sc)

[0320] It should be noted that the "9" is because the precision (number of bits) of the diff is 512 elements (9 bits). If the precision is different, different values ​​are assigned.

[0321] CCLM prediction unit 31044 derives CCLM prediction parameter a using the value obtained by further shifting the value of table DivTableM by sc and the difference in chromatic difference ChromaDelta, wherein the value of table DivTableM is referenced to the value obtained by shifting the luminance difference d (=diff-1) by a specified scale shift value sc that depends on d (d>>sc).

[0322] a=(ChromaDelta*(LMDivTable_2N[d>>sc]>>sc)+add)>>shiftBb=C_MIN-((a*Y_MIN)>>shiftA)

[0323] Here, add = (shiftB > 0) ? 1 << (shiftB - 1) : 0

[0324] Alternatively, when the derivation of the cyclic correction term add is always constructed with a 1-bit right shift as shown below, a simplified effect is obtained that eliminates the divergence between whether the shift number is positive or not.

[0325] add = (1 << shiftB) >> 1

[0326] It should be noted that the application location of the shift implemented by sc does not depend on the above situation. The CCLM prediction parameter a can also be derived as shown below.

[0327] a=(ChromaDelta*(LMDivTable_2N[d>>sc]+add)>>(shiftB+sc)

[0328] b=C_MIN-((a*Y_MIN)>>shiftA)

[0329] Here, add = (shiftB + sc > 0) ? 1 << (shiftB + sc - 1) : 0

[0330] or

[0331] add=(1<<(shiftB+sc))>>1

[0332] It should be noted that in the above process, the case of d=0 was excluded in the division of 1 / d, and only the table for d>=1 and above was used. That is to say, the table for d=diff-1 was used instead of d=diff, but the table for d=diff could also be used.

[0333] In other words, as shown in LMDivTable_2N[d>>sc] above, by referencing the inverse table LMDivTable_2N based on the index (d>>sc) obtained by shifting the value to reduce its size, the number of elements in LMDivTable_2N is reduced, thus reducing the table size. Furthermore, by further right-shifting the value LMDivTable_2N of the inverse table as shown in LMDivTable_2N[d>>sc]>>sc and (LMDivTable_2N[d>>sc]+add)>>(shiftB+sc), the size is adjusted without degrading performance, achieving the same effect of reducing table size.

[0334] (A variation of Example 3)

[0335] In the above implementation, for a table where d = diff - 1, the number of elements stored in the table is reduced, and the elements that are not retained are derived by calculation, thereby reducing the required memory. In this variation, an example of reducing the required memory by setting a table where diff (0 <= diff <= 511) will be described. Hereinafter, the difference processing for processing example 3 will be described.

[0336] For example, when the number of elements in a table is 2^N and N = 5, Table 1 is established as shown in Table 2.

[0337] [Table 2]

[0338]

[0339]

[0340] Specifically, if expressed as a formula, then using the derived k, refer to the reciprocal table LMDivTable_2N' in LMDivTable_2N'[d / k] / k (=LMDivTable_2N[d>>sc]>>sc) through the following formula. LMDivTable_2N'[] is a table with "0" inserted at the beginning of LMDivTable_2N[] and the last element deleted.

[0341] range = diff >> N

[0342] sc=ShiftTableE”_2N[range+1]

[0343] k = 2^(sc)

[0344] ShiftTableE"_2N[] is a table that inserts "0" before ShiftTableE'_2N[].

[0345] ShiftTableE"_2N[]={0,0,1,2,2,3,3,3,3,4,4,4,4,4,4,4,4,5,5,5,5,5,5,5,5,5,5,5,5,5,5,5,5,5...}

[0346] CCLM prediction unit 31044 derives CCLM prediction parameter a using the value obtained by further shifting the value of table LMDivTable_2N' by sc and the difference in chromatic difference ChromaDelta, wherein the value of table LMDivTable_2N' is referenced by the value obtained by shifting the luminance difference diff by a specified scale shift value sc that depends on diff (diff >> sc).

[0347] a=(ChromaDelta*(LMDivTable_2N'[diff>>sc]>>sc)+add)>>shiftBb=C_MIN-((a*Y_MIN)>>shifiA)

[0348] Here,

[0349] add=(shiftB>0)? 1<<(shiftB-1):0

[0350] or

[0351] add = (1 << shiftB) >> 1

[0352] LMDivTable_2N'[] is a table that inserts "0" at the beginning of LMDivTable_2N'[] and deletes the last element.

[0353] The effect of processing Example 3 is the same as that of the modified example.

[0354] (Processing Example 4)

[0355] The CCLM prediction parameter derivation unit 310442 can combine the processing examples 1 to 3 described above. In this case, the CCLM prediction parameter derivation unit 310442 derives the scale shift value sc corresponding to the difference value (d) of luminance, and derives the value of the table DivTableM_2N (DivTableM_2N[d>>sc]) which is obtained by right-shifting the difference value (d) of luminance by sc. Then, it multiplies the value r obtained by shifting the value exp corresponding to the exponent part corresponding to the above d>>sc with the difference value of color difference, thereby deriving the CCLM prediction parameters. The CCLM prediction parameters (a, b) are derived according to the following formula.

[0356] a=(ChromaDelta*r+add)>>shiftB

[0357] b=MinChromaValue-((a*MinLumaValue)>>shiftA)

[0358] d = diff - 1

[0359] sc = (DN) - clz(d >> N, DN)

[0360] exp=clz(d>>sc,N)+(16-NP-1))

[0361] r=(DivTableM_2N[d>>sc]+offsetM)<<exp>>sc

[0362] add=(shiftB>0)? 1<<(shiftB-1):0

[0363] When using an offset (with the number of elements retained set to 2^N from the beginning of the table), offsetM = 2^(P-1). Without using an offset, offsetM = 0.

[0364] Here,

[0365] D: Represents the range of diff values ​​(1..2^D), from (Example 1) to (Example 3).

[0366] Use D=9

[0367] N: An integer representing log2 of the number of elements in the DivTable, 0 < N <= D

[0368] P: The number of bits in the mantissa of the value expressed by the exponent as an integer multiple of 1 / diff (2^16), where 0 <= P-1 <= 16-N

[0369] It should be noted that the order of application of the scale shift value sc and the exponential shift value exp is not limited to the above. For example, the CCLM prediction parameter derivation unit 310442 derives sc corresponding to the difference value (d) of luminance, and uses the shift value (shiftB+sc-exp) derived from exp and sc to shift the value obtained by multiplying DivTableM_2N[d>>sc] with the difference value of color difference, thereby deriving the CCLM prediction parameter. In this case, DivTableM_2N[d>>sc] uses the value d>>sc obtained by right-shifting the difference value (d) of luminance by sc as an index (feature location) through a table reference.

[0370] a=(ChromaDelta*r+add)>>(shiftB+sc-exp)

[0371] b=MinChromaValue-((a*MinLumaValue)>>shiftA)

[0372] d = diff - 1

[0373] sc = (DN) - clz(d >> N, DN)

[0374] exp=clz(d>>sc,N)+(16-N-(P-1))

[0375] r=(DivTableM_2N[d>>sc]+offsetM)

[0376] add=(shiftB+sc-exp>0)? 1<<(shiftB+sc-exp-1):0

[0377] offsetM = 2^(P-1) or 0

[0378] Alternatively, the ChromaDelta value can be normalized first using the shift value shiftB.

[0379] a=((ChromaDelta+add)>>shiftB)*r>>(sc-exp)

[0380] (Example of table values)

[0381] The following shows an example of table values.

[0382] <Example 1> The case where N=6, P=5, and there is an offset M of 2^(P-1).

[0383] DivTableM_2N

[64] ={0,0,5,0,10,5,2,0,12,10,7,5,4,2,1,0,14,12,11,10,8,7,6,5,4,4,3,2 ,2,1,1,0,15,14,13,12,12,11,10,10,9,8,8,7,7,6,6,5,5,4,4,4,3,3,3,2,2,2,1,1,1,1,0,0}

[0384] ShiftTableE_2N

[64] ={12,11,10,10,9,9,9,9,8,8,8,8,8,8,8,8,7,7,7,7,7,7,7,7,7,7, 7,7,7,7,7,7,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6}

[0385] <Example 1 Variation> Depending on the rounding method, the table can also be as shown below.

[0386] DivTableM_2N

[64] ={0,0,5,0,10,5,2,0,12,10,7,5,4,2,1,0,14,12,11,10,8,7,6,5,4,4,3,2 ,2,1,1,0,15,14,14,13,12,11,11,10,9,9,8,8,7,7,6,6,5,5,4,4,4,3,3,3,2,2,2,1,1,1,1,0}

[0387] ShiftTableE_2N is common to <Example 1>.

[0388] If N=6 and P=5, then the storage required for the storage table is (5-1)*2^6 = 4*64 = 256 (bits). Compared with the above implementation, 256 / 16896 = 1.515%, which can significantly reduce the storage requirement.

[0389] <Example 2> Case where N=5, P=5, and there is an offset of 2^(P-1).

[0390] DivTableM_2N

[32] ={0,0,5,0,10,5,2,0,12,10,7,5,4,2,1,0,14,12,11,10,8,7,6,5,4,4,3,2,2,1,1,0}

[0391] ShiftTableE_2N

[32] ={12,11,10,10,9,9,9,9,8,8,8,8,8,8,8,8,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7}

[0392] At this point, (N=5, P=5, D=9, with offset), if the derivations of a and b are summarized, it will look like the following.

[0393] Let d = diff - 1

[0394] sc=(DN)-clz(d>>N,DN)=(9-5)-clz(d>>5,9-5)=4-clz(d>>5,4)exp=clz (d>>sc, N)+(16-N-(P-1))=clz(d>>sc, 5)+(16-5-(5-1)=clz(d>>sc, 5)+7

[0395] offsetM=2^(P-1)=2^(5-1)=16

[0396] r=(DivTableM_2N[d>>sc]+offsetM)=DiVTableM_2N[d>>sc]+16

[0397] add=(shinB+Sc-exp>0)? 1<<(shiftB+sc-exp-1):0,

[0398] a=(ChromaDelta*r+add)>>(shiftB+sc-exp)

[0399] b=MinChromaValue-((a*MinLumaValue)>>shiftA)

[0400] At this point, if ShiftTableE_2N is used instead of c1z, a and b are calculated as shown below.

[0401] Set as: d=diff-1sc=(DN)-clz(d>>N, DN)=(DN)-((DN)-(16-(P-1)-ShitTableE_2N [d>>5]))=4-(4-(16-4)-ShiftTableE_2N[d>>5]))=12-ShiftTableE_2N[d>>5]

[0402] exp = ShinTableE_2N[d>>5]

[0403] offsetM=2^(P-1)=2^(5-1)=16

[0404] r=(DivTableM_2N[d>>sc]+offsetM)=DiVTableM_2N[d>>sc]+16

[0405] add=(shiftB+sc-exp>0)? 1<<(shiftB+sc-exp-1):0,

[0406] a=(ChromaDelta*r+add)>>(shiftB+sc-exp)

[0407] b=MinChromaValue-((a*MinLumaValue)>>shiftA)

[0408] <Example 2 Variation> Depending on the rounding method, the table can also be as shown below.

[0409] DivTableM_2N

[32] ={0,0,5,0,10,5,2,0,12,10,7,5,4,2,1,0,14,13,11,10,9,8,7,6,5,4,3,3,2,1,1,0}

[0410] ShiftTableE_2N is common to <Example 2>.

[0411] It should be noted that, in the case of no offset (offsetM=0), the value obtained by adding 2^(P-1) to each element of the aforementioned offset DivTableM_2N is stored and used.

[0412] <Example 3> The case where N=6, P=4, and there is an offset of 2^(P-1).

[0413] DivTableM_2N

[64] ={0,0,3,0,5,3,1,0,6,5,4,3,2,1,1,0,7,6,5,5,4,4,3,3,2,2,1,1 ,1,1,0,0,7,7,7,6,6,5,5,5,4,4,4,4,3,3,3,3,2,2,2,2,2,1,1,1,1,1,1,1,0,0,0,0}

[0414] ShiftTableE_2N

[64] ={13,12,11,11,10,10,10,10,9,9,9,9,9,9,9,9,8,8,8,8,8,8,8,8,8, 8,8,8,8,8,8,8,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7}

[0415] <Example 3 Variation> Depending on the rounding method, the table can also be as shown below.

[0416] DivTableM_2N

[32] ={0,0,3,0,5,3,1,0,6,5,4,3,2,1,1,0,7,6,6,5,4,4,3,3,2,2,2,1,1,1,1,0}

[0417] ShiftTab1eE_2N is common to <Example 3>.

[0418] <Example 4> The case where N=5, P=4, and there is an offset of 2^(P-1).

[0419] DivTableM_2N

[32] ={0,0,3,0,5,3,1,0,6,5,4,3,2,1,1,0,7,6,5,5,4,4,3,3,2,2,1,1,1,1,0,0}

[0420] ShiftTableE_2N

[32] ={13,12,11,11,10,10,10,10,9,9,9,9,9,9,9,9,8,8,8,8,8,8,8,8,8,8,8,8,8,8,8,8,}

[0421] <Example 4 Variation> Depending on the rounding method, the table can also be as shown below.

[0422] DivTableM_2N

[32] ={0,0,3,0,5,3,1,0,6,5,4,3,2,1,1,0,7,6,6,5,4,4,3,3,2,2,2,1,1,1,1,0}

[0423] ShiftTableE_2N is common to <Example 4>.

[0424] It should be noted that if N=5 and P=4, the storage required for the storage table is (4-1)*2^5=3*32=96 (bits). Compared with the situation described in the above implementation, 96 / 16896=0.568%, which can significantly reduce the storage requirement.

[0425] The above table can be used in the processing examples 5 and 6 described below.

[0426] (Processing Example 5)

[0427] In Example 5, an example is given where the precision shiftA of parameter a is set to be variable in formula (CCLM-1). Specifically, the precision shiftA of parameter a is derived from the shift value exp derived from the difference value diff of brightness.

[0428] Below, the CCLM prediction parameter derivation unit 310442 derives the term (diffC / diff) which is equivalent to the slope of linear prediction from the difference value of luminance diff and the difference value of chromatic difference diffC, as parameter a (here, the value of parameter a is derived as 2^shiftA (=1<<shiftA) times, and then integerized).

[0429] First, the CCLM prediction parameter derivation unit 310442 derives the number v, which is equivalent to the mantissa of the diff, using the method already explained.

[0430] idx = diff >> sc

[0431] exp=16-(P-1)-ceil(log2(idx))-sc

[0432] msb = 1 << (P-1)

[0433] v = DivTable_2N'[idx]|msb

[0434] Here, P is a specified constant (e.g., P = 4) corresponding to the number of bits in the mantissa of the reciprocal table (1 / diff). Furthermore, exp is a variable that decreases as the brightness difference diff increases (a variable that decreases proportionally to the logarithm of diff), which can be derived from the table. It should be noted that when idx = 0, log2(idx) is set to 0. Here, the value of exp is obtained by subtracting other values ​​from 16 because the reciprocal table is created based on (2^16) / diff. Hereafter, the bit width used as the base for the reciprocal table is denoted as baseA. Without restricting the bit width of parameter a, shiftA = baseA. It should be noted that the maximum value of v is (1 << P) - 1, and the number of bits in v is P bits.

[0435] exp=16-(P-1)-ShiftTableE”_2N[idx]-sc

[0436] In the case of deriving parameter a through the product of diffC and v, CCLM prediction parameter derivation unit 310442 adjusts the number of bits of parameter a by further right shifting shift_a as shown below.

[0437] add = (1 << shift_a) >> 1

[0438] a=(diffC*v+add)>>shift_a

[0439] b=C_Y_MIN-((a*Y_MIN)>>shift_a)

[0440] Here, shift_a is a value derived from the bit depth bitDepth of the image. expC is a constant that limits the bit width of parameter a, for example, using 0 to 3. The bit width of parameter a is the bit width of diffC + the bit width of v - shift_a, which is bitDepth + P - (bitDepth - 8 - expC) = P + 8 - expC, reduced to a small value that does not depend on the bit depth of the image. For example, when P = 4 and expC = 3, the bit width of a is 9 bits.

[0441] shift_a = bitDepth-8-expC

[0442] CCLM prediction parameter derivation unit 310442 adjusts the value of shiftA by subtracting the values ​​exp and expC derived using the brightness difference diff from the initial value of shiftA (=baseA, for example 16).

[0443] shiftA = 16 - (exp + expC)

[0444] The CCLM prediction filter unit 310443 uses formula (CCLM-1) and CCLM prediction parameters (a, b) and the above-adjusted shiftA to output the prediction image predSamples[][].

[0445] In this way, the bit width of a used for the derivation of b or the generation of the image predSamples[][] can be reduced by adaptively deriving the shift amount of a using the brightness difference diff and bit depth bitDepth. This simplifies the product of a and the brightness value refSamples[][] in the formula (CCLM-1).

[0446] In Example 5, the product of diffC*v in the derivation of a is reduced from 11 bits * 16 bits in Example 1 to 11 bits * P bits, thus simplifying the product. When P = 4, 11 bits * 4 bits = 15 bits.

[0447] In Example 5, the product of a*Y_MIN in the derivation of b is reduced from 27 bits * 10 bits in Example 1 to P + 8 - expC bits * 10 bits, thus simplifying the product. When P = 4 and expC = 3, it becomes 9 bits * 4 bits = 13 bits.

[0448] In process 5, the product of the formula (CCLM-1) is reduced from 27 bits * 10 bits in process example 1 to P + 8 - expC bits * 10 bits, thus simplifying the product. When P = 4 and expC = 3, 9 bits * 4 bits = 13 bits.

[0449] It should be noted that the expC in Processing Example 5 is different from that in Processing Example 5 and is a constant that does not depend on the color difference. Therefore, it can also be called expConst, etc., instead of expC.

[0450] (Processing Example 6)

[0451] In Example 6, we will explain an example where the precision shiftA of the parameter a used for the product with the luminance value in the formula (CCLM-1) is set to be variable. Specifically, the upper limit of the number of bits of a (the threshold of a) is set to max_a_bits, and the precision shiftA of the parameter a is derived from diffC, which is the difference in chromatic difference (C_Y_MAX-C_Y_MIN). It should be noted that this can be interpreted as an example of representing the parameter a by raising the mantissa of a fixed number of bits to the power of 2. Hereinafter, max_a_bits represents the number of bits in the mantissa of a, and expC represents the number of bits in the exponent.

[0452] The CCLM prediction parameter derivation unit 310442 hereby derives the parameter a, which is equivalent to the slope (diffC / diff*2^shiftA), from the denominator (diff, the difference in brightness) and the numerator (diffC, the difference in color difference). Hereafter, parameter a is rounded to 2^shiftA (that is, 1 << shiftA) for integerization.

[0453] CCLM prediction parameter derivation unit 310442 first derives idx for referencing the reciprocal table DivTable_2N' compressed from diff, and then derives the number v, which is equivalent to the reciprocal of diff.

[0454] diff = Y_MAX - Y_MIN

[0455] range = (diff >> N) + 1

[0456] sc = ceil(log2(range))

[0457] idx = diff >> sc

[0458] Here, N is a specified constant, such as 5.

[0459] msb = 1 << (P-1)

[0460] v = DivTable_2N'[idx]|msb

[0461] Here, P is the number of bits in the mantissa portion of the reciprocal table (1 / diff) (the portion kept in the DivTable_2N' table), and msb is offsetM.

[0462] CCLM prediction parameter derivation unit 310442 derives the shift value exp corresponding to the difference value diff with brightness.

[0463] exp=16-(P-1)-ceil(log2(diff+1))=16-(P-1)-ceil(log2(idx))-sc

[0464] In addition, exp can be derived by referring to the table.

[0465] exp=16-(P-1)-ShiftTableE”_2N[idx]-sc

[0466] CCLM prediction parameter derivation unit 310442 derives the shift value expC corresponding to the absolute value absDiffC of the color difference value diffC.

[0467] diffC = C_Y_MAX - C_Y_MIN

[0468] absDiffC=(diffC<0?-diffC:diffC)

[0469] rangeDiffC=(absDiffC>>(max_a_bits-P-1))

[0470] expC=ceil(log2(rangeDiffC+1))

[0471] Here, it is also preferable to set the value of max_a_bits to P+1. In this case, rangeDiffC = absDiffC. Therefore, the CCLM prediction parameter derivation section 310442 omits rangeDiffC and derives expC as shown below.

[0472] expC = ceil(log2(absDiffC+1))

[0473] In the case of deriving parameter a by the product of diffC and v, CCLM prediction parameter derivation unit 310442 further derives parameter a, which limits the number of bits, by right shifting diffC*v by expC as shown below.

[0474] add = (1 << expC) >> 1

[0475] a = (diffC * v + add) >> expC

[0476] b=C_Y_MIN-((a*Y_MIN)>>expC) / / shift_a=expC

[0477] The CCLM prediction parameter derivation unit 310442 adjusts the value of shiftA by subtracting exp derived using the denominator (diff value of brightness) and expC derived using the numerator (diff value of color difference) from the initial value of shiftA (e.g., 16).

[0478] shiftA = 16 - (exp + expC)

[0479] For example, when max_a_bits = 5, a is represented with 5-bit precision (-16 to 15). Furthermore, expC is a variable whose absolute value increases as the difference diffC increases (a variable that increases proportionally to the logarithm of absDiffC).

[0480] expC = ceil(log2(rangeDiffC))

[0481] It should be noted that when rangeDiffC = 0, expC is set to 0. expC can be derived by referring to the table.

[0482] expC=ShiftTableE”_2N[rangeDiffC+1]

[0483] The CCLM prediction filter unit 310443 uses formula (CCLM-1) and CCLM prediction parameters (a, b) and the above-adjusted shiftA to output the prediction image predSamples[][].

[0484] In this way, the bit depth of a can be reduced by adaptively deriving the shift amount of a from the color difference diffC. This suppresses the degradation of precision and simplifies the product of a and the luminance value refSamples[][] in the formula (CCLM-1).

[0485] In Example 6, the product of diffC*v in the derivation of a is reduced from 11 bits * 16 bits in Example 1 to 11 bits * P bits, thus simplifying the product. When P = 4, 11 bits * 4 bits = 15 bits.

[0486] In Example 6, the product of a*Y_MIN in the derivation of b is reduced from 27 bits * 10 bits in Example 1 to max_a_bits bits * 10 bits, thus simplifying the product. When max_a_bits = 5, 5 bits * 10 bits = 15 bits.

[0487] In process 6, the product of the formula (CCLM-1) is reduced from 27 bits * 10 bits in process example 1 to max_a_bits bits * 10 bits, thus simplifying the product. When max_a_bits = 5, 5 bits * 10 bits = 15 bits.

[0488] (Processing Example 7)

[0489] The following description will combine the processing examples 1-3 (processing example 4) focusing on table reduction with processing examples 2, 5, and 6 focusing on bit width reduction as processing example 7. Some previously explained content will be omitted, and a brief explanation will be provided. Furthermore, shiftB, which limits the diff to a specified number of bits, is set to 0. The method for deriving the CCLM prediction parameters a and b is shown below. Wherein, when C_Y_MAX = C_Y_MIN, a = 0 and b = C_Y_MIN are set.

[0490] CCLM prediction parameter derivation unit 310442 derives the index idx used to refer to the reciprocal table DivTableM_2N' and the variable exp used to adjust the precision from the brightness difference value diff, and derives the value v which is equivalent to the reciprocal of diff.

[0491] shiftA=baseA=16

[0492] diff = Y_MAX - Y_MIN

[0493] range = diff >> N

[0494] sc = ceil(log2(range+1))

[0495] idx = diff >> sc

[0496] exp=baseA-(P-1)-ceil(log2(idx))-sc

[0497] msb = 1 << (P-1)

[0498] v = (DivTableM_2N'[idx]|msb)

[0499] Here, baseA is the number of bits used as the base (e.g., 16) when deriving the reciprocal table DivTableM_2N'[idx], N is a constant corresponding to the number of elements in DivTableM_2N' (2^N), P is a constant corresponding to the number of bits in the mantissa of the exponential representation of (2^16 / diff) in base 2, and max_a_bits is a constant corresponding to the number of bits in the mantissa of the exponential representation of a in base 2. Msb is also a constant. For example, N = 5, P = 4, max_a_bits = 5, msb = 2^(P-1) = 8 can be used. An example of DivTableM_2N'[] with msb = 2^(P-1) (with offset) is shown below.

[0500] DivTableM_2N'

[32] ={0,0,0,3,0,5,3,1,0,6,5,4,3,2,1,1,0,7,6,5,5,4,4,3,3,2,2,1,1,1,1,0}

[0501] CCLM prediction parameter derivation unit 310442 derives the variable expC used to limit the bit width of parameter a from the difference value diffC of the color difference.

[0502] max_a_bits = 5

[0503] diffC = C_Y_MAX - C_Y_MIN

[0504] absDiffC=(diffC<0?-diffC:diffC)

[0505] rangeDiffC=(absDiffC>>(max_a_bits-P-1))

[0506] expC=ceil(log2(rangeDiffC+1))

[0507] shift_a=expC

[0508] CCLM prediction parameter derivation unit 310442 further derives the parameter a, which limits the number of bits, by right-shifting diffC*v by shift_a as described below.

[0509] add = 1 << shift_a >> 1

[0510] a=(diffC*v+add)>>shift_a

[0511] In this configuration, shift_a = expC, so the derivations of a and b can be replaced by the following formula.

[0512] add = (1 << expC) >> 1

[0513] a = (diffC * v + add) >> expC

[0514] Therefore, the bit width (precision) of a can be kept constant regardless of the size of diffC.

[0515] CCLM prediction parameter derivation unit 310442 adjusts the value of shiftA by subtracting the exp derived using diff and the expC derived using diffC from the initial value of shiftA (e.g., 16).

[0516] shiftA - = exp + expC

[0517] Furthermore, with an initial value of 16, shiftA can also be derived as shown below.

[0518] shiftA = 16 - exp - expC

[0519] Furthermore, since the initial value baseA of shiftA is also used in the derivation of expC and cancels each other out, shiftA can be directly derived using constants P, idx, and rangeDiffC as a result.

[0520] shiftA=baseA-(baseA-(P-1)-ceil(log2(idx))-sc)-ceil(log2(rangeDiffC+1))

[0521] =P-1+ceil(log2(idx))+sc-ceil(log2(rangeDiffC+1))

[0522] CCLM prediction parameter derivation unit 310442 derives parameter b using parameter a, which limits the bit width.

[0523] b=C_Y_MIN-((a*Y_MIN)>>shiftA)

[0524] The CCLM prediction filter unit 310443 uses formula (CCLM-1) and CCLM prediction parameters (a, b) and the above-adjusted shiftA to output the prediction image predSamples[][].

[0525] It should be noted that, as shown in Processing Example 6, expC can be set to a specified constant independently of diffC. However, expC is less than the value obtained by adding P (the number of bits in the mantissa of (2^16 / diff)) to the bit depth bitDepth of the image. For example, in the case of bitDepth = 10 and P = 4, expC is set to be less than 14.

[0526] expC < bitDepth + P

[0527] Therefore, the precision of 'a' is slightly reduced, but the processing is simplified.

[0528] Based on the above configuration, the effects described in Processing Examples 1 to 3, 5, and 6 are obtained. If the main effects are described again, at least the following effects are obtained.

[0529] As illustrated in Example 3, the simplification effect of table size reduction achieved by referencing the table via diff >> sc can be obtained.

[0530] It can achieve the simplification effect of the product achieved by reducing the bit width, as illustrated in Examples 5 and 6.

[0531] (Other examples)

[0532] It should be noted that the above processing example illustrates the reduction of storage space required for storing tables used in CCLM processing, but the technical concept of this invention can also be used to reduce storage space required for storing other information and the bit width of multiplication. For example, it can also be applied to tables used for deriving the converted motion vectors.

[0533] One aspect of the present invention is a CCLM prediction unit that generates a prediction image through CCLM prediction. It is characterized by comprising: a CCLM prediction parameter derivation unit that derives CCLM prediction parameters (a, b) using luminance difference values, chrominance difference values, and a table; and a CCLM prediction filtering unit that generates a chrominance prediction image using a luminance reference image and the CCLM prediction parameters (a, b). The CCLM prediction parameter derivation unit derives the CCLM prediction parameter a by shifting the value obtained by multiplying the elements of the table referenced by the luminance difference values ​​with the chrominance difference values.

[0534] The CCLM prediction unit of one embodiment of the present invention is characterized in that the CCLM prediction parameter derivation unit derives the difference value of brightness from the first pixel with the largest brightness value in the adjacent block and the second pixel with the smallest brightness value in the adjacent block, derives the difference value of color difference from the pixel value of the color difference between the first pixel and the second pixel, derives the scale shift value sc corresponding to the difference value of brightness, multiplies the value of the table with the difference value of color difference, wherein the value of the table is referenced by the index idx after the difference value of brightness is shifted right by sc, and the CCLM prediction parameter a is derived by further shifting the value obtained by multiplication.

[0535] The CCLM prediction unit of one embodiment of the present invention is characterized in that the CCLM prediction parameter derivation unit multiplies the value obtained by adding an offset to the value of the table referenced by idx with the difference value of color difference.

[0536] The CCLM prediction unit of one embodiment of the present invention is characterized in that the CCLM prediction parameter derivation unit derives a first shift value corresponding to the logarithm of the absolute difference value of the color difference, multiplies the value of the table referenced by idx with the difference value of the color difference, and further shifts the value obtained by multiplication by a shift value expC to derive the CCLM prediction parameter a.

[0537] The CCLM prediction unit of one embodiment of the present invention is characterized in that the CCLM prediction parameter derivation unit derives a second shift value corresponding to the logarithm of the difference value diff of the brightness, and derives the CCLM prediction parameter b using the color difference value of the second pixel, the CCLM prediction parameter a, the brightness value of the second pixel, and the first shift value and the second shift value.

[0538] The CCLM prediction unit of one embodiment of the present invention is characterized in that the first shift value and the second shift value are derived by referring to a table.

[0539] (Hardware implementation and software implementation)

[0540] Furthermore, each of the aforementioned motion picture decoding device 31 and motion picture encoding device 11 can be implemented in hardware using logic circuits formed on an integrated circuit (IC chip), or in software using a CPU (Central Processing Unit).

[0541] In the latter case, the aforementioned devices include: a CPU that executes commands for programs that perform various functions; a ROM (Read Only Memory) that stores the programs; a RAM (Random Access Memory) that expands the programs; and a memory that stores the programs and various data, etc., such as a storage device (recording medium). Furthermore, the objective of embodiments of the present invention is to achieve this by supplying a recording medium containing program code (executable form program, intermediate code program, source program) of the software implementing the aforementioned functions, i.e., the control program of the aforementioned devices, in a computer-readable manner to the aforementioned devices, wherein the computer (or CPU, MPU) reads the program code recorded on the recording medium and executes it.

[0542] As recording media, the following can be used: magnetic tapes, cassette tapes, etc.; disks including floppy disks (registered trademark) / hard disks, CD-ROMs (Compact Disc Read-Only Memory), MO discs (Magneto-Optical Disc), MD discs (Mini Disc), DVDs (Digital Versatile Disc), CD-Rs (CD Recordable), Blu-ray discs (Blu-ray Disc); cards (including memory cards) / optical cards; semiconductor memories such as mask ROMs / EPROMs (Erasable Programmable Read-Only Memory), EEPROMs (Electrically Erasable and Programmable Read-Only Memory), and flash memory ROMs; or PLDs (Programmable Logic Devices), FPGAs (Field Programmable Gate Arrays). Array (Field Programmable Gate Array) and other logic circuits.

[0543] Furthermore, the aforementioned devices can be configured to connect to a communication network and supply the program code via the communication network. This communication network need only be capable of transmitting program code and is not particularly limited. For example, the Internet, intranet, extranet, LAN (Local Area Network), ISDN (Integrated Services Digital Network), VAN (Value-Added Network), CATV (Community Antenna television / Cable Television) communication network, Virtual Private Network, telephone line network, mobile communication network, satellite communication network, etc., can be used. Furthermore, the transmission medium constituting this communication network need only be a medium capable of transmitting program code and is not limited to a specific configuration or type. For example, it can be used in wired networks such as IEEE (Institute of Electrical and Electronic Engineers) 1394, USB, power line transmission, cable TV lines, telephone lines, and ADSL (Asymmetric Digital Subscriber Line) lines, as well as in wireless networks such as IrDA (Infrared Data Association), infrared (like remote controls), Bluetooth (registered trademark), IEEE 802.11 wireless, HDR (High Data Rate), NFC (Near Field Communication), DLNA (Digital Living Network Alliance, registered trademark), portable telephone networks, satellite lines, and terrestrial digital broadcasting networks. It should be noted that the embodiments of the present invention can also be implemented in the form of computer data signals embedded in a carrier wave, which embody the above-mentioned program code through electronic transmission.

[0544] The embodiments of the present invention are not limited to the embodiments described above, and various modifications can be made within the scope of the claims. That is, embodiments obtained by combining technical solutions with appropriate modifications within the scope of the claims are also included within the technical scope of the present invention.

[0545] (Mutual references between related applications)

[0546] This application claims priority to Japanese Patent Application No. 2018-238526, filed on December 20, 2018, and incorporates the entire contents of which hereinafter by reference.

[0547] Explanation of reference numerals in the attached figures

[0548] 31 Image Decoding Device

[0549] 301 Entropy Decoding Department

[0550] 302 Parameter Decoding Unit

[0551] 303 Inter-frame Prediction Parameter Decoding Unit

[0552] 304 Intra-frame Prediction Parameter Decoding Unit

[0553] 308 Predictive Image Generation Unit

[0554] 309 Inter-frame Prediction Image Generation Unit

[0555] 310 Intra-frame Prediction Image Generation Unit

[0556] 3104 Forecasting Department

[0557] 31044 CCLM Prediction Unit (Prediction Image Generation Device)

[0558] 310441 Downsampling Unit

[0559] 310442 CCLM Prediction Parameter Derivation Section (Parameter Derivation Section)

[0560] 310443 CCLM Predictive Filtering Unit

[0561] 311 Inverse quantization / inverse transform unit

[0562] 312 Addition Department

[0563] 11 Image encoding device

[0564] 101 Predictive Image Generation Unit

[0565] 102 Subtraction Section

[0566] 103 Transformation / Quantization Section

[0567] 104 Entropy Coding Department

[0568] 105 Inverse Quantization / Inverse Transformation Unit

[0569] 107 Loop Filter

[0570] 110 Encoding Parameter Determination Unit

[0571] 111 Parameter Encoding Section

[0572] 112 Inter-frame Prediction Parameter Coding Unit

[0573] 113 Intra-frame Prediction Parameter Coding Unit.

Claims

1. A predictive image generation apparatus for generating a predictive image based on intra-frame prediction parameters including a cross-component linear model (CCLM) prediction mode, the predictive image generation apparatus comprising: The CCLM prediction parameter derivation unit is configured to derive (i) the CCLM prediction parameters of the first cross-component linear model and (ii) the CCLM prediction parameters of the second cross-component linear model. as well as The CCLM predictive filter is configured to derive the predicted image by adding the value of the second CCLM prediction parameter to a value derived by right-shifting the product of the first luminance sample value and the value of the first CCLM prediction parameter by a first shift value. in, The first CCLM prediction parameter is derived by right-shifting a first variable by a second shift value, wherein the first variable is the sum of the second variable and a rounding parameter used to derive the first CCLM prediction parameter, wherein the second variable is the product of: (i) the difference in color difference between the first and second color difference sample values, and (ii) the value indicated by the index in the reference table. The second shift value is derived by adding the logarithm of the absolute value of the difference between the predetermined value and the color difference. The second CCLM prediction parameter is derived by subtracting a third variable from the second color difference sample value, wherein the third variable is derived by using (i) the product of the value of the first CCLM prediction parameter and the second luminance sample value, and (ii) the second shift value. The first shift value is derived by using (i) the second shift value and (ii) the difference in brightness between the third brightness sample value and the second brightness sample value.

2. A video decoding apparatus for decoding encoded data, the video decoding apparatus comprising: The CCLM prediction parameter derivation unit is configured to derive (i) the CCLM prediction parameters of the first cross-component linear model and (ii) the CCLM prediction parameters of the second cross-component linear model. as well as The CCLM predictive filter is configured to derive a predicted image by adding the value of the second CCLM prediction parameter to a value derived by right-shifting the product of the first luminance sample value and the value of the first CCLM prediction parameter by a first shift value. in, The first CCLM prediction parameter is derived by right-shifting a first variable by a second shift value, wherein the first variable is the sum of the second variable and a rounding parameter used to derive the first CCLM prediction parameter, wherein the second variable is the product of: (i) the difference in color difference between the first and second color difference sample values, and (ii) the value indicated by the index in the reference table. The second shift value is derived by adding the logarithm of the absolute value of the difference between the predetermined value and the color difference. The second CCLM prediction parameter is derived by subtracting a third variable from the second color difference sample value, wherein the third variable is derived by using (i) the product of the value of the first CCLM prediction parameter and the second luminance sample value, and (ii) the second shift value. The first shift value is derived by using (i) the second shift value and (ii) the difference in brightness between the third brightness sample value and the second brightness sample value.

3. A video encoding apparatus for encoding video images, the video encoding apparatus comprising: The CCLM prediction parameter derivation unit is configured to derive (i) the CCLM prediction parameters of the first cross-component linear model and (ii) the CCLM prediction parameters of the second cross-component linear model. as well as The CCLM predictive filter is configured to derive a predicted image by adding the value of the second CCLM prediction parameter to a value derived by right-shifting the product of the first luminance sample value and the value of the first CCLM prediction parameter by a first shift value. in, The first CCLM prediction parameter is derived by right-shifting a first variable by a second shift value, wherein the first variable is the sum of the second variable and a rounding parameter used to derive the first CCLM prediction parameter, wherein the second variable is the product of: (i) the difference in color difference between the first and second color difference sample values, and (ii) the value indicated by the index in the reference table. The second shift value is derived by adding the logarithm of the absolute value of the difference between the predetermined value and the color difference. The second CCLM prediction parameter is derived by subtracting a third variable from the second color difference sample value, wherein the third variable is derived by using (i) the product of the value of the first CCLM prediction parameter and the second luminance sample value, and (ii) the second shift value. The first shift value is derived by using (i) the second shift value and (ii) the difference in brightness between the third brightness sample value and the second brightness sample value.

4. A method for deriving the motion vector of each of a plurality of sub-blocks, the method comprising: Derive (i) the CCLM prediction parameters of the first cross-component linear model and (ii) the CCLM prediction parameters of the second cross-component linear model; as well as The predicted image is derived by adding the value of the second CCLM prediction parameter to a value derived by right-shifting the product of the first luminance sample value and the value of the first CCLM prediction parameter by a first shift value. in, The first CCLM prediction parameter is derived by right-shifting a first variable by a second shift value, wherein the first variable is the sum of the second variable and a rounding parameter used to derive the first CCLM prediction parameter, wherein the second variable is the product of: (i) the difference in color difference between the first and second color difference sample values, and (ii) the value indicated by the index in the reference table. The second shift value is derived by adding the logarithm of the absolute value of the difference between the predetermined value and the color difference. The second CCLM prediction parameter is derived by subtracting a third variable from the second color difference sample value, wherein the third variable is derived by using (i) the product of the value of the first CCLM prediction parameter and the second luminance sample value, and (ii) the second shift value. The first shift value is derived by using (i) the second shift value and (ii) the difference in brightness between the third brightness sample value and the second brightness sample value.

Images