Image Encoding / Decoding Apparatus and Method

Abstract

The present invention provides an image decoding device and an image encoding device, which can eliminate the combination of transform skip and sign data hiding and quantization-dependent, and achieve the effect of reducing the installation costs of hardware and software. An image decoding device includes: a header decoding unit that decodes from encoded data a flag indicating whether quantization-dependent can be used and a flag prohibiting transform skip prediction error quantization; and a TU decoding unit that decodes transform coefficients in an RRC mode in which the decoding start position, i.e., the LAST position, of the transform coefficients within a TU block is encoded or in a TSRC mode in which the LAST position is not encoded. When the TU decoding unit decodes transform coefficients of transform skip in the TSRC mode, quantization-dependent is performed in the RRC mode. When the TU decoding unit decodes transform coefficients of transform skip in the RRC mode, quantization-dependent is not performed in the RRC mode.

Description

Technical Field

[0001] Embodiments of the present invention relate to an image decoding apparatus and an image encoding apparatus. Background Art

[0002] In order to efficiently transmit or record an image, an image encoding apparatus that generates encoded data by encoding the image and an image decoding apparatus that generates a decoded image by decoding the encoded data are used.

[0003] As a specific image encoding method, for example, H.264 / AVC (Advanced Video Coding), HEVC (High-Efficiency Video Coding), etc. can be cited.

[0004] In such an image encoding method, images (pictures) constituting a moving image are managed by a hierarchical structure, and are encoded / decoded for each CU. The hierarchical structure includes slices obtained by dividing the image, coding tree units (CTUs: Coding Tree Units) obtained by dividing the slices, coding units (sometimes also referred to as coding units (Coding Units: CUs)) obtained by dividing the coding tree units, and transform units (TUs: Transform Units) obtained by dividing the coding units.

[0005] In addition, in such an image encoding method, a prediction image is usually generated based on a partial decoded image obtained by encoding / decoding an input image, and a prediction error (sometimes also referred to as a "differential image" or a "residual image") obtained by subtracting the prediction image from the input image (original image) is encoded. As a method for generating a prediction image, inter-picture prediction (inter-frame prediction) and intra-picture prediction (intra-frame prediction) can be cited.

[0006] In addition, as recent image encoding and decoding techniques, Non-Patent Document 1 can be cited. Non-Patent Document 1 discloses "Sign Data Hiding (SDH)", which is a technique for estimating the positive or negative signs of some transform coefficients without encoding them for the transform coefficients calculated by inverse quantization of the quantization transform coefficients related to the prediction error in order to improve the encoding efficiency. In addition, "Dependent Quantization (DQ)", which switches between two quantizers with different bit depths for quantization and inverse quantization, is disclosed.

[0007] On the other hand, in applications that use images in the medical field, artworks, etc., there are times when a decoded image with no or substantially no degradation caused by encoding is required. To achieve such lossless or near-lossless, a technique that does not use transformation (inverse transformation) or quantization (inverse quantization) in encoding (decoding) has been disclosed. One of these techniques is called "Transform Skip (TS)". Transform Skip is also regarded as one type of transformation (Identical Transform), and whether it is Transform Skip is encoded as the type of transformation.

[0008] In addition, "Transform Skip Residual Coding (TSRC)", which performs encoding different from that of normal prediction error (Regular Residual Coding, RRC) in Transform Skip, is also known.

[0009] Prior Art Documents

[0010] Non-Patent Documents

[0011] Non-Patent Document 1: "Versatile Video Coding (Draft 8)", JVET-Q2001, Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11, 7 - 17 January 2020 Summary of the Invention

[0012] Problems to be Solved by the Invention

[0013] In the technology of Non-Patent Document 1, as a combination of Transform Skip, symbol data hiding, dependency quantization, Transform Skip Residual Quantization (TSRC), and encoding of normal prediction error (RRC), symbol data hiding and dependency quantization are not performed in Transform Skip Residual Quantization (TSRC). Thus, Transform Skip, symbol data hiding, and dependency quantization are excluded. However, in the encoding of normal prediction error (RRC), there is a combination of Transform Skip with symbol data hiding and dependency quantization, so there is still a problem that the image decoding device and the image encoding device are complex. In particular, there is a problem that the installation and cost increase of the hardware and software for combinations with small implementation effects are disproportionate.

[0014] Technical Solution

[0015] To solve the above problems, an image decoding apparatus according to one aspect of the present invention is characterized by comprising: a header decoding unit that decodes from encoded data a flag ph_dep_quant_enabled_flag indicating whether dependent quantization can be used and a flag slice_ts_residual_coding_disabled_flag that prohibits conversion skip prediction error quantization; and a TU decoding unit that decodes transform coefficients in an RRC mode that encodes the decoding start position, i.e., the LAST position, of the transform coefficients within a TU block (block) or in a TSRC mode that does not encode the LAST position. When the TU decoding unit decodes transform coefficients with conversion skip in the TSRC mode (slice_ts_residual_coding_disabled_flag = 0), it performs dependent quantization in the RRC mode. When the TU decoding unit decodes transform coefficients with conversion skip in the RRC mode (slice_ts_residual_coding_disabled_flag = 1), it does not perform dependent quantization in the RRC mode.

[0016] The above image decoding apparatus is characterized by comprising: a header decoding unit that decodes from encoded data a flag ph_dep_quant_enabled_flag indicating whether dependent quantization can be used and a flag slice_ts_residual_coding_disabled_flag that prohibits conversion skip prediction error quantization; and a TU decoding unit that decodes transform coefficients. When ph_dep_quant_enabled_flag is 1 and slice_ts_residual_coding_disabled_flag is 0, the TU decoding unit performs dependent quantization and decodes the transform coefficients. In other cases, when ph_dep_quant_enabled_flag is 0 or slice_ts_residual_coding_disabled_flag is 1, the TU decoding unit does not perform dependent quantization but decodes the transform coefficients.

[0017] The above-described image decoding apparatus is characterized by including: a header decoding unit that decodes from the encoded data a flag ph_dep_quant_enabled_flag indicating whether dependent quantization can be used, a flag pic_sign_data_hiding_enabled_flag indicating whether symbol data hiding can be used, and a flag slice_ts_residual_coding_disabled_flag that prohibits the conversion skip of predictive error quantization; and a TU decoding unit that decodes transform coefficients. When ph_dep_quant_enabled_flag is 1, or pic_sign_data_hiding_enabled_flag is 0, or slice_ts_residual_coding_disabled_flag is 1, the TU decoding unit does not perform symbol data hiding.

[0018] By adopting such a configuration, the combination of conversion skip, symbol data hiding, and dependent quantization can be eliminated, and the effect of reducing the installation costs of hardware and software can be achieved.

[0019] Advantageous Effects

[0020] According to the above configuration, any of the above problems can be solved. Description of the Drawings

[0021] Figure 1 It is a schematic diagram showing the configuration of the image transmission system of the present embodiment.

[0022] Figure 2 It is a diagram showing the configurations of a transmission device equipped with the moving image encoding apparatus of the present embodiment and a receiving device equipped with the moving image decoding apparatus. PROD_A represents the transmission device equipped with the moving image encoding apparatus, and PROD_B represents the receiving device equipped with the moving image decoding apparatus.

[0023] Figure 3 It is a diagram showing the configurations of a recording device equipped with the moving image encoding apparatus of the present embodiment and a reproducing device equipped with the moving image decoding apparatus. PROD_C represents the recording device equipped with the moving image encoding apparatus, and PROD_D represents the reproducing device equipped with the moving image decoding apparatus.

[0024] Figure 4 It is a diagram showing the hierarchical structure of the data of the encoded stream.

[0025] Figure 5 It is a diagram showing an example of the division of CTUs.

[0026] Figure 6 It is a schematic diagram showing the configuration of the moving image decoding apparatus.

[0027] Figure 7 It is a flowchart for explaining the general operation of a moving image decoding device.

[0028] Figure 8 It is a schematic diagram showing the configuration of a TU decoding unit.

[0029] Figure 9 It is a diagram showing syntax elements related to quantization conversion coefficients.

[0030] Figure 10 It is a diagram showing syntax elements related to quantization conversion coefficients.

[0031] Figure 11a It is a syntax table related to RRC.

[0032] Figure 11b It is a syntax table related to RRC.

[0033] Figure 11c It is a syntax table related to RRC.

[0034] Figure 12a It is a syntax table related to TSRC.

[0035] Figure 12b It is a syntax table related to TSRC.

[0036] Figure 13 It is a syntax table related to the slice header of this embodiment.

[0037] Figure 14a It is a syntax table related to RRC of this embodiment.

[0038] Figure 14b It is a syntax table related to RRC of this embodiment.

[0039] Figure 14c It is a syntax table related to RRC of this embodiment.

[0040] Figure 15 It is a syntax table related to TU of this embodiment.

[0041] Figure 16a It is a syntax table related to RRC of this embodiment.

[0042] Figure 16b It is a syntax table related to RRC of this embodiment.

[0043] Figure 16c It is a syntax table related to RRC of this embodiment.

[0044] Figure 17It is a block diagram showing the configuration of a moving image encoding device. Detailed implementation

[0045] [Embodiment 1]

[0046] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings.

[0047] Figure 1 It is a schematic diagram showing the configuration of the image transmission system 1 of the present embodiment.

[0048] The image transmission system 1 is a system that transmits an encoded stream obtained by encoding an image to be encoded, decodes the transmitted encoded stream, and displays the image. The image transmission system 1 is configured to include: a moving image encoding device (image encoding device) 11, a network 21, a moving image decoding device (image decoding device) 31, and an image display device (image display device) 41.

[0049] The moving image encoding device 11 is input with an image T.

[0050] The network 21 transmits the encoded stream Te generated by the moving image encoding device 11 to the moving image decoding device 31. The network 21 is the Internet, a wide area network (WAN: Wide Area Network), a local area network (LAN: Local Area Network), or a combination thereof. The network 21 is not necessarily limited to a two-way communication network, and may also be a one-way communication network that transmits broadcast waves such as digital terrestrial broadcasting and satellite broadcasting. In addition, the network 21 may be replaced by a storage medium such as a DVD (Digital Versatile Disc, registered trademark) or a BD (Blue-ray Disc, registered trademark) that stores the encoded stream Te.

[0051] The moving image decoding device 31 decodes the encoded stream Te transmitted by the network 21 respectively, and generates one or more decoded images Td after decoding.

[0052] The image display device 41 displays all or part of one or more decoded images Td generated by the moving image decoding device 31. The image display device 41 includes, for example, a display device such as a liquid crystal display or an organic EL (Electro-luminescence) display. As the form of the display, a fixed type, a mobile type, an HMD, etc. can be cited. In addition, when the moving image decoding device 31 has high processing power, an image with high image quality is displayed, and when it only has low processing power, an image that does not require high processing power and high display ability is displayed.

[0053] <Operator>

[0054] The following describes the operators used in this specification.

[0055] >>>> represents a right shift, <<< represents a left shift, & represents a bitwise AND, | represents a bitwise OR, |= represents an OR assignment operator, and || represents a logical OR.

[0056] x? y : z is a ternary operator that takes y when x is true (non-zero) and takes z when x is false (0).

[0057] Clip3(a, b, c) is a function that clips c to a value between a and b (inclusive). It returns a if c < a, b if c > b, and c otherwise (where a <= b).

[0058] abs(a) is a function that returns the absolute value of a.

[0059] Int(a) is a function that returns the integer value of a.

[0060] floor(a) is a function that returns the largest integer less than or equal to a.

[0061] ceil(a) is a function that returns the smallest integer greater than or equal to a.

[0062] a / d represents the division of a by d (truncating the fractional part).

[0063] <Structure of the encoded stream Te>

[0064] Before describing the moving image encoding device 11 and the moving image decoding device 31 of the present embodiment in detail, the data structure of the encoded stream Te generated by the moving image encoding device 11 and decoded by the moving image decoding device 31 will be described.

[0065] Figure 4 It is a diagram showing the hierarchical structure of the data in the encoded stream Te. The encoded stream Te illustratively includes a sequence and a plurality of pictures constituting the sequence. In Figure 4 it shows diagrams respectively representing the encoded video sequence of a given sequence SEQ, the encoded picture of a specified picture PICT, the encoded slice of a specified slice S, the encoded slice data of the specified slice data, the encoded tree units included in the encoded slice data, and the encoded units included in the encoded tree units.

[0066] (Encoded video sequence)

[0067] In the encoded video sequence, a set of data for the moving image decoding device 31 to refer to for decoding the sequence SEQ to be processed is defined. The sequence SEQ is as Figure 4As shown in the encoded video sequence, it includes a Video Parameter Set, a Sequence Parameter Set (SPS), a Picture Parameter Set (PPS), a picture header, a picture PICT, and Supplemental Enhancement Information (SEI).

[0068] The Video Parameter Set (VPS) specifies a set of encoding parameters common to multiple images in an image composed of multiple layers, and a set of encoding parameters for the multiple layers included in the image and associated with each layer.

[0069] In the Sequence Parameter Set (SPS), a set of encoding parameters for the moving image decoding device 31 to refer to for decoding the target sequence is specified. For example, the width and height of a picture are specified. It should be noted that multiple SPSs can exist. In this case, any one of the multiple SPSs is selected from the PPS.

[0070] In the Picture Parameter Set (PPS), a set of encoding parameters for the moving image decoding device 31 to refer to for each picture in the target sequence is specified. For example, it includes a reference value for the quantization width for picture decoding (pic_init_qp_minus26), a flag indicating the application of weighted prediction (weighted_pred_flag), and a scaling list (quantization matrix). It should be noted that multiple PPSs can exist. In this case, any one of the multiple PPSs is selected from each picture in the target sequence.

[0071] In the picture header, encoding parameters common to all slices included in one encoded picture are defined. For example, it includes encoding parameters related to POC (Picture Order Count) and segmentation.

[0072] (Encoded picture)

[0073] In the encoded picture, a set of data for the moving image decoding device 31 to refer to for decoding the picture PICT to be processed is specified. As shown in the encoded picture of Figure 4 it includes slices 0 to slice NS - 1 (NS is the total number of slices included in the picture PICT).

[0074] It should be noted that hereinafter, when it is not necessary to distinguish each of slices 0 to slice NS - 1, the code subscripts may sometimes be omitted in the description. The same also applies to the data included in the following-described encoded stream Te and other data with subscripts marked.

[0075] (Coded slice)

[0076] In the coded slice, a set of data for a moving picture decoding device 31 to refer to for decoding a slice S to be processed is defined. As shown in the coded slice of Figure 4 , it includes a slice header and slice data.

[0077] The slice header includes a set of coding parameters for a moving picture decoding device 31 to refer to for determining a decoding method of an object slice. The slice type specification information (slice_type) that specifies the slice type is an example of the coding parameters included in the slice header.

[0078] As slice types that can be specified by the slice type specification information, the following can be cited: (1) an I slice that uses only intra prediction during coding, (2) a P slice that uses uni-directional prediction or intra prediction during coding, and (3) a B slice that uses uni-directional prediction, bi-directional prediction, or intra prediction during coding, etc. It should be noted that inter prediction is not limited to uni-directional prediction and bi-directional prediction, and more reference pictures can also be used to generate a predicted image. Hereinafter, the case of referring to a P slice and a B slice means a slice including a block that can use inter prediction.

[0079] It should be noted that the slice header may also include a reference to a picture parameter set PPS (pic_parameter_set_id).

[0080] (Coded slice data)

[0081] In the coded slice data, a set of data for a moving picture decoding device 31 to refer to for decoding the slice data to be processed is defined. As shown in the coded slice header of Figure 4 , it includes CTUs. A CTU is a block of a fixed size (e.g., 64×64) that constitutes a slice, and is sometimes also referred to as the largest coding unit (LCU: Largest Coding Unit).

[0082] (Coding tree unit)

[0083] In Figure 4In the coding tree unit, a set of data is defined for the moving image decoding device 31 to refer to for decoding the CTU to be processed. The CTU is divided into coding units CU, which are the basic units of coding processing, by recursive quadtree partitioning (QT (Quad Tree) partitioning), binary tree partitioning (BT (Binary Tree) partitioning), or ternary tree partitioning (TT (Ternary Tree) partitioning). The BT partitioning and TT partitioning are collectively referred to as multi-tree partitioning (MT (Multi Tree) partitioning). The nodes of the tree structure obtained by recursive quadtree partitioning are called coding nodes. The intermediate nodes of the quadtree, binary tree, and ternary tree are coding nodes, and the CTU itself is defined as the uppermost coding node.

[0084] CT (Coding Tree) includes a partitioning flag indicating whether QT partitioning is performed as CT information. Figure 5 Examples of partitioning are shown in

[0085] The CU is the terminal node of the coding node and is not further partitioned. The CU is the basic unit of coding processing.

[0086] (Coding Unit)

[0087] As Figure 4 shown in the coding unit of

[0088] a set of data is defined for the moving image decoding device 31 to refer to for decoding the coding unit to be processed. Specifically, the CU consists of a CU header CUH, prediction parameters, transform parameters, quantized transform coefficients, etc. The prediction mode, etc. is defined in the CU header.

[0089] The types of prediction (prediction mode CuPredMode) at least include two types: intra-frame prediction (MODE_INTRA) and inter-frame prediction (MODE_INTER). Moreover, intra-block copy prediction (MODE_IBC) can also be included. Intra-frame prediction and intra-block copy prediction are predictions within the same picture, and inter-frame prediction refers to prediction processing between different pictures (for example, between display times, between picture layers).

[0090] The conversion / quantization process is performed on a CU basis, and the quantization conversion coefficients can also be entropy-coded on a sub-block basis such as 4×4.

[0091] The predicted image is derived from the prediction parameters attached to the block. Among the prediction parameters, there are prediction parameters for intra-frame prediction and inter-frame prediction.

[0092] (Configuration of the moving image decoding device)

[0093] The configuration of the moving image decoding device 31 of the present embodiment ( Figure 6 ) will be described.

[0094] The moving image decoding device 31 is configured to include: an entropy decoding unit 301, a parameter decoding unit (predicted image decoding device) 302, a loop filter 305, a reference picture memory 306, a prediction parameter memory 307, a predicted image generation unit (predicted image generation device) 308, an inverse quantization / inverse conversion unit 311, and an addition unit 312. It should be noted that, in cooperation with the moving image encoding device 11 described later, there is also a configuration in which the moving image decoding device 31 does not include the loop filter 305.

[0095] The parameter decoding unit 302 further includes: a header decoding unit 3020, a CT information decoding unit 3021, and a CU decoding unit 3022 (prediction mode decoding unit), and the CU decoding unit 3022 includes a TU decoding unit 3024. These may be collectively referred to as a decoding module. The header decoding unit 3020 decodes parameter set information such as VPS, SPS, and PPS, and a slice header (slice information) from the encoded data. The CT information decoding unit 3021 decodes CT from the encoded data. The CU decoding unit 3022 decodes CU from the encoded data. The TU decoding unit 3024 decodes QP update information (quantization correction value) and quantization conversion coefficients (residual_coding) from the encoded data.

[0096] When the TU decoding unit 3024 includes a prediction error in the TU, it decodes QP update information and quantization conversion coefficients from the encoded data. The derivation of the quantization conversion coefficients can also have multiple modes (for example, the RRC mode and the TSRC mode). Specifically, different processes can be implemented by using the derivation of the normal prediction error of the conversion (RRC: Regular Residual Coding) and the derivation of the prediction error in the transform skip mode without using the conversion (TSRC: Transform Skip Residual Coding). The QP update information is the difference value from the predicted value of the quantization parameter QP, that is, the quantization parameter predicted value qPpred.

[0097] In addition, examples of using CTUs and CUs as processing units are described below, but this is not limited to this example, and processing can also be performed in units of sub-CUs. Alternatively, CTUs and CUs can be renamed as blocks, and sub-CUs can be renamed as sub-blocks, and processing can be performed in units of blocks or sub-blocks.

[0098] The entropy decoding unit 301 performs entropy decoding on the encoded stream Te input from the outside and analyzes each code (syntax element). There are the following methods in entropy encoding: a method of performing variable-length encoding of a syntax element using a context (probability model) adaptively selected according to the syntax element type and surrounding conditions; and a method of performing variable-length encoding of a syntax element using a predefined table or calculation formula. As an example of the former, CABAC (Context Adaptive Binary Arithmetic Coding) can be cited. Among the analyzed codes, there are prediction information for generating a predicted image and prediction errors for generating a differential image.

[0099] The entropy decoding unit 301 outputs the separated code to the parameter decoding unit 302. The separated code is, for example, the prediction mode CuPredMode. The control of which code to decode is performed based on the instruction of the parameter decoding unit 302.

[0100] (Basic Process)

[0101] Figure 7 This is a flowchart for explaining the general operation of the moving image decoding device 31.

[0102] (S1100: Parameter Set Information Decoding) The header decoding unit 3020 decodes parameter set information such as VPS, SPS, PPS, SEI, and PH from the encoded data.

[0103] (S1200: Slice Information Decoding) The header decoding unit 3020 decodes the slice header (slice information) from the encoded data.

[0104] Hereinafter, the moving image decoding device 31 derives the decoded image of each CTU by repeating the processing of S1300 to S5000 for each CTU included in the target picture.

[0105] (S1300: CTU Information Decoding) The CT information decoding unit 3021 decodes the CTU from the encoded data.

[0106] (S1400: CT Information Decoding) The CT information decoding unit 3021 decodes the CT from the encoded data.

[0107] (S1500: CU Decoding) The CU decoding unit 3022 performs S1510 and S1520 to decode the CU from the encoded data. In addition, the CU decoding unit 3022 decodes the differential CuQpDeltaVal of the quantization parameter in units of CU from the encoded data, and derives the quantization parameter.

[0108] (S1510: CU Information Decoding) The CU decoding unit 3022 decodes CU information, prediction information, TU split flag split_transform_flag, CU residual flags cbf_cb, cbf_cr, cbf_luma, etc. from the encoded data.

[0109] (S1520: TU Information Decoding) When the TU includes prediction error, the TU decoding unit 3024 decodes the QP update information and the quantized transform coefficients from the encoded data.

[0110] (S2000: Predicted Image Generation) The predicted image generation unit 308 generates a predicted image for each block included in the target CU based on the prediction information.

[0111] (S3000: Inverse Quantization / Inverse Transformation) The inverse quantization / inverse transformation unit 311 performs inverse quantization / inverse transformation processing for each TU included in the target CU.

[0112] (S4000: Decoded Image Generation) The adder 312 generates the decoded image of the target CU by adding the predicted image supplied by the predicted image generation unit 308 and the prediction error supplied by the inverse quantization / inverse transformation unit 311.

[0113] (S5000: Loop Filter) The loop filter 305 performs loop filtering such as deblocking filtering, SAO (Sample Adaptive Filter), and ALF (Adaptive Loop Filter) on the decoded image to generate the decoded image.

[0114] (Derivation of Quantized Transform Coefficients, Residual Coding)

[0115] In lossless coding and when the pixel correlation of the original image is small, sometimes the coding efficiency is high without performing transformation. The technique of not performing transformation is called Transform Skip.

[0116] Transform Skip is also called Identical Transform, and only scales the transform coefficients corresponding to the quantization parameter. The syntax element transform_skip_flag is used to indicate whether it is Transform Skip. transform_skip_flag can also be indicated for each color component (cIdx) of Y, Cb, and Cr.

[0117] The derivation of the normal prediction error using transformation (RRC: Regular Residual Coding) and the derivation of the prediction error in the transform skip mode (TSRC: Transform Skip Residual Coding) are different in both the encoding method and the decoding method of the prediction error.

[0118] Figure 8 is a block diagram of the TU decoding unit 3024, including an RRC unit 30241 and a TSRC unit 30242. The RRC unit 30241 is a processing unit that derives the normal prediction error using transformation, and the TSRC unit 30242 is a processing unit that derives the prediction error in the transform skip mode.

[0119] Figure 9 The sps_transform_skip_enabled_flag in (a) of is a flag indicating whether to notify the transform_skip_flag in each TU. sps_transform_skip_enabled_flag = 1 indicates that the transform_skip_flag is notified in each TU. sps_transform_skip_enabled_flag = 0 indicates that the transform_skip_flag is not notified in each TU. In the case where the sps_transform_skip_enabled_flag is not notified, it is estimated to be 0.

[0120] Figure 9 The min_qp_prime_ts_minus4 in (a) of is a parameter used to derive the minimum quantization parameter QpPrimeTsMin in the transform skip mode. QpPrimeTsMin is derived from 4 + min_qp_prime_ts_minus4.

[0121] Figure 10 The transform_skip_flag[x0][y0][cIdx] in (b) of indicates whether to apply transformation to the block at the position (x0, y0) of the picture with respect to the color component cIdx. In the case where transform_skip_flag = 1 (transform skip mode), transformation is not applied to this block. In the case where transform_skip_flag = 0, whether to apply transformation to this block depends on other parameters.

[0122] Figures 11 and 12 are the syntax representing the encoding methods of the transform coefficients (prediction errors) under RRC and TSRC.

[0123] (RRC section, RRC mode)

[0124] In the normal predictive error coding method without conversion skipping (Figure 11), the RRC section 30241 decodes the syntax element (not shown) representing the LAST position, and derives the LAST position (LastSignificantCoeffX, LastSignificantCoeffY). The LAST position is the position of the last non-zero coefficient when scanning the transform coefficients of the TU from the low-frequency component to the high-frequency component. In the case of encoding and decoding the transform coefficients sequentially from the high-frequency component, the LAST position represents the position of the first decoded quantized transform coefficient. Next, the RRC section 30241 decodes the coded_sub_block_flag with reference to the LAST position. The coded_sub_block_flag is a flag indicating whether the sub-block includes non-zero coefficients. The sub-block is an area obtained by dividing the TU in units of 4×4. If coded_sub_block_flag = 1 (the sub-block includes non-zero coefficients), the RRC section 30241 decodes the sig_coeff_flag. The sig_coeff_flag is a flag indicating whether the coefficient value is non-zero. If sig_coeff_flag = 1 (the coefficient value is non-zero), the RRC section 30241 decodes the abs_level_gtx_flag, par_level_flag, abs_remainder, and dec_abs_level. These are syntax elements representing the absolute value of the coefficient. The RRC section 30241 derives the absolute value of the coefficient based on these syntax elements. In addition, the RRC section 30241 determines whether the coeff_sign_flag has been notified with reference to signHidden (described below), the absolute value of the coefficient, and the position. If it has been notified, the RRC section 30241 decodes the coeff_sign_flag. The coeff_sign_flag[n] is a flag indicating the sign of the quantized transform coefficient value at the scanning position n. The RRC section 30241 derives the coefficient value based on the absolute value of the coefficient and the coeff_sign_flag.

[0125] As described above, the RRC section 30241 is characterized by decoding the LAST position in the sub-block of the TU.

[0126] In the prediction error coding method (Fig. 12) in the case of not performing conversion (conversion skip mode), the TSRC unit 30242 decodes the coded_sub_block_flag for each sub-block. If coded_sub_block_flag = 1 (the sub-block includes non-zero coefficients), the TSRC unit 30242 decodes the sig_coeff_flag[xC][yC] of the transform coefficient at the position (xC, yC) within the sub-block. If sig_coeff_flag = 1 (the coefficient value is non-zero), the TSRC unit 30242 decodes the coeff_sign_flag, abs_level_gtx_flag, par_level_flag, and abs_remainder. These are syntax elements representing the absolute value of the coefficient. The TSRC unit 30242 derives the absolute value of the coefficient based on these syntax elements.

[0127] As described above, the TSRC unit 30242 is characterized by not decoding the LAST position in the sub-block of the TU.

[0128] (Derivation of Quantized Transform Coefficients, Sign Data Hiding)

[0129] Regarding the positive or negative sign of non-zero quantized transform coefficients, in addition to deriving it based on the flag notified for each coefficient, the sign of the coefficient value can also be estimated by referring to other parameters (sign data hiding). Figure 9 The sps_sign_data_hiding_enabled_flag in (a) is a flag indicating whether the bit position of the coefficient can be estimated in the picture referring to a certain SPS. sps_sign_data_hiding_enabled_flag = 0 indicates that the bit position cannot be estimated. sps_sign_data_hiding_enabled_flag = 1 indicates that the bit position can be estimated. In the case where sps_sign_data_hiding_enabled_flag is not notified, it is estimated as 0.

[0130] Figure 9 The pic_sign_data_hiding_enabled_flag in (b) is a flag indicating whether the bit position of the quantized transform coefficient can be estimated in the existing picture. pic_sign_data_hiding_enabled_flag = 0 indicates that the bit position cannot be estimated. pic_sign_data_hiding_enabled_flag = 1 indicates that the bit position can be estimated. In the case where pic_sign_data_hiding_enabled_flag is not notified, it is estimated as 0.

[0131] The coeff_sign_flag[n] in FIG. 11 is a flag indicating the sign of the quantized transform coefficient value at the scan position n. When coeff_sign_flag[n] = 0, the coefficient value is positive. When this is not the case (coeff_sign_flag[n] = 1), the coefficient value is negative. When coeff_sign_flag[n] is not notified, it is estimated to be 0.

[0132] In the RRC unit 30241, a flag signHidden indicating whether the sign of a specific transform coefficient is derived by estimation is derived. As shown in the following arithmetic expression, for example, the RRC unit 30241 derives signHidden with reference to pic_sign_data_hiding_enabled_flag, ph_dep_quant_enabled_flag (described later), and the distribution of non-zero coefficients. The distribution of non-zero coefficients can be, for example, the difference value between firstSigScanPosSb and lastSigScanPosSb. If this difference value is greater than a specific value, the RRC unit 30241 can set SignHidden to 1, and if this is not the case, it can set SignHidden to 0. The specific value can be 3, for example.

[0133]

[0134] Here, lastSigScanPosSb is the position of the transform coefficient on the high-frequency side at the end within the sub-block (the position of the first decoded transform coefficient), and firstSigScanPosSb is the position of the transform coefficient on the low-frequency side at the beginning within the sub-block.

[0135] As shown in FIG. 11, when the coefficient value is non-zero (AbsLevel[xC][yC]>0) and any of the following conditions is satisfied, the coeff_sign_flag is notified, and the RRC unit 30241 decodes the coeff_sign_flag to derive the sign of the coefficient.

[0136] · signHidden is 0.

[0137] · The coefficient is not at the position where it is finally decoded within the sub-block (n!= firstSigScanPosSb).

[0138] If this is not the case (when the coefficient value is zero, or when signHidden is non-zero and the coefficient position is at the last decoded position within the sub-block), then the coeff_sign_flag is not signaled. Then, the RRC unit 30241 determines the sign of the coefficient (SYN1102) based on whether sumAbsLevel is odd or even. sumAbsLevel is the sum of the absolute values of the decoded transform coefficients within the sub-block. Specifically, when (sumAbsLevel % 2) == 1 is true, the sign of the coefficient is estimated to be negative.

[0139] (Derivation of Quantized Transform Coefficients, Dependent Quantization)

[0140] As quantization methods, there are two methods: scalar quantization and dependent quantization. The inverse quantization / inverse transform unit 311 inverse-quantizes the transform coefficients. In the case of dependent quantization, a part of the inverse quantization process can also be further performed in the RRC unit 30241.

[0141] The inverse quantization / inverse transform unit 311 derives the linear scaling value ls[x][y] based on the quantization parameter qP, rectNonTsFlag, and the values of the quantization matrix m[][] as described below. The inverse quantization / inverse transform unit 311 switches the derivation method of ls[][] in the case where dependent quantization is effective and transform skip is invalid and in other cases.

[0142]

[0143] First, when scalar quantization is effective, the transform coefficient is uniquely derived from the quantized transform coefficient and the quantization parameter. For example, in FIG. 11 or FIG. 12, the transform coefficient value d is derived using the following equation. TransCoeffLevel[x0][y0][cIdx][xC][yC] = AbsLevel[xC][yC] * (1 - 2 * coeff_sign_flag[n]) d[x0][y0][cIdx][xC][yC] = TransCoeffLevel[x0][y0][cidx][xC][yC] * 1s[xC][yC] + ((1 << bdShift) >> 1)) >> bdShift

[0144] Here, AbsLevel is the quantized transform coefficient value, and ls and bdShift are parameters derived from the quantization parameter qP.

[0145] On the other hand, the dependent quantization has two quantizers with different bit levels. The parity of the intermediate values (AbsLevelPass1, AbsLevel) of the quantization conversion coefficients is used to switch four states QState. Then, quantization / inverse quantization is performed according to QState. It should be noted that quantization / inverse quantization using quantization parameters, that is, scaling, is performed separately.

[0146] QState = QStateTransTanle[QState][AbsLevelPassl[xC][yC] & 1]

[0147] TransCoeffLevel[x0][[y0][cIdx][xC][yC] = (2 * AbsLevel[xC][yC] - (QState > 1? 1 : 0)) * (1 - 2 * coeff_sign_flag[n])

[0148] d[x0][y0][cIdx][xC[yC] = (TransCoeffLeve1[x0][y0][cIdx][xC[yC] * 1s[xC][yC] + ((1 << bdShift) >> 1)) >> bdShift

[0149] Here, QState is the state, and QStateTransTable[][] is the table used in state transition. For example, QStateTransTable[][] = {{0, 2}, {2, 0}, {1, 3}, {3, 1}}.

[0150] QState can also be derived by the following formula without using QStateTransTable[][]. QState = (32040 >> ((QState << 2) + ((AbsLevelPassl[xC][yC] & 1) << 1))) & 3

[0151] According to the value of QState, even if AbsLevel is the same, different TransCoeffLevel (or d) is derived. QState is derived by referring to the previously decoded quantization conversion coefficient value. Therefore, compared with the normal scalar (inverse) quantization, (inverse) quantization with good coding efficiency using the correlation between coefficients can be performed.

[0152] Figure 9The sps_dep_quant_enabled_flag in (b) is a flag indicating whether dependent quantization can be performed in the picture referring to a certain SPS. sps_dep_quant_enabled_flag = 0 indicates that dependent conversion cannot be performed. sps_dep_quant_enabled_flag = 1 indicates that dependent quantization can be performed.

[0153] Figure 9 The ph_dep_quant_enabled_flag in (b) is a flag indicating whether dependent quantization can be performed in the existing picture. ph_dep_quant_enabled_flag = 0 indicates that dependent conversion cannot be performed. ph_dep_quant_enabled_flag = 1 indicates that dependent quantization can be performed. In the case where ph_dep_quant_enabled_flag is not notified, it is estimated to be 0.

[0154] In the normal prediction error coding method (RRC), the decoding start position of the non-zero transform coefficient called the LAST position is encoded in a manner suitable for the case where non-zero transform coefficients are concentrated in a partial area of the low-frequency component, and the quantized transform coefficients are decoded by scanning the low-frequency component side from the LAST position. In addition, in order to use sign data hiding and dependent quantization, the RRC unit 30241 derives parameters such as the sum of the absolute values of the transform coefficients required for these processes and the state (QState) of dependent quantization. On the other hand, in the prediction error coding (TSRC) using the transform skip mode, non-zero transform coefficients are not concentrated in a partial area, so the LAST position is not encoded, and the quantized transform coefficients are decoded by scanning the entire block. In addition, sign data hiding and dependent quantization are not used, so the TSRC unit 30242 does not derive the above parameters required for these processes.

[0155] Furthermore, even in the case of the transform skip mode, there are times when the coding efficiency is good when using the normal prediction error coding method. Therefore, the normal prediction error can also be utilized in the transform skip mode. For example, it is also possible to notify Figure 10 the slice_ts_residual_coding_disabled_flag shown in (a).

[0156] The slice_ts_residual_coding_disabled_flag indicates whether the TSRC mode, i.e., residual_ts_coding(), is used for decoding the prediction error of the blocks skipped by the applied transform (transform skip blocks) in the current slice. slice_ts_residual_coding_disabled_flag = 1 indicates that the RRC mode, i.e., residual_coding(), is used for parsing the prediction error of the transform skip blocks. slice_ts_residual_coding_disabled_flag = 0 indicates that residual_ts_coding() is used for parsing the prediction error of the transform skip blocks. In the case where slice_ts_residual_coding_disabled_flag is not signaled, it is estimated to be 0.

[0157] As Figure 10 shown in (b) of, in the case of!transform_skip_flag[x0][y0][0] || slice_ts_residual_coding_disabled_flag, processing can be performed by the RRC unit 30241. In other cases, processing can be performed by the TSRC unit 30242. That is, in addition to the case where transform_skip_flag is 0, the transform coefficients (prediction errors) can also be signaled by RRC in the case where slice_ts_residual_coding_disabled_flag is not 0.

[0158] By referring to slice_ts_residual_coding_disabled_flag, even in the case of the transform skip mode, the normal coding method (RRC mode) for prediction errors can be used.

[0159] (Embodiment 2)

[0160] As Figure 13As shown, slice_ts_residual_coding_disabled_flag can be set to refer to sps_transform_skip_enabled_flag. When sps_transform_skip_enabled_flag = 1, slice_ts_residual_coding_disabled_flag is notified. slice_ts_residual_coding_disabled_flag is a flag indicating whether to use the TSRC mode for decoding the prediction error of the block with transform skip applied in the existing slice. Therefore, when it is notified in the SPS that the transform skip mode is not used, the TSRC mode is not used, and there is no need to notify slice_ts_residual_coding_disabled_flag. Thus, the code amount can be reduced, and the effect of improving the coding efficiency can be achieved.

[0161] (Embodiment 3)

[0162] The installation costs of hardware and software can be reduced by making transform skip and dependent quantization work, and making symbol data hiding and dependent quantization operate exclusively.

[0163] As shown in FIG. 14, the RRC unit 30241 can switch the four states of dependent quantization QState (SYN1401, SYN1402, SYN1404, SYN1405) with reference to slice_ts_residual_coding_disabled_flag. Specifically, when ph_dep_quant_enabled_flag is 1 and slice_ts_residual_coding_disabled_flag is 0, QState (SYN1401, SYN1402) is updated. Conversely, when ph_dep_quant_enabled_flag is 0 or slice_ts_residual_coding_disabled_flag is 1, the update of QState is not performed. Thus, the quantized transform coefficients of transform skip can also be decoded in the RRC mode. Therefore, even when transform skip is effective, if slice_ts_residual_coding_disabled_flag = 1, the RRC unit 30241 does not perform dependent quantization. Through the above conditions, transform skip and dependent quantization can operate exclusively.

[0164] Alternatively, when updating QState without using QStateTransTable[][], the variable stateVal can be used.

[0165] stateVal = (ph_dCp_quant_enabled_flag &&!slice_ts_residual_coding_disabled_flag)? 32040 : 0

[0166] QState = (stateVal >> ((QState << 2) + ((AbsLevelPassl[xC][yC] & 1) << 1))) & 3

[0167] When stateVal is 0, QState is always 0, meaning that no dependent quantization is performed. Therefore, the transformation can skip the actions exclusive to dependent quantization.

[0168] In addition, as shown in Figure 14, the slice_ts_residual_coding_disabled_flag (SYN1403) can be used to derive the value of signHidden that determines whether to apply symbol data hiding. Specifically, when ph_dep_quant_enabled_flag is 1, or pic_sign_data_hiding_enabled_flag is 0, or slice_ts_residual_coding_disabled_flag is 1, signHidden is set to 0.

[0169]

[0170] By doing so, when ph_dep_quant_enabled_flag is 0, pic_sign_data_hiding_enabled_flag is 1, and slice_ts_residual_coding_disabled_flag is 0, symbol data hiding is performed according to the distribution of non-zero coefficients. The distribution of non-zero coefficients, for example, refers to the difference value between firstSigScanPosSb and lastSigScanPosSb. Therefore, the transformation can skip the actions exclusive to symbol data hiding.

[0171] In addition, the RRC unit 30241 can use the transform_skip_flag instead of the slice_ts_residual_coding_disabled_flag to make the transformation skip the actions exclusive to symbol data hiding and dependent quantization.

[0172] As Figure 15As shown, first, the TU decoding unit 3024 notifies the RRC unit 30241 of the transform_skip_flag (SYN1501, SYN1502, SYN1503). Next, as shown in FIG. 16, when ph_dep_quant_enabled_flag is 1 and transform_skip_flag is 0, the RRC unit 30241 updates QState. Otherwise, when ph_dep_quant_enabled_flag is 0 or slice_ts_residual_coding_disabled_flag is 1, QState is not updated. In this way, the RRC unit 30241 does not perform dependency quantization when transform skip is effective (SYN1601, SYN1602, SYN1604, SYN1605).

[0173] In addition, the transform_skip_flag can be used to derive the value of signHidden. Specifically, when ph_dep_quant_enabled_flag is 1, or pic_sign_data_hiding_enabled_flag is 0, or transform_skip_flag is 1, the RRC unit 30241 sets signHidden to 0 (SYN1603).

[0174]

[0175] By adopting such a configuration, the combination of transform skip, symbol data hiding, and dependency quantization can be eliminated, and the effect of reducing the installation costs of hardware and software can be achieved.

[0176] The inverse quantization / inverse transformation unit 311 scales (inverse quantizes) the quantized transformation coefficients input from the entropy decoding unit 301 to obtain the transformation coefficients d[][]. These quantized transformation coefficients are coefficients obtained by performing transformations such as DCT (Discrete Cosine Transform) and DST (Discrete Sine Transform) on the prediction error and then quantizing them during the encoding process. When transform_skip_flag is 0, the inverse quantization / inverse transformation unit 311 performs inverse frequency transformations such as inverse DCT and inverse DST on the scaled transformation coefficients d[][] to calculate the prediction error res[][]. When transform_skip_flag is 1, the inverse quantization / inverse transformation unit 311 sets res[x][y] = d[x][y]. The inverse quantization / inverse transformation unit 311 outputs the prediction error to the addition unit 312.

[0177] Note that the inverse conversion and the conversion are processed in pairs, so the conversion and the inverse conversion can be explained interchangeably. Alternatively, when the inverse conversion is referred to as the conversion, the conversion can be referred to as the forward conversion. For example, when the inverse non-separable conversion is referred to as the non-separable conversion, the non-separable conversion can be referred to as the forward non-separable conversion. In addition, the separable conversion is simply referred to as the conversion.

[0178] The adder 312 adds the predicted image of the block input from the predicted image generation unit 308 and the prediction error input from the inverse quantization / inverse conversion unit 311 pixel by pixel to generate the decoded image of the block. The adder 312 stores the decoded image of the block in the reference picture memory 306 and outputs it to the loop filter 305.

[0179] (Configuration of the moving image encoding device)

[0180] Next, the configuration of the moving image encoding device 11 of the present embodiment will be described. Figure 17 FIG. is a block diagram showing the configuration of the moving image encoding device 11 of the present embodiment. The moving image encoding device 11 is configured to include: a predicted image generation unit 101, a subtraction unit 102, a transform / quantization unit 103, an inverse quantization / inverse conversion unit 105, an adder 106, a loop filter 107, a prediction parameter memory (prediction parameter storage unit, frame memory) 108, a reference picture memory (reference image storage unit, frame memory) 109, an encoding parameter determination unit 110, a parameter encoding unit 111, and an entropy encoding unit 104.

[0181] The predicted image generation unit 101 generates a predicted image for each region, that is, each CU, of each image of the divided image T. The predicted image generation unit 101 performs the same operation as the predicted image generation unit 308 described above, and the description thereof is omitted.

[0182] The subtraction unit 102 subtracts the pixel value of the predicted image of the block input from the predicted image generation unit 101 from the pixel value of the image T to generate a prediction error. The subtraction unit 102 outputs the prediction error to the transform / quantization unit 103.

[0183] The transform / quantization unit 103 calculates transform coefficients by frequency transformation for the prediction error input from the subtraction unit 102, and derives quantized transform coefficients by quantization. The transform / quantization unit 103 outputs the quantized transform coefficients to the entropy encoding unit 104 and the inverse quantization / inverse conversion unit 105.

[0184] The inverse quantization / inverse conversion unit 105 is the same as the inverse quantization / inverse conversion unit 311 in the moving image decoding device 31 ( Figure 10 ), and the description thereof is omitted. The calculated prediction error is output to the adder 106.

[0185] The entropy encoding unit 104 is input with quantized transform coefficients from the transform / quantization unit 103 and encoding parameters from the parameter encoding unit 111. The encoding parameters are, for example, predMode indicating the prediction mode. predMode can be any one of MODE_INTRA indicating intra-frame prediction, MODE_INTER indicating inter-frame prediction, or MODE_IBC indicating intra-block copy prediction in which a block within a copied picture is set as a predicted image.

[0186] The entropy encoding unit 104 performs entropy encoding on the segmentation information, prediction parameters, quantized transform coefficients, etc., to generate and output an encoded stream Te.

[0187] The parameter encoding unit 111 includes: a header encoding unit 1110 (not shown), a CT information encoding unit 1111, a CU encoding unit 1112 (prediction mode encoding unit), an inter-frame prediction parameter encoding unit 112, and an intra-frame prediction parameter encoding unit 113. The CU encoding unit 1112 further includes a TU encoding unit 1114.

[0188] Hereinafter, the schematic operations of each module will be described. The parameter encoding unit 111 performs encoding processing on parameters such as header information, segmentation information, prediction information, and quantized transform coefficients.

[0189] The CT information encoding unit 1111 encodes QT, MT (BT, TT) segmentation information, etc., based on the encoded data.

[0190] The CU encoding unit 1112 encodes CU information, prediction information, TU segmentation flag, CU residual flag, etc.

[0191] The TU encoding unit 1114 encodes QP update information and quantized transform coefficients when the TU includes a prediction error.

[0192] The CT information encoding unit 1111 and the CU encoding unit 1112 supply syntax elements such as inter-frame prediction parameters, intra-frame prediction parameters, and quantized transform coefficients to the entropy encoding unit 104.

[0193] The adder 106 adds the pixel values of the predicted image of the block input from the predicted image generation unit 101 and the prediction error input from the inverse quantization / inverse transform unit 105 on a per-pixel basis to generate a decoded image. The adder 106 stores the generated decoded image in the reference picture memory 109.

[0194] The loop filter 107 performs a deblocking filter, SAO, and ALF on the decoded image generated by the adder 106. It should be noted that the loop filter 107 does not necessarily include the above three filters. For example, it can be configured with only a deblocking filter.

[0195] The SAO is a filter that adds an offset corresponding to the classification result to each sample, and the ALF is a filter that uses the sum of products of the transmitted filter coefficients and the reference image (or the difference between the reference image and the target pixel).

[0196] The prediction parameter memory 108 stores the prediction parameters generated by the encoding parameter determination unit 110 at a predetermined position for each target picture and each CU.

[0197] The reference picture memory 109 stores the decoded images generated by the loop filter 107 at a predetermined position for each target picture and each CU.

[0198] The encoding parameter determination unit 110 selects one set from a plurality of sets of encoding parameters. The encoding parameters refer to the above-mentioned QT, BT or TT segmentation information, prediction parameters, or parameters generated in association with these and to be encoded. The prediction image generation unit 101 generates a prediction image using these encoding parameters.

[0199] The encoding parameter determination unit 110 calculates the RD cost value representing the amount of information and the encoding error for each of the plurality of sets. The encoding parameter determination unit 110 selects the set of encoding parameters for which the calculated cost value is the smallest. Thus, the entropy encoding unit 104 outputs the selected set of encoding parameters as the encoded stream Te. The encoding parameter determination unit 110 stores the determined encoding parameters in the prediction parameter memory 108.

[0200] Note that a part of the moving image encoding device 11 and the moving image decoding device 31 of the above-described embodiments can be implemented by a computer. For example, the entropy decoding unit 301, the parameter decoding unit 302, the loop filter 305, the predicted image generation unit 308, the inverse quantization / inverse transformation unit 311, the addition unit 312, the predicted image generation unit 101, the subtraction unit 102, the transformation / quantization unit 103, the entropy encoding unit 104, the inverse quantization / inverse transformation unit 105, the loop filter 107, the encoding parameter determination unit 110, and the parameter encoding unit 111. In this case, it can be achieved by recording a program for implementing the control function on a computer-readable recording medium, and causing the computer system to read and execute the program recorded on the recording medium. Note that the "computer system" mentioned here refers to the computer system built in either the moving image encoding device 11 or the moving image decoding device 31, and is a computer system including hardware such as an OS and peripheral devices. In addition, the "computer-readable recording medium" refers to removable media such as floppy disks, magneto-optical disks, ROMs, CD-ROMs, and storage devices such as hard disks built in the computer system. Moreover, the "computer-readable recording medium" also includes: a recording medium that dynamically stores a program for a short time, such as a communication line in the case of transmitting a program via a network such as the Internet or a communication line such as a telephone line; and a recording medium that stores a program for a fixed time, such as a volatile memory inside a computer system of a server or a client in this case. In addition, the above program can be a program for implementing a part of the foregoing functions, or a program that can implement the foregoing functions by combining with a program already recorded in the computer system.

[0201] In addition, a part or all of the moving image encoding device 11 and the moving image decoding device 31 of the above-described embodiments can be implemented as an integrated circuit such as an LSI (Large Scale Integration). Each functional block of the moving image encoding device 11 and the moving image decoding device 31 can be processorized individually, or a part or all of them can be integrated and processorized. In addition, the method of integrating into an integrated circuit is not limited to LSI, and can also be implemented by a dedicated circuit or a general-purpose processor. In addition, in the case where an integrated circuit technology replacing LSI appears with the progress of semiconductor technology, an integrated circuit based on that technology can also be used.

[0202] As described above, one embodiment of the present invention has been described in detail with reference to the drawings, but the specific configuration is not limited to the above-described embodiment, and various design changes and the like can be made without departing from the gist of the present invention.

[0203] [Application Example]

[0204] The above-described moving image encoding apparatus 11 and moving image decoding apparatus 31 can be mounted on various apparatuses that transmit, receive, record, and reproduce moving images and utilized. It should be noted that the moving image can be a natural moving image captured by a camera or the like, or an artificial moving image (including CG and GUI) generated by a computer or the like.

[0205] First, with reference to Figure 2 , a case where the above-described moving image encoding apparatus 11 and moving image decoding apparatus 31 are used for transmission and reception of a moving image will be described.

[0206] Figure 2 FIG. shows a block diagram illustrating the configuration of a transmission apparatus PROD_A on which the moving image encoding apparatus 11 is mounted. As Figure 2 shown, the transmission apparatus PROD_A includes: an encoding unit PROD_A1 that obtains encoded data by encoding a moving image; a modulation unit PROD_A2 that obtains a modulation signal by modulating a carrier wave using the encoded data obtained by the encoding unit PROD_A1; and a transmission unit PROD_A3 that transmits the modulation signal obtained by the modulation unit PROD_A2. The above-described moving image encoding apparatus 11 is used as this encoding unit PROD_A1.

[0207] The transmission apparatus PROD_A may further include a camera PROD_A4 that captures a moving image as a supply source of the moving image input to the encoding unit PROD_A1, a recording medium PROD_A5 on which a moving image is recorded, an input terminal PROD_A6 for inputting a moving image from the outside, and an image processing unit A7 that generates or processes an image. Figure 2 FIG. exemplifies a configuration in which the transmission apparatus PROD_A includes all of these, but a part thereof may be omitted.

[0208] It should be noted that the recording medium PROD_A5 may be a medium on which an unencoded moving image is recorded, or a medium on which a moving image encoded in a recording encoding method different from the transmission encoding method is recorded. In the latter case, it is preferable to interpose a decoding unit (not shown) that decodes the encoded data read from the recording medium PROD_A5 according to the recording encoding method between the recording medium PROD_A5 and the encoding unit PROD_A1.

[0209] In addition, Figure 2 FIG. shows a block diagram illustrating the configuration of a reception apparatus PROD_B on which the moving image decoding apparatus 31 is mounted. As Figure 2As shown, the receiving device PROD_B includes: a receiving unit PROD_B1 that receives a modulated signal; a demodulating unit PROD_B2 that obtains encoded data by demodulating the modulated signal received by the receiving unit PROD_B1; and a decoding unit PROD_B3 that obtains a moving image by decoding the encoded data obtained by the demodulating unit PROD_B2. The moving image decoding device 31 is used as the decoding unit PROD_B3.

[0210] The receiving device PROD_B may further include a display PROD_B4 for displaying a moving image, which is a destination for supplying the moving image output from the decoding unit PROD_B3, a recording medium PROD_B5 for recording the moving image, and an output terminal PROD_B6 for outputting the moving image to the outside. Figure 2 An example is shown in which the receiving device PROD_B has all of these configurations, but some of them may be omitted.

[0211] It should be noted that the recording medium PROD_B5 may be a medium for recording an unencoded moving image, or a medium encoded in a recording encoding method different from the transmission encoding method. In the latter case, it is preferable to interpose an encoding unit (not shown) that encodes the moving image obtained from the decoding unit PROD_B3 according to the recording encoding method between the decoding unit PROD_B3 and the recording medium PROD_B5.

[0212] It should be noted that the transmission medium for transmitting the modulated signal may be wireless or wired. In addition, the transmission scheme for transmitting the modulated signal may be broadcast (here, it refers to a transmission scheme in which the transmission destination is not predetermined) or communication (here, it refers to a transmission scheme in which the transmission destination is predetermined). That is, the transmission of the modulated signal can be achieved by any one of wireless broadcast, wired broadcast, wireless communication, and wired communication.

[0213] For example, a terrestrial digital broadcast station (broadcasting equipment, etc.) / reception station (television receiver, etc.) is an example of a transmitting device PROD_A / receiving device PROD_B that transmits and receives a modulated signal by wireless broadcast. In addition, a cable television broadcast station (broadcasting equipment, etc.) / reception station (television receiver, etc.) is an example of a transmitting device PROD_A / receiving device PROD_B that transmits and receives a modulated signal by wired broadcast.

[0214] In addition, servers (workstations, etc.) / clients (TV receivers, personal computers, smartphones, etc.) for VOD (Video On Demand) services, moving image sharing services, etc. that use the Internet are an example of a transmission device PROD_A / reception device PROD_B that communicates and transceives modulated signals (usually, either wireless or wired is used as the transmission medium in a LAN, and wired is used as the transmission medium in a WAN). Here, personal computers include desktop PCs, laptop PCs, and tablet PCs. In addition, smartphones also include multifunctional portable phone terminals.

[0215] It should be noted that, in addition to the function of decoding the encoded data downloaded from the server and displaying it on the monitor, the client of the moving image sharing service also has the function of encoding the moving images captured by the camera and uploading them to the server. That is, the client of the moving image sharing service functions as both the transmission device PROD_A and the reception device PROD_B.

[0216] Next, with reference to Figure 3 a case where the above moving image encoding device 11 and moving image decoding device 31 are used for recording and reproducing moving images will be described.

[0217] Figure 3 FIG. shows a block diagram showing the configuration of a recording device PROD_C equipped with the above moving image encoding device 11. As Figure 3 shown, the recording device PROD_C includes an encoding unit PROD_C1 that obtains encoded data by encoding a moving image, and a writing unit PROD_C2 that writes the encoded data obtained by the encoding unit PROD_C1 to a recording medium PROD_M. The above moving image encoding device 11 is used as this encoding unit PROD_C1.

[0218] It should be noted that the recording medium PROD_M can be (1) a type of recording medium built into the recording device PROD_C such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive), (2) a type of recording medium connected to the recording device PROD_C such as an SD memory card or a USB (Universal Serial Bus) flash drive, or (3) a recording medium such as a DVD (Digital Versatile Disc, registered trademark) or a BD (Blu-ray Disc, registered trademark) loaded into a drive device (not shown) built into the recording device PROD_C.

[0219] In addition, the recording device PROD_C may further include a camera PROD_C3 that captures a moving image as a supply source of the moving image input to the encoding unit PROD_C1, an input terminal PROD_C4 for inputting a moving image from the outside, a receiving unit PROD_C5 for receiving a moving image, and an image processing unit PROD_C6 for generating or processing an image. Figure 3 The configuration in which the recording device PROD_C includes all of these is illustrated in Figure 3 , and a part thereof may be omitted.

[0220] It should be noted that the receiving unit PROD_C5 may receive an unencoded moving image or encoded data encoded in a transmission encoding method different from the recording encoding method. In the latter case, it is preferable to interpose a transmission decoding unit (not shown) that decodes the encoded data encoded in the transmission encoding method between the receiving unit PROD_C5 and the encoding unit PROD_C1.

[0221] Examples of such a recording device PROD_C include a DVD recorder, a BD recorder, an HDD (Hard Disk Drive) recorder, etc. (in this case, the input terminal PROD_C4 or the receiving unit PROD_C5 is the main supply source of the moving image). In addition, a camcorder (in this case, the camera PROD_C3 is the main supply source of the moving image), a personal computer (in this case, the receiving unit PROD_C5 or the image processing unit C6 is the main supply source of the moving image), a smartphone (in this case, the camera PROD_C3 or the receiving unit PROD_C5 is the main supply source of the moving image), etc. are also examples of such a recording device PROD_C.

[0222] In addition, Figure 3 a block diagram showing the configuration of a playback device PROD_D equipped with the above-described moving image decoding device 31 is shown in Figure 3 . As Figure 3 shown, the playback device PROD_D includes a reading unit PROD_D1 that reads the encoded data written on the recording medium PROD_M and a decoding unit PROD_D2 that obtains a moving image by decoding the encoded data read by the reading unit PROD_D1. The above-described moving image decoding device 31 is used as this decoding unit PROD_D2.

[0223] It should be noted that the recording medium PROD_M may be (1) a type of recording medium built into the playback device PROD_D such as an HDD or an SSD, (2) a type of recording medium connected to the playback device PROD_D such as an SD memory card or a USB flash drive, or (3) a recording medium such as a DVD or a BD loaded into a drive device (not shown) built into the playback device PROD_D.

[0224] In addition, the reproduction device PROD_D may further include a display PROD_D3 for displaying a moving image that is the supply destination of the moving image output by the decoding unit PROD_D2, an output terminal PROD_D4 for outputting the moving image to the outside, and a transmission unit PROD_D5 for transmitting the moving image. Figure 3 The configuration in which the reproduction device PROD_D includes all of these is illustrated as an example, and a part thereof may be omitted.

[0225] It should be noted that the transmission unit PROD_D5 may transmit an unencoded moving image or encoded data encoded in a transmission encoding method different from the recording encoding method. In the latter case, it is preferable to interpose an encoding unit (not shown) that encodes the moving image in the transmission encoding method between the decoding unit PROD_D2 and the transmission unit PROD_D5.

[0226] Examples of such a reproduction device PROD_D include a DVD player, a BD player, an HDD player, etc. (in this case, the output terminal PROD_D4 connected to a television receiver or the like is the main supply destination of the moving image). In addition, a television receiver (in this case, the display PROD_D3 is the main supply destination of the moving image), a digital signage (also called an electronic billboard, an electronic bulletin board, etc., the display PROD_D3 or the transmission unit PROD_D5 is the main supply destination of the moving image), a desktop PC (in this case, the output terminal PROD_D4 or the transmission unit PROD_D5 is the main supply destination of the moving image), a laptop or tablet PC (in this case, the display PROD_D3 or the transmission unit PROD_D5 is the main supply destination of the moving image), a smartphone (in this case, the display PROD_D3 or the transmission unit PROD_D5 is the main supply destination of the moving image), etc. are also examples of such a reproduction device PROD_D. (Hardware implementation and software implementation)

[0227] In addition, each block of the above moving image decoding device 31 and moving image encoding device 11 may be implemented in a hardware manner by a logic circuit formed on an integrated circuit (IC chip), or may be implemented in a software manner using a CPU (Central Processing Unit).

[0228] In the latter case, each of the above-described devices includes: a CPU that executes commands of a program for implementing each function, a ROM (Read Only Memory) that stores the above-described program, a RAM (Random Access Memory) that expands the above-described program, and a storage device (recording medium) such as a memory that stores the above-described program and various data. Further, the object of the embodiment of the present invention can also be achieved by the following method: A recording medium on which program codes (executable form program, intermediate code program, source program) of a software for implementing the above-described functions, that is, a control program of each of the above-described devices, are recorded in a computer-readable manner is supplied to each of the above-described devices, and the program codes recorded in the recording medium are read and executed by the computer (or CPU, MPU).

[0229] As the above-described recording medium, for example, the following can be used: magnetic tapes such as tapes, cassette tapes, and cartridge tapes; disk types including floppy disks (registered trademark) / hard disks, optical disks such as CD-ROMs (Compact Disc Read-Only Memories) / MO disks (Magneto-Optical discs) / MDs (Mini Discs, mini magneto-optical discs) / DVDs (Digital Versatile Discs: registered trademark) / CD-Rs (CD Recordables) / Blu-ray Discs (registered trademark); card types such as IC cards (including memory cards) / optical cards; semiconductor memory types such as mask ROMs / EPROMs (Erasable Programmable Read-Only Memories) / EEPROMs (Electrically Erasable and Programmable Read-Only Memories, registered trademark) / flash ROMs; or logic circuit types such as PLDs (Programmable logic devices) / FPGAs (Field Programmable Gate Arrays).

[0230] In addition, each of the above-described devices may be configured to be connectable to a communication network, and the above program code may be supplied via the communication network. The communication network is not particularly limited as long as it can transmit the program code. For example, the Internet, an intranet, an extranet, a LAN (Local Area Network), an ISDN (Integrated Services Digital Network), a VAN (Value-Added Network), a CATV (Community Antenna television / Cable Television) communication network, a virtual private network, a telephone line network, a mobile communication network, a satellite communication network, etc. may be used. In addition, the transmission medium constituting the communication network is also not limited to a specific configuration or type as long as it can transmit the program code. For example, it can be used in wired media such as IEEE (Institute of Electrical and Electronic Engineers) 1394, USB, power line transmission, cable TV line, telephone line, ADSL (Asymmetric Digital Subscriber Line) line, etc., or in wireless media such as IrDA (Infrared Data Association), infrared rays like a remote control, Bluetooth (registered trademark), IEEE802.11 wireless, HDR (High Data Rate), NFC (Near Field Communication), DLNA (Digital Living Network Alliance, registered trademark), a mobile phone network, a satellite line, a terrestrial digital broadcast network, etc. In addition, the embodiments of the present invention can also be implemented in the form of a computer data signal embedded in a carrier wave that embodies the above program code by electronic transmission.

[0231] The embodiments of the present invention are not limited to the above-described embodiments, and various modifications can be made within the scope shown in the claims. That is, embodiments obtained by combining technical solutions appropriately modified within the scope shown in the claims are also included in the technical scope of the present invention.

[0232] Industrial Applicability

[0233] Embodiments of the present invention can preferably be applied to a moving image decoding device that decodes encoded data obtained by encoding image data, and a moving image encoding device that generates encoded data obtained by encoding image data. Further, it can preferably be applied to a data structure of encoded data generated by the moving image encoding device and referred to by the moving image decoding device.

[0234] Main Component Symbol Explanation

[0235] 31 Moving image decoding device

[0236] 301 Entropy decoding unit

[0237] 302 Parameter decoding unit

[0238] 3020 Header decoding unit

[0239] 308 Predicted image generation unit

[0240] 311 Inverse quantization / inverse transformation unit

[0241] 312 Addition unit

[0242] 11 Moving image encoding device

[0243] 101 Predicted image generation unit

[0244] 102 Subtraction unit

[0245] 103 Transformation / quantization unit

[0246] 104 Entropy encoding unit

[0247] 105 Inverse quantization / inverse transformation unit

[0248] 107 Loop filter

[0249] 110 Encoding parameter determination unit

[0250] 111 Parameter encoding unit

[0251] 1110 Header encoding unit

[0252] 1111 CT information encoding unit

[0253] 1112 CU encoding unit (prediction mode encoding unit)

[0254] 1114 TU encoding unit

[0255] 311 Inverse quantization / inverse transformation unit

[0256] 3111 Inverse quantization unit

[0257] 3112 Inverse transformation unit

Claims

1. A decoding unit for decoding residual encoded information, characterized in that, The decoding unit includes: a header decoding unit configured to (i) decode a transform skip dependent quantization flag in a sequence parameter set, where the transform skip dependent quantization flag specifies whether there is a transform skip flag transform_skip_flag; and (ii) decode a disable transform skip prediction error quantization flag that specifies whether to use transform skip prediction error coding when the value of the transform skip dependent quantization flag is equal to one; wherein, when the value of the transform skip dependent quantization flag is equal to zero, the disable transform skip prediction error quantization flag is not decoded, and a transform unit TU decoding unit configured to (i) decode the transform skip flag that specifies whether to apply a transform to an associated block when the value of the transform skip dependent quantization flag is equal to one; and (ii) decode transform coefficient data encoded by normal prediction error coding, the normal prediction error coding being used to generate a syntax element specifying the position of the last valid coefficient in the scan order in a transform block, or decode transform skip prediction error coding, the transform skip prediction error coding being untransformed coding, wherein: the TU decoding unit is configured to use the normal prediction error coding when the value of the transform skip flag is equal to zero or the value of the disable transform skip prediction error quantization flag is equal to one, and otherwise, the TU decoding unit is configured to use the transform skip prediction error coding.

2. The decoding unit according to claim 1, characterized in that, When the disable transform skip prediction error quantization flag does not exist, it is inferred that the value of the disable transform skip prediction error quantization flag is equal to zero.

3. An encoding unit for encoding residual information, characterized in that, The encoding unit includes: a header encoding unit configured to (i) encode a transform skip dependent quantization flag in a sequence parameter set, where the transform skip dependent quantization flag specifies whether there is a transform skip flag transform_skip_flag; and (ii) encode a disable transform skip prediction error quantization flag that specifies whether to use transform skip prediction error coding when the value of the transform skip dependent quantization flag is equal to one, wherein, when the value of the transform skip dependent quantization flag is equal to zero, the disable transform skip prediction error quantization flag is not encoded, and a transform unit TU encoding unit configured to (i) encode the transform skip flag that specifies whether to apply a transform to an associated block when the value of the transform skip dependent quantization flag is equal to one; and (ii) encode transform coefficient data by normal prediction error coding, the normal prediction error coding being used to generate a syntax element specifying the position of the last valid coefficient in the scan order in a transform block, or encode transform skip prediction error coding, the transform skip prediction error coding being untransformed coding, wherein: The TU coding unit is configured to use the coding of the normal prediction error when the value of the transform skip flag is equal to zero or the value of the flag that prohibits transform skip prediction error quantization is equal to one, and otherwise, the TU coding unit is configured to use the transform skip prediction error coding.

4. The encoding unit according to claim 3, wherein In the case where the flag that prohibits transform skip prediction error quantization does not exist, it is inferred that the value of the flag that prohibits transform skip prediction error quantization is equal to zero.

5. A method for decoding residual coding information, characterized in that The method includes: decoding a flag for transform skip dependent quantization in a sequence parameter set, where the flag for transform skip dependent quantization specifies whether there is a transform skip flag transform_skip_flag; when the value of the flag for transform skip dependent quantization is equal to one, decoding a flag that prohibits transform skip prediction error quantization that specifies whether to use transform skip prediction error coding, where, when the value of the flag for transform skip dependent quantization is equal to zero, the flag that prohibits transform skip prediction error quantization is not decoded; when the value of the flag for transform skip dependent quantization is equal to one, decoding the transform skip flag that specifies whether to apply a transform to an associated block; and decoding transform coefficient data encoded by the coding of the normal prediction error, where the coding of the normal prediction error is used to generate a syntax element that specifies the position of the last valid coefficient in the scan order in a transform block, or decoding a transform skip prediction error coding, where the transform skip prediction error coding is an untransformed coding, where: the coding of the normal prediction error is used when the value of the transform skip flag is equal to zero or the value of the flag that prohibits transform skip prediction error quantization is equal to one, and otherwise, the transform skip prediction error coding is used.

6. The method according to claim 5, characterized in that In the case where the flag that prohibits transform skip prediction error quantization does not exist, it is inferred that the value of the flag that prohibits transform skip prediction error quantization is equal to zero.

7. A method for encoding residual information, characterized in that, The method includes: encoding a flag for transform skip dependent quantization in a sequence parameter set, where the flag for transform skip dependent quantization specifies whether there is a transform skip flag transform_skip_flag; when the value of the flag for transform skip dependent quantization is equal to one, encoding a flag that prohibits transform skip prediction error quantization that specifies whether to use transform skip prediction error coding, where, when the value of the flag for transform skip dependent quantization is equal to zero, the flag that prohibits transform skip prediction error quantization is not encoded; when the value of the flag for transform skip dependent quantization is equal to one, encoding the transform skip flag that specifies whether to apply a transform to an associated block; and encoding transform coefficient data by the coding of the normal prediction error, where the coding of the normal prediction error is used to generate a syntax element that specifies the position of the last valid coefficient in the scan order in a transform block, or encoding a transform skip prediction error coding, where the transform skip prediction error coding is an untransformed coding, where: Use the coding of the normal prediction error in the case where the value of the conversion skip flag is equal to zero or the value of the flag that prohibits conversion skip prediction error quantization is equal to one, and otherwise, use the conversion skip prediction error coding.

8. The method according to claim 7, wherein In the case where the flag that prohibits conversion skip prediction error quantization does not exist, infer that the value of the flag that prohibits conversion skip prediction error quantization is equal to zero.

Images