Image processing device and method

Abstract

The present disclosure relates to an image processing device and method capable of achieving a reduction in coding efficiency degradation. The present invention relates to: setting an upper limit value of the number of bits assigned to a processing target subblock by allocating the number of bits to a non-zero subblock; deriving a syntax element value about the processing target subblock by using coefficient data derived from image data so that the number of bits does not exceed the upper limit value; and encoding the derived syntax element value to generate encoded data. For example, the present disclosure is applicable to an image processing device, an image encoding device, an image decoding device, a transmitting device, a receiving device, a transmitting and receiving device, an information processing device, an imaging device, a reproducing device, an electronic device, an image processing method, or an information processing method.

Description

Technical Field

[0001] The present disclosure relates to an image processing device and method, and particularly to an image processing device and method capable of suppressing a decrease in encoding efficiency. Background Art

[0002] Generally, in CABAC for image coding, the more bits of context coding are used when expressing a syntax element value, the better the coding efficiency is and the more image quality degradation can be suppressed. However, as the number of bits of context coding increases, the amount of processing increases. Therefore, a method of limiting the number of bits of context coding in units of sub-blocks has been proposed (for example, non-patent document 1).

[0003] Reference List

[0004] Non-patent documents

[0005] Non-Patent Document 1: T.-D. Chuang, S.-T. Hsiang, Z.-Y. Lin, C.-Y. Chen, Y.-W. Huang, S.-M. Lei (MediaTek), “CE7 (Tests 7.1, 7.2, 7.3 and 7.4): Constraints on context-coded bins for coefficient coding”, JVET-M0173, 13th Meeting of the Joint Video Exploration Team (JVET) of ITU-T SG 16WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11: Marrakech, MA, January 9-18, 2019. Summary of the invention

[0006] Problems to be solved by the present invention

[0007] However, in the case of this method, since the number of bits of context encoding is uniformly limited, coefficient data cannot be locally increased or decreased according to the characteristics of the image, and there is a possibility that the encoding efficiency is unnecessarily reduced.

[0008] The present disclosure has been made in view of such circumstances, and can suppress a decrease in encoding efficiency.

[0009] Solution to the problem

[0010] An image processing device on one aspect of the present technology is an image processing device including: a syntax element value derivation unit that sets an upper limit value for the number of bits allocated to a processing target sub-block by allocating the number of bits among non-zero sub-blocks, and derives syntax element values ​​about the processing target sub-block by using coefficient data derived from image data so that the number of bits does not exceed the upper limit value; and an encoding unit that encodes the derived syntax element values ​​and generates encoded data.

[0011] An image processing method on one aspect of the present technology is an image processing method including: setting an upper limit value of the number of bits allocated to a processing target sub-block by allocating the number of bits among non-zero sub-blocks, and deriving a syntax element value about the processing target sub-block by using coefficient data derived from image data so that the number of bits does not exceed the upper limit value; and encoding the derived syntax element value and generating encoded data.

[0012] An image processing device on another aspect of the present technology is an image processing device including: a decoding unit that decodes encoded data and generates syntax element values; and a coefficient data exporting unit that sets an upper limit value for the number of bits allocated to a processing target sub-block by allocating the number of bits among non-zero sub-blocks, and exports coefficient data corresponding to image data regarding the processing target sub-block by using the generated syntax element values ​​so that the number of bits does not exceed the upper limit value.

[0013] An image processing method on the other hand of the present technology is an image processing method including: decoding encoded data and generating syntax element values; and setting an upper limit value of the number of bits allocated to a processing target sub-block by allocating the number of bits among non-zero sub-blocks, and deriving coefficient data corresponding to the image data about the processing target sub-block by using the generated syntax element values ​​so that the number of bits does not exceed the upper limit value.

[0014] An image processing device according to another aspect of the present technology is an image processing device comprising: a syntax element value derivation unit which sets an upper limit value for the number of bits allocated to a processing target sub-block by allocating the number of bits allocated to a zero sub-block to a non-zero sub-block, and derives syntax element values ​​regarding the processing target sub-block by using coefficient data derived from image data so that the number of bits does not exceed the upper limit value; and an encoding unit which encodes the derived syntax element values ​​and generates encoded data.

[0015] Another aspect of the present technology is an image processing method including: setting an upper limit value of the number of bits allocated to a processing target sub-block by allocating the number of bits allocated to a zero sub-block to a non-zero sub-block, and deriving a syntax element value about the processing target sub-block by using coefficient data derived from image data so that the number of bits does not exceed the upper limit value; and encoding the derived syntax element value and generating encoded data.

[0016] An image processing device according to another aspect of the present technology is an image processing device comprising: a decoding unit that decodes encoded data and generates syntax element values; and a coefficient data exporting unit that sets an upper limit value for the number of bits allocated to a processing target sub-block by allocating the number of bits allocated to a zero sub-block to a non-zero sub-block, and exports coefficient data corresponding to the image data regarding the processing target sub-block by using the generated syntax element values ​​so that the number of bits does not exceed the upper limit value.

[0017] Another aspect of the present technology is an image processing method including: decoding encoded data and generating syntax element values; and setting an upper limit value for the number of bits allocated to a processing target sub-block by allocating the number of bits allocated to a zero sub-block to a non-zero sub-block, and deriving coefficient data corresponding to image data about the processing target sub-block by using the generated syntax element values ​​so that the number of bits does not exceed the upper limit value.

[0018] In an image processing device and method of one aspect of the present technology, an upper limit value of the number of bits allocated to a processing target subblock is set by allocating the number of bits among non-zero subblocks, a syntax element value is derived with respect to the processing target subblock by using coefficient data derived from image data so that the number of bits does not exceed the upper limit value, and the derived syntax element value is encoded and encoded data is generated.

[0019] In an image processing device and method on another aspect of the present technology, encoded data is decoded and syntax element values ​​are generated, and an upper limit value of the number of bits allocated to a processing target sub-block is set by allocating the number of bits among non-zero sub-blocks, and coefficient data corresponding to the image data is derived with respect to the processing target sub-block by using the generated syntax element values ​​so that the number of bits does not exceed the upper limit value.

[0020] In an image processing device and method according to another aspect of the present technology, an upper limit value of the number of bits allocated to a processing target subblock is set by allocating the number of bits allocated to a zero subblock to a non-zero subblock, a syntax element value is derived with respect to the processing target subblock by using coefficient data derived from image data so that the number of bits does not exceed the upper limit value, and the derived syntax element value is encoded and encoded data is generated.

[0021] In an image processing device and method of another aspect of the present technology, encoded data is decoded and syntax element values ​​are generated, and an upper limit value of the number of bits allocated to a processing target sub-block is set by allocating the number of bits allocated to a zero sub-block to a non-zero sub-block, and coefficient data corresponding to the image data is derived with respect to the processing target sub-block by using the generated syntax element values ​​so that the number of bits does not exceed the upper limit value. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 is a diagram illustrating sub-blocks.

[0023] Figure 2 This is a diagram illustrating method #1.

[0024] Figure 3 This is a diagram illustrating an application example of method #1.

[0025] Figure 4 is a block diagram showing a main configuration example of an encoding device.

[0026] Figure 5 is a block diagram showing a main configuration example of CABAC.

[0027] Figure 6 is a flowchart illustrating an example of the flow of encoding processing.

[0028] Figure 7 is a flowchart illustrating an example of the flow of syntax element value derivation processing.

[0029] Figure 8 is a diagram showing an example of syntax.

[0030] Fig. 9 is Figure 8 A diagram showing an example of the syntax is shown below.

[0031] Fig.10 is Fig. 9 A diagram showing an example of the syntax is shown below.

[0032] Fig.11 is Fig.10 A diagram showing an example of the syntax is shown below.

[0033] Fig.12 is a flowchart illustrating an example of the flow of CABAC processing.

[0034] Fig.13 is a block diagram showing a main configuration example of a decoding device.

[0035] Fig.14 is a block diagram showing a main configuration example of CABAC.

[0036] Fig.15 is a flowchart illustrating an example of the flow of a decoding process.

[0037] Fig.16 is a flowchart illustrating an example of the flow of CABAC processing.

[0038] Fig.17 : is a flowchart illustrating an example of the flow of coefficient data derivation processing.

[0039] Fig.18 This is a diagram illustrating method #2.

[0040] Fig.19 This is a diagram illustrating an application example of method #2.

[0041] Fig. 20 is a flowchart illustrating an example of the flow of syntax element value derivation processing.

[0042] Fig.21 is a diagram showing an example of syntax.

[0043] Fig. 22 is Fig.21 A diagram showing an example of the syntax is shown below.

[0044] Fig.23 is Fig. 22 A diagram showing an example of the syntax is shown below.

[0045] Fig.24 is Fig.23 A diagram showing an example of the syntax is shown below.

[0046] Fig.25 : is a flowchart illustrating an example of the flow of coefficient data derivation processing.

[0047] Fig.26 This is a diagram illustrating method #3.

[0048] Fig. 27 This is a diagram illustrating an application example of method #3.

[0049] Fig.28 is a flowchart illustrating an example of the flow of syntax element value derivation processing.

[0050] Fig.29 is a diagram showing an example of syntax.

[0051] Fig.30 is Fig.29 A diagram showing an example of the syntax is shown below.

[0052] Fig.31 is Fig.30 A diagram showing an example of the syntax is shown below.

[0053] Fig.32 is Fig.31 A diagram showing an example of the syntax is shown below.

[0054] Fig.33 : is a flowchart illustrating an example of the flow of coefficient data derivation processing.

[0055] Fig.34 This is a diagram illustrating method #4.

[0056] Fig.35 This is a diagram illustrating an application example of method #4.

[0057] Fig.36 is a flowchart illustrating an example of the flow of syntax element value derivation processing.

[0058] Fig.37 is Fig.36 Next, a flowchart showing an example of the flow of syntax element value derivation processing will be described.

[0059] Fig.38 is a diagram showing an example of syntax.

[0060] Fig.39 is Fig.38 A diagram showing an example of the syntax is shown below.

[0061] Fig.40 is Fig.39 A diagram showing an example of the syntax is shown below.

[0062] Fig.41 is Fig.40 A diagram showing an example of the syntax is shown below.

[0063] Fig.42 : is a flowchart illustrating an example of the flow of coefficient data derivation processing.

[0064] Fig.43 is Fig.42 The following is a flowchart illustrating an example of the flow of coefficient data derivation processing.

[0065] Fig.44 This is a diagram illustrating method #5.

[0066] Fig.45 This is a diagram illustrating an application example of method #5.

[0067] Fig.46 This is a diagram illustrating method #6.

[0068] Fig.47 It is a diagram illustrating an application example of method #6.

[0069] Fig.48 This is a diagram illustrating method #7.

[0070] Fig.49 It is a diagram illustrating an application example of method #7.

[0071] Fig.50 is a flowchart illustrating an example of the flow of syntax element value derivation processing.

[0072] Fig.51 is a diagram showing an example of syntax.

[0073] Fig.52 is Fig.51 Following are diagrams showing examples of syntax.

[0074] Fig.53 is Fig.52 Following are diagrams showing examples of syntax.

[0075] Fig.54 : is a flowchart illustrating an example of the flow of coefficient data derivation processing.

[0076] Fig.55 This is a diagram for explaining method #8.

[0077] Fig.56 It is a diagram illustrating an application example of method #8.

[0078] Fig.57 is a flowchart illustrating an example of the flow of syntax element value derivation processing.

[0079] Fig.58 : is a flowchart illustrating an example of the flow of coefficient data derivation processing.

[0080] Fig.59 is a block diagram showing a main configuration example of an image encoding device.

[0081] Fig.60 is a flowchart showing an example of the flow of image encoding processing.

[0082] Fig.61 is a block diagram showing a main configuration example of an image decoding device.

[0083] Fig.62 is a flowchart showing an example of the flow of image decoding processing.

[0084] Fig.63 is a block diagram showing a main configuration example of a computer. DETAILED DESCRIPTION

[0085] A mode for carrying out the present disclosure (hereinafter, embodiment) is described below. Note that the description will be presented in the following order.

[0086] 1. CABAC

[0087] 2. First implementation (transferring the number of bits of the zero sub-block to the non-zero sub-block)

[0088] 3. Second implementation (sharing the maximum bit in non-zero sub-blocks)

[0089] 4. Third implementation (using the extra bits of a non-zero sub-block in another non-zero sub-block)

[0090] 5. Fourth Embodiment (Synthesis of #1, #2 and #3)

[0091] 6. Fifth Implementation (Application of #1)

[0092] 7. Sixth Implementation (Extending the Unit for Managing the Number of Bits for Context Coding to a Size Larger than TU (CU, VPDU, CTU, etc.)

[0093] 8. Seventh Implementation Method (Eliminating the Upper Limit of the Maximum Number of Bits for Context Encoding in the Syntax Structure)

[0094] 9. Eighth Implementation Method (Maximizing the Number of Bits of a Non-Zero Sub-Block Next to a Zero Sub-Block)

[0095] 10. Ninth Embodiment (Applied to Image Coding Device and Image Decoding Device)

[0096] 11. Appendix

[0097] <1.CABAC>

[0098] <Documents supporting technical content and technical terms, etc.>

[0099] The scope disclosed in the present technology is not limited to the contents described in the embodiments, but covers the contents described in the following non-patent documents and the like known at the time of filing and the contents of other documents cited in the following non-patent documents.

[0100] Non-patent document 1: (as mentioned above)

[0101] Non-patent document 2: ITU-T Recommendation H.264 (04 / 2017) “Advanced video coding for generic audiovisual services”, April 2017

[0102] Non-patent document 3: ITU-T Recommendation H.265 (12 / 2016) “High efficiency video coding”, December 2016

[0103] Non-Patent Document 4: J. Chen, E. Alshina, G. J. Sullivan, J.-R. Ohm, J. Boyce, “Algorithm Description of Joint Exploration Test Model (JEM7)”, JVET-G1001, ITU-TSG 16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11, Seventh Meeting of the Joint Video Exploration Team (JVET): Turin, Italy, July 13-21, 2017

[0104] Non-Patent Document 5: B. Bross, J. Chen, S. Liu, “Versatile Video Coding (Draft 3)”, JVET-L1001, 12th Meeting of the Joint Video Experts Group (JVET) of ITU-T SG 16WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11: Macau, China, October 3-12, 2018

[0105] Non-Patent Document 6: J.J. Chen, Y.Ye, S.Kim, “Algorithm description for Versatile Video Coding and Test Model 3 (VTM 3)”, JVET-L1002, 12th Meeting of the Joint Video Experts Group (JVET) of ITU-T SG 16WP 3 and ISO / IEC JTC1 / SC 29 / WG 11: Macau, China, October 3-12, 2018

[0106] Non-Patent Document 7: J. Boyce (Intel), Y. Ye (InterDigital), Y.-W. Huang (Mediatek), M. Karczewicz (Qualcomm), E. Francois (Technicolor), W. Husak (Dolby), J. Ridge (Nokia), A. Abbas (GoPro), “Two tier test model”, JVET-J0093, 10th Meeting of the Joint Video Experts Group (JVET) of ITU-T SG 16WP 3 and ISO / IEC JTC1 / SC 29 / WG 11: San Diego, USA, April 10-20, 2018

[0107] Non-Patent Document 8: S. Yoo, J. Choi, J. Heo, J. Choi, L. Li, J. Lim, S. Kim (LGE), “Non-CE7: Residual rearrangement for transform skipped blocks”, JVET-M0278, 13th Meeting of the Joint Video Experts Group (JVET) of ITU-T SG 16WP3 and ISO / IEC JTC 1 / SC 29 / WG 11: Marrakech, MA, January 9-18, 2019

[0108] Non-Patent Document 9: B. Bross, T. Nguyen, P. Keydel, H. Schwarz, D. Marpe, T. Wiegand (HHI), “Non-CE8: Unified Transform Type Signalling and Residual Coding for Transform Skip”, JVET-M0464, 13th Meeting of the Joint Video Experts Group (JVET) of ITU-T SG 16WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11: Marrakech, MA, January 9-18, 2019

[0109] Non-Patent Document 10: Y. Zhao, H. Gao, H. Yang, J. Chen (Huawei), “CE6: Sub-block transform for inter blocks (Test 6.4.1)”, JVET-M0140, 13th Meeting of the Joint Video Experts Group (JVET) of ITU-T SG 16WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11: Marrakech, MA, January 9-18, 2019

[0110] Non-Patent Document 11: S. De-Luxan-Hernandez, V. George, J. Ma, T. Nguyen, H. Schwarz, D. Marpe, T. Wiegand (HHI), “CE3: Intra Sub-Partitions Coding Mode (Tests 1.1.1 and 1.1.2)”, JVET-M0102, Thirteenth Meeting of the Joint Video Experts Group (JVET) of ITU-T SG 16WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11: Marrakech, MA, January 9-18, 2019

[0111] Non-Patent Document 12: B. Bross, J. Chen, S. Liu, “Versatile Video Coding (Draft 4)”, JVET-M1001, 12th Meeting of the Joint Video Experts Group (JVET) of ITU-T SG 16WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11: Macau, China, October 3-12, 2018

[0112] That is, the contents described in the above-mentioned non-patent documents are also the basis for determining the support requirements. For example, even if the quadtree block structure and the quadtree plus binary tree (QTBT) block structure described in the above-mentioned non-patent documents are not directly described in the examples, they are within the scope of the disclosure of the present technology and meet the support requirements of the claims. In addition, for example, even if technical terms such as parsing, grammar, and semantics are not directly described in the examples, they are similarly within the scope of the disclosure of the present technology and meet the support requirements of the claims.

[0113] In addition, in this specification, unless otherwise specified, a "block" (not a block indicating a processing unit) used as a partial area or a processing unit of an image (picture) in the specification indicates any partial area in the picture, and its size, shape, characteristics, etc. are not limited. For example, a "block" includes any partial area (processing unit), such as a transform block (TB), a transform unit (TU), a prediction block (PB), a prediction unit (PU), a minimum coding unit (SCU), a coding unit (CU), a maximum coding unit (LCU), a coding tree block (CTB), a coding tree unit (CTU), a transform block, a subblock, a macroblock, a tile, a slice, etc. described in the non-patent document mentioned above.

[0114] In addition, when specifying the size of such a block, the block size can be specified not only directly but also indirectly. For example, the block size can be specified using identification information that identifies the size. In addition, for example, the block size can be specified by a ratio or difference relative to the size of a reference block (e.g., LCU or SCU). For example, in the case where information for specifying the block size is sent as a syntax element or the like, the information for indirectly specifying the size as described above can be used as the information. In this way, the amount of information can be reduced and the coding efficiency can be improved. In addition, specifying the block size also includes specifying a range of block sizes (e.g., specifying a range of allowable block sizes).

[0115] In addition, in the present specification, encoding includes not only the entire process of converting an image into a bit stream, but also a part of the process. For example, encoding includes not only the process including prediction processing, orthogonal transformation, quantization, arithmetic coding, etc., but also includes the process collectively referred to as quantization and arithmetic coding, and the process including prediction processing, quantization, and arithmetic coding. Similarly, decoding includes not only the entire process of converting a bit stream into an image, but also a part of the process. For example, decoding includes not only the process including inverse arithmetic decoding, inverse quantization, inverse orthogonal transformation, prediction processing, etc., but also includes the process including inverse arithmetic decoding and inverse quantization, and the process including inverse arithmetic decoding, inverse quantization, and prediction processing.

[0116] <Number of context coded bits>

[0117] Context-based adaptive binary arithmetic coding (CABAC) used for image coding such as high-efficiency video coding (HEVC) is a coding method that binarizes syntax element values ​​representing coefficient data derived from image data by a predetermined method, and performs arithmetic coding by switching an appearance frequency model (context) of each bit in the binarized bit string. The coefficient data is represented by syntax elements such as sig_flag indicating a code of the coefficient, gt1_flag indicating whether the absolute value of the coefficient is 1, par_flag indicating whether the coefficient is odd or even, and gt2_flag indicating whether the absolute value of the coefficient is 2.

[0118] Generally, when the number of bits of context coding is larger when expressing a syntax element value, the coding efficiency is better and the image quality degradation can be suppressed more. However, as the number of bits of context coding increases, the amount of processing increases. Therefore, for example, in Non-Patent Document 1, a method of limiting the number of bits of context coding in units of sub-blocks is proposed.

[0119] However, in the case of this method, since the number of bits for context coding is uniformly limited, the coefficient data cannot be locally increased or decreased according to the characteristics of the image, and there is a possibility that the coding efficiency is unnecessarily reduced. In other words, there is a risk of reducing the image quality of the decoded image.

[0120] <Variable limit on the number of bits for context coding>

[0121] Therefore, the limit on the number of bits of context coding in each sub-block is made variable. In this way, the coefficient data can be further locally increased or decreased according to the characteristics of the image, so that the reduction in coding efficiency can be suppressed. In other words, the reduction in image quality of the decoded image can be suppressed.

[0122] At this time, for example, the number of bits in units of blocks (pictures, slices, CTUs, virtual pipeline data units (VPDUs), CUs, access units (AUs), TUs, etc.) can be similar to the case where there are a fixed number of bits for context coding in each sub-block. In this way, it is possible to suppress a decrease in coding efficiency while suppressing an increase in processing volume.

[0123] For example, the number of bits for context coding may be allocated between non-zero sub-blocks in a block, and an upper limit value of the number of bits for context coding allocated to each sub-block may be set. Then, for a processing target sub-block, a syntax element value may be derived from coefficient data corresponding to image data while preventing the number of bits for context coding from exceeding the upper limit value.

[0124] Furthermore, for example, with respect to a processing target subblock, a syntax element value can be parsed while preventing the number of bits of context encoding from exceeding an upper limit value and coefficient data corresponding to image data can be derived.

[0125] Furthermore, for example, the number of bits allocated to a zero subblock in a block may be distributed among non-zero subblocks, and an upper limit value of the number of bits allocated to context coding for each subblock may be set. Then, for a processing target subblock, a syntax element value may be derived from coefficient data corresponding to image data while preventing the number of bits for context coding from exceeding the upper limit value.

[0126] Furthermore, for example, with respect to a processing target subblock, a syntax element value can be parsed while preventing the number of bits of context encoding from exceeding an upper limit value and coefficient data corresponding to image data can be derived.

[0127] <block>

[0128] For example, Figure 1 The block 10 shown in A of FIG. 1 is a processing target block. The block 10 is, for example, an 8×8 pixel TU, and includes four sub-blocks (sub-block 11-1, sub-block 11-2, sub-block 11-3, and sub-block 11-4). Hereinafter, in the case where it is not necessary to describe each sub-block separately from each other, it is referred to as sub-block 11. That is, in Figure 1 In the example of A, the block 10 includes 2x2 sub-blocks 11. Therefore, each sub-block 11 includes 4x4 pixels.

[0129] exist Figure 1 In A, the numerical value in each sub-block 11 indicates the number of bits of context coding generated in the sub-block 11. For example, when the number of bits of context coding of each sub-block 11 is limited to 30 bits, the number of bits of context coding generated in each sub-block 11 is limited to 30 bits or less.

[0130] Note that Figure 1As shown in B, the processing order of each sub-block 11 is the Morton code order (Z-order curve).

[0131] <2. First Embodiment>

[0132] <Method #1>

[0133] For example, Figure 2 As shown in the top row of the table in , the number of context-coded bits in the zero sub-block can be transferred to the non-zero sub-block.

[0134] A zero subblock is a subblock whose coefficients are all zero (0). In addition, a non-zero subblock is a subblock in which there is at least one non-zero coefficient. That is, the number of bits of context coding allocated to the zero subblock in the case where the upper limit value of the number of bits of context coding is uniformly set for each subblock as in the method described in Non-Patent Document 1 is allocated to the non-zero subblock.

[0135] For example, from Figure 2 As shown in the second row from the top of the table shown in , the sub-block coefficient flag (coded_sub_block_flag) is placed outside the existing loop and made independent.

[0136] The sub-block coefficient flag is a flag indicating whether the sub-block contains a non-zero coefficient. For example, in the case of coded_sub_block_flag = 1, it indicates that the sub-block corresponding to the flag is a non-zero sub-block, and in the case of coded_sub_block_flag = 0, it indicates that the sub-block corresponding to the flag is a zero sub-block. For each sub-block, the processing related to the derivation of the syntax element value is subjected to loop processing, but in this method, the processing related to the sub-block coefficient flag is performed outside the loop processing. That is, the sub-block coefficient flags of all sub-blocks in the TU are first parsed.

[0137] Then, count the sub-blocks with coded_sub_block_flag=0, i.e., zero sub-blocks. Then, the number of zero sub-blocks × the number of bits of context coding of one sub-block is transferred (added) to the non-zero sub-blocks. Since the number of bits of context coding is not consumed in the zero sub-block, this number is transferred to the non-zero sub-blocks.

[0138] In addition, the management of the number of bits for context coding is turned off within a TU (predetermined unit). That is, the number of bits for context coding as described above is allocated to each block.

[0139] So, like from Figure 2As shown in the third row from the top of the table shown in , the number of bits for additional context coding in the zero sub-block can be used as the number of bits for context coding in the non-zero sub-block, and the reduction in coding efficiency can be suppressed. In other words, the reduction in image quality of the decoded image can be suppressed.

[0140] For example, Figure 3 As shown in A of FIG. 1 , in the case where a restriction of 30 bits is set for each subblock, even when subblocks 11-2 and 11-3 are zero subblocks, the number of bits of their additional context coding cannot be used for subblocks 11-1 or 11-4 which are non-zero subblocks. That is, the number of bits of context coding in subblocks 11-1 and 11-4 cannot be greater than 30 bits.

[0141] On the other hand, in the case of method #1, if Figure 3 As shown in FIG. 1B, even when subblocks 11-2 and 11-3 are zero subblocks, the number of bits of their additional context encoding can be allocated to subblock 11-1 or subblock 11-4, which is a non-zero subblock. That is, in those subblocks, syntax element values ​​of 30 bits or more can be generated. Therefore, the reduction in encoding efficiency can be suppressed.

[0142] <Encoding device>

[0143] Figure 4 : is a block diagram showing an example of the configuration of an encoding device as one aspect of an image processing device to which the present technology is applied. Figure 4 The encoding device 100 shown in FIG. 1 is a device that encodes coefficient data derived from image data by CABAC and generates encoded data.

[0144] Notice, Figure 4 The main contents such as processing units and data flow are shown, but in Figure 4 That is, in the encoding device 100, there may be Figure 4 The processing units shown as blocks in FIG. 1 may also exist without being shown in FIG. Figure 4 A process or data flow is shown as arrows or the like.

[0145] like Figure 4As shown in , the encoding device 100 includes a sequencer 101 and a CABAC 102. The sequencer 101 acquires coefficient data and the like derived from image data. In addition, the sequencer 101 appropriately acquires control information such as flag information and the like. The sequencer 101 uses the control information to derive syntax element values. In addition, the sequencer 101 provides the derived syntax element values ​​and the like to the CABAC 102. The CABAC 102 acquires the syntax element values ​​and the like provided from the sequencer 101. The CABAC 102 binarizes the syntax element values ​​by a predetermined method, performs arithmetic coding by switching the context of each bit in the binarized bit string, and generates encoded data. The CABAC 102 outputs the generated encoded data to the outside of the encoding device 100.

[0146] Note that these processing units (sequencer 101 and CABAC 102) have arbitrary configurations. For example, each processing unit may include a logic circuit that implements the above-mentioned processing. In addition, each processing unit may include, for example, a central processing unit (CPU), a read-only memory (ROM), a random access memory (RAM), etc., and use them to execute a program to implement the above-mentioned processing. Of course, each processing unit can have two configurations, and part of the above-mentioned processing can be implemented by a logic circuit, while the rest can be implemented by executing a program. The configurations of the various processing units can be independent of each other. For example, some processing units can implement part of the above-mentioned processing by a logic circuit, and some other processing units can execute a program to implement the above-mentioned processing, while still other processing units can implement the above-mentioned processing by both the logic circuit and the execution of the program.

[0147] <cabac>

[0148] Figure 5 1 is a block diagram showing a main configuration example of CABAC 102. Note that Figure 5 The main contents such as processing units and data flow are shown, but in Figure 5 That is, in CABAC 102, there may be Figure 5 The processing units shown as blocks in FIG. 1 may also exist without being shown in FIG. Figure 5 A process or data flow is shown as arrows or the like.

[0149] like Figure 5 As shown in , the CABAC 102 includes a binarization unit 121 , a selection unit 122 , a context model 123 , an arithmetic coding unit 124 , an arithmetic coding unit 125 , and a selection unit 126 .

[0150] The binarization unit 121 acquires the syntax element value supplied from the sequencer 101 , performs binarization using a method defined for each syntax element, and generates a binarized bit string. The binarization unit 121 supplies the binarized bit string to the selection unit 122 .

[0151] The selection unit 122 obtains the binarized bit string and the flag information isBypass provided from the binarization unit 121. The selection unit 122 selects the supply destination of the binarized bit string based on the value of isBypass. For example, in the case of isBypass=0, the selection unit 122 determines that it is a normal mode and provides the binarized bit string to the context model 123. In addition, in the case of isBypass=1, the selection unit 122 determines that it is a bypass mode and provides the binarized bit string to the arithmetic coding unit 125.

[0152] The context model 123 dynamically switches the context model to be applied according to the encoding target and the surrounding situation. For example, the context model 123 holds the context variable ctx, and when the binarized bit string is acquired from the selection unit 122, the context variable ctx corresponding to each bit position (binIdx) of the bit string defined for each syntax element is read. The context model 123 provides the binarized bit string and the read context variable ctx to the arithmetic coding unit 124.

[0153] When the arithmetic coding unit 124 acquires the binarized bit string and the context variable ctx provided from the context model 123, the arithmetic coding unit 124 refers to the probability state of the context variable ctx and performs arithmetic coding (context coding) on ​​the value of the bit in binIdx of the binarized bit string in the CABAC normal mode. The arithmetic coding unit 124 provides the coded data generated by the context coding to the selection unit 126. In addition, the arithmetic coding unit 124 provides the context variable ctx after the context coding process to the context model 123, and causes the context model 123 to maintain the context variable ctx.

[0154] The arithmetic coding unit 125 performs arithmetic coding (bypass coding) on ​​the binarized bit string supplied from the selection unit 122 in the CABAC bypass mode. The arithmetic coding unit 125 supplies the coded data generated by the bypass coding to the selection unit 126.

[0155] The selection unit 126 acquires the flag information isBypass and selects the coded data to be output based on the value of isBypass. For example, in the case of isBypass=0, the selection unit 126 determines that it is a normal mode, acquires the coded data provided from the arithmetic coding unit 124, and outputs it to the outside of the CABAC 102 (encoding device 100). In addition, in the case of isBypass=1, the selection unit 126 determines that it is a bypass mode, acquires the coded data provided from the arithmetic coding unit 125, and outputs it to the outside of the CABAC 102 (encoding device 100).

[0156] Note that these processing units (binarization unit 121 to selection unit 126) have arbitrary configurations. For example, each processing unit may include a logic circuit that implements the above-mentioned processing. In addition, each processing unit may include, for example, a CPU, ROM, RAM, etc., and use them to execute a program to implement the above-mentioned processing. Of course, each processing unit can have two configurations, and a part of the above-mentioned processing can be implemented by a logic circuit, and the rest can be implemented by executing a program. The configurations of each processing unit can be independent of each other. For example, some processing units can implement a part of the above-mentioned processing by a logic circuit, and some other processing units can execute a program to implement the above-mentioned processing, while other processing units can implement the above-mentioned processing by both the execution of a logic circuit and a program.

[0157] <Encoding Process Flow>

[0158] Next, we will refer to Figure 6 The flowchart describes an example of the flow of the encoding process performed by the encoding device 100.

[0159] When the encoding process is started, in step S101 , the sequencer 101 of the encoding device 100 performs syntax element value derivation processing, and derives syntax element values ​​from coefficient data (coefficient data derived from image data) input to the encoding device 100 .

[0160] In step S102 , the CABAC 102 performs CABAC processing, encodes the syntax element value derived in step S101 by CABAC, and generates encoded data. The CABAC 102 outputs the generated encoded data to the outside of the encoding device 100 .

[0161] When the process of step S102 ends, the encoding process ends.

[0162] <Flow of Syntax Element Value Derivation Process>

[0163] Next, we will refer to Figure 7 The flowchart is described in Figure 6 An example of the flow of the syntax element value derivation processing performed in step S101. Figures 8 to 11 is a diagram showing an example of the syntax of residual coding. A description will be given in conjunction with these drawings when necessary. Note that Figures 8 to 11 In the syntax of , the threshold values ​​(TH1 to TH4) can have values ​​such as those described below.

[0164] TH1=6, TH2=28, TH3=2, TH4=4.

[0165] Of course, the value of each threshold is arbitrary and is not limited to this example.

[0166] When the syntax element value derivation process is started, the sequencer 101 encodes the sub-block coefficient flag (coded_sub_block_flag) in the processing target block in step S121, for example Fig. 9 Shown in the second to eleventh rows from the top.

[0167] In step S122, the sequencer 101 counts the number of non-zero sub-blocks (numNonZeroSbk) based on the value of the sub-block coefficient flag, for example Fig. 9 As shown in the thirteenth to sixteenth rows from the top. That is, the sequencer 101 counts the number of sub-blocks of which coded_sub_block_flag=1 of the block to be processed.

[0168] In step S123, the sequencer 101 sets a temporary upper limit value (remBinPass1Tmp) for the number of bits of context coding for each sub-block, for example Fig. 9 As shown in the eighteenth and nineteenth rows from the top. For example, in Figure 3 In the case of , since the sub-block includes 4x4 coefficient data and sig_flag, gt1_flag, par_flag, and gt2_flag are derived, the maximum number of bits for context coding of each sub-block is 64 bits. The sequencer 101 sets the 64 bits to the temporary upper limit value (remBinPass1Tmp) mentioned above. That is, the temporary upper limit value (remBinPass1Tmp) can be set based on the size of the sub-block.

[0169] In step S124, the sequencer 101 sets the number of bits (remBinPass1InZeroSbk) for the additional context coding of all zero sub-blocks based on the number of zero sub-blocks (numZeroSBk), for example Fig. 9 As shown in the twentieth and twenty-first rows from the top. Since the number of subblocks in the processing target block is known from the number of subblock coefficient flags, etc., the number of zero subblocks (numZeroSBk) can be obtained according to the number of non-zero subblocks (numNonZeroSbk) counted in step S122. Since the number of bits required for the zero subblock is zero, the temporary upper limit value (remBinPass1Tmp) set in step S123 is the number of bits for additional context coding. Therefore, the number of bits for additional context coding of all zero subblocks (remBinPass1InZeroSbk) is derived by multiplying the number of zero subblocks (numZeroSBk) by the temporary upper limit value (remBinPass1Tmp).

[0170] In step S125, the sequencer 101 sets the average number of extra context coding bits that can be additionally allocated for each non-zero sub-block (remBinPass1Extra), for example Fig. 9 As shown in the 22nd and 23rd rows from the top. The average number of bits for the extra context coding (remBinPass1Extra) can be derived by dividing the number of bits for the extra context coding of all zero sub-blocks (remBinPass1InZeroSbk) derived in step S124 by the number of zero sub-blocks (numZeroSBk). Note that in order to avoid the division, a lookup table can be used to derive the average number of bits for the extra context coding (remBinPass1Extra).

[0171] Next, the process proceeds to the process for each sub-block. That is, the process proceeds to the process in the loop of the for statement for each sub-block in the syntax. In step S126, the sequencer 101 selects a processing target sub-block (subSetId=i) from among the sub-blocks in the processing target block.

[0172] In step S127, sequencer 101 sets an upper limit value (remBinPass1) for the number of bits of context coding for the processing target sub-block based on the temporary upper limit value (remBinPass1Tmp) and the average number of bits of additional context coding that can be additionally dispatched from the zero sub-block (remBinPass1Extra), for example Fig. 9 as shown in the twenty-ninth and thirtieth lines from the top. In Fig. 9 the case of the example of, the upper limit value (remBinPass1) for the number of bits of context coding is derived by adding the temporary upper limit value (remBinPass1Tmp) and the average number of bits of additional context coding that can be additionally dispatched from the zero sub-block (remBinPass1Extra).

[0173] In step S128, sequencer 101 derives the value of the syntax element for the processing target sub-block so as not to exceed the upper limit value of the number of bits of context coding. This processing corresponds to Fig.10 the syntax shown in and the subsequent drawings.

[0174] In step S129, sequencer 101 determines whether all sub-blocks have been processed. In the case where it is determined that there is an unprocessed sub-block in the processing target block, the processing returns to step S126, and the subsequent processing is repeated. That is, each processing in steps S126 to S129 is executed for each sub-block in the processing target block. Then, in step S129, in the case where it is determined that all sub-blocks in the processing target block have been processed, the syntax element value derivation processing ends, and the processing returns to Figure 6 .

[0175] By executing the processing of each step as described above, sequencer 101 can make the limit of the number of bits of context coding for each sub-block variable. For example, the number of bits of context coding assigned to the zero sub-block in the processing target block can be assigned to the non-zero sub-block. Therefore, the number of bits of context coding assigned to the zero sub-block can also be used. Therefore, a decrease in coding efficiency can be suppressed.

[0176] Note that by encoding the sub-block coefficient flag (coded_sub_block_flag) before the processing for each sub-block and by counting the number of zero sub-blocks and the number of non-zero sub-blocks, the number of bits of additional context coding can be more freely allocated (the restrictions on allocation can be reduced).

[0177] In addition, by turning off the management of the number of bits of context coding in a block (e.g., TU) as described above, an increase in the processing amount can be suppressed.

[0178] <Flow of CABAC processing>

[0179] Next, we will refer to Fig.12 The flowchart is described in Figure 6 An example of the flow of the CABAC processing performed in step S102 of FIG.

[0180] When the CABAC process is started, in step S151 , the binarization unit 121 of the CABAC 102 inputs a syntax element value (syncVal) that is a process target.

[0181] In step S152 , the binarization unit 121 performs the binarization process defined for each syntax element, and derives a bit string (synBins) of the syntax element value (syncVal).

[0182] In step S153 , the selection unit 122 reads the context variable ctx corresponding to each bit position (binIdx) of the bit string defined for each syntax element and the flag isBypass indicating whether it is a bypass mode.

[0183] In step S154, the selection unit 122 determines whether it is the bypass mode. In the case isBypass=0 and it is determined that it is the normal mode, the process proceeds to step S155.

[0184] In step S155, the arithmetic coding unit 124 performs context coding. That is, the arithmetic coding unit 124 encodes the value of the bit at the bit position (binIdx) of the bit string (synBins) in the CABAC normal mode with reference to the probability state of the context variable ctx. When the processing of step S155 ends, the processing proceeds to step S157.

[0185] Furthermore, in the case isBypass=1 and it is determined in step S154 that it is the bypass mode, the process proceeds to step S156.

[0186] In step S156, the arithmetic coding unit 125 performs bypass coding. That is, the arithmetic coding unit 125 encodes the value of the bit at the bit position (binIdx) of the bit string (synBins) in the CABAC bypass mode. When the process of step S156 ends, the process proceeds to step S157.

[0187] In step S157 , the selection unit 126 determines whether a predetermined interruption condition A is satisfied. The interruption condition A is defined based on the value of the bit string from binIdx=0 to binIdx=k (current binIdx position k) and the binarization method for each syntax element.

[0188] In the case where it is determined that the interruption condition A is not satisfied, the process returns to step S153, and the processes of step S153 and subsequent steps are performed for the next bit position (binIdx). That is, the processes of steps S153 to S157 are performed for each bit position (binIdx).

[0189] Then, in step S157, in the case where it is determined that the interruption condition A is satisfied, the CABAC processing ends, and the processing returns to Figure 6 .

[0190] By performing the CABAC process in this manner, syntax element values ​​can be encoded and encoded data can be generated.

[0191] <Decoding device>

[0192] Fig.13 : is a block diagram showing an example of the configuration of a decoding device as one aspect of an image processing device to which the present technology is applied. Fig.13 The decoding device 200 shown in FIG. 1 is a device that decodes coded data and generates coefficient data corresponding to image data.

[0193] Notice, Fig.13 The main contents such as processing units and data flow are shown, but in Fig.13 That is, in the decoding device 200, there may be Fig.13 The processing units shown as blocks in FIG. 1 may also exist without being shown in FIG. Fig.13 A process or data flow is shown as arrows or the like.

[0194] like Fig.13 As shown in , the decoding device 200 includes a CABAC 201 and a parser 202. The CABAC 201 acquires coded data, performs arithmetic decoding on the coded data by switching the context of each bit in the binarized bit string to generate a binarized bit string, and converts the binarized bit string into a plurality of values ​​using a predetermined method to generate a syntax element value. The CABAC 201 provides the generated syntax element value to the parser 202. The parser 202 acquires the syntax element value and parses it to derive coefficient data corresponding to the image data. The parser 202 outputs the derived coefficient data to the outside of the decoding device 200.

[0195] Note that these processing units (CABAC 201 and parser 202) have arbitrary configurations. For example, each processing unit may include a logic circuit that implements the above-mentioned processing. In addition, each processing unit may include, for example, a CPU, ROM, RAM, etc., and use them to execute a program to implement the above-mentioned processing. Of course, each processing unit can have two configurations, and part of the above-mentioned processing can be implemented by a logic circuit, while the rest can be implemented by executing a program. The configurations of the various processing units can be independent of each other. For example, some processing units can implement part of the above-mentioned processing by a logic circuit, and some other processing units can execute a program to implement the above-mentioned processing, while other other processing units can implement the above-mentioned processing by both the execution of a logic circuit and a program.

[0196] <cabac>

[0197] Fig.14 2 is a block diagram showing a main configuration example of CABAC 201. Note that Fig.14 The main contents such as processing units and data flow are shown, but in Fig.14 That is, in CABAC 201, there may be Fig.14 The processing units shown as blocks in FIG. 1 may also exist without being shown in FIG. Fig.14 A process or data flow is shown as arrows or the like.

[0198] like Fig.14 As shown in , CABAC 201 includes a selection unit 221 , a context model 222 , an arithmetic decoding unit 223 , an arithmetic decoding unit 224 , a selection unit 225 , and a multi-value conversion unit 226 .

[0199] The selection unit 221 obtains the coded data and the flag information isBypass input to the decoding device 200. The selection unit 221 selects the supply destination of the coded data based on the value of isBypass. For example, in the case of isBypass=0, the selection unit 221 determines that it is a normal mode and supplies the coded data to the context model 222. In addition, in the case of isBypass=1, the selection unit 221 determines that it is a bypass mode and supplies the binarized bit string to the arithmetic decoding unit 224.

[0200] The context model 222 dynamically switches the context model to be applied according to the decoding target and the surrounding situation. For example, the context model 222 holds the context variable ctx, and when the encoded data is acquired from the selection unit 221, the context variable ctx corresponding to each bit position (binIdx) of the bit string defined for each syntax element is read. The context model 222 provides the encoded data and the read context variable ctx to the arithmetic decoding unit 223.

[0201] When the arithmetic decoding unit 223 obtains the encoded data and the context variable ctx provided from the context model 222, the arithmetic decoding unit 223 refers to the probability state of the context variable ctx and performs arithmetic decoding (context decoding) on ​​the value of the bit in binIdx of the binarized bit string in the CABAC normal mode. The arithmetic decoding unit 223 provides the binarized bit string generated by the context decoding to the selection unit 225. In addition, the arithmetic decoding unit 223 provides the context variable ctx after the context decoding process to the context model 222, and causes the context model 222 to maintain the context variable ctx.

[0202] The arithmetic decoding unit 224 performs arithmetic decoding (bypass decoding) on ​​the encoded data supplied from the selection unit 221 in the CABAC bypass mode. The arithmetic decoding unit 224 supplies the binarized bit string generated by the bypass decoding to the selection unit 225.

[0203] The selection unit 225 acquires the flag information isBypass and selects a binarized bit string to be supplied to the multi-value conversion unit 226 based on the value of isBypass. For example, in the case of isBypass=0, the selection unit 225 determines that it is a normal mode, acquires the binarized bit string supplied from the arithmetic decoding unit 223, and supplies it to the multi-value conversion unit 226. Also, in the case of isBypass=1, the selection unit 225 determines that it is a bypass mode, acquires the binarized bit string supplied from the arithmetic decoding unit 224, and supplies it to the multi-value conversion unit 226.

[0204] The multi-value conversion unit 226 acquires the binarized bit string supplied from the selection unit 225 , converts the binarized bit string into multiple values ​​using a method defined for each syntax element, and generates a syntax element value. The multi-value conversion unit 226 supplies the syntax element value to the parser 202 .

[0205] Note that these processing units (selection unit 221 to multi-value conversion unit 226) have any configuration. For example, each processing unit may include a logic circuit that implements the above-mentioned processing. In addition, each processing unit may include, for example, a CPU, ROM, RAM, etc., and use them to execute a program to implement the above-mentioned processing. Of course, each processing unit can have two configurations, and a part of the above-mentioned processing can be implemented by a logic circuit, and the rest can be implemented by executing a program. The configuration of each processing unit can be independent of each other. For example, some processing units can implement a part of the above-mentioned processing by a logic circuit, and some other processing units can execute a program to implement the above-mentioned processing, and other processing units can implement the above-mentioned processing by both the execution of a logic circuit and a program.

[0206] <Decoding Process Flow>

[0207] Next, we will refer to Fig.15 The flowchart describes an example of the flow of the decoding process performed by the decoding device 200.

[0208] When starting the decoding process, in step S201, the CABAC 201 of the decoding device 200 performs CABAC processing to decode the encoded data input to the decoding device 200 through CABAC and generate syntax element values. In step S202, the parser 202 performs coefficient data derivation processing to parse the syntax element values and derive coefficient data corresponding to the image data. The parser 202 outputs the derived coefficient data to the outside of the decoding device 200.

[0209] When the processing of step S202 ends, the decoding process ends.

[0210] <Flow of CABAC processing>

[0211] Next, a flowchart of Fig.16 will be referred to describe an example of the flow of CABAC processing performed in step S201 of Fig.15

[0212] When starting the CABAC processing, in step S221, the selection unit 221 of the CABAC 201 reads the context variable ctx corresponding to each bit position (binIdx) of the bin string defined for each syntax element and the flag isBypass indicating whether it is in bypass mode.

[0213] In step S222, the selection unit 221 determines whether it is in bypass mode. When isBypass = 0 and it is determined to be in the normal mode, the process proceeds to step S223.

[0214] In step S223, the arithmetic decoding unit 223 performs context decoding. That is, the arithmetic decoding unit 223 decodes the encoded data in the CABAC normal mode with reference to the probability state of the context variable ctx and generates the value of the bit at the bit position (binIdx) of the bit string (synBins). When the processing of step S223 ends, the process proceeds to step S225.

[0215] In addition, when isBypass = 1 and it is determined to be in bypass mode in step S222, the process proceeds to step S224.

[0216] In step S224, the arithmetic decoding unit 224 performs bypass decoding. That is, the arithmetic decoding unit 224 decodes the encoded data in the CABAC bypass mode and generates the value of the bit at the bit position (binIdx) of the bit string (synBins). When the processing of step S224 ends, the process proceeds to step S225.

[0217] In step S225 , the selection unit 225 determines whether a predetermined interruption condition A is satisfied. The interruption condition A is defined based on the value of the bit string from binIdx=0 to binIdx=k (current binIdx position k) and the binarization method for each syntax element.

[0218] In the case where it is determined that the interruption condition A is not satisfied, the process returns to step S221, and the processes of step S221 and subsequent steps are performed to generate the value of the next bit position (binIdx). That is, the processes of steps S221 to S225 are performed for each bit position (binIdx).

[0219] Then, in step S225 , in a case where it is determined that the interruption condition A is satisfied, the process proceeds to step S226 .

[0220] In step S226 , the multi-value conversion unit 226 derives the syntax element value (syncVal) from the bit string (synBins) through the multi-value conversion process defined for each syntax element.

[0221] In step S227 , the multi-value conversion unit 226 supplies the derived syntax element value (syncVal) to the parser 202 .

[0222] By performing CABAC processing in this manner, encoded data can be decoded and syntax element values ​​can be generated.

[0223] <Flow of coefficient data export processing>

[0224] Next, we will refer to Fig.17 The flowchart is described in Fig.15 The coefficient data derivation process is an example of the flow of the coefficient data derivation process performed in step S202 of FIG. Figure 7 ) is basically executed in a similar process. That is, the coefficient data derivation process ( Fig.17 The processing of each step of steps S251 to S259 in the above example is performed in the same manner as the syntax element value derivation processing ( Figure 7 Each processing of steps S121 to S129) is performed in a basically similar manner.

[0225] However, the parser 202 decodes the sub-block coefficient flag (coded_sub_block_flag) in the processing target block in step S251, for example Fig. 9 As shown in the second row to the eleventh row from the top. When the process of step S251 ends, the process proceeds to step S252.

[0226] Furthermore, in step S258, the parser 202 parses the value of the syntax element regarding the processing target subblock so as not to exceed the upper limit value of the number of bits of context coding, and derives coefficient data. This processing corresponds to Fig.10 and the syntax shown in the subsequent drawings. When the processing of step S258 ends, the processing proceeds to step S259.

[0227] By performing the processing of each step as described above, the parser 202 can make the limit of the number of bits of context coding of each sub-block variable. For example, the number of bits of context coding assigned to the zero sub-block in the processing target block can be allocated to the non-zero sub-block. Therefore, the number of bits of context coding assigned to the zero sub-block can also be used. Therefore, the reduction in coding efficiency can be suppressed.

[0228] Note that by decoding the subblock coefficient flag (coded_sub_block_flag) before processing for each subblock and by counting the number of zero subblocks and the number of non-zero subblocks, the number of bits for additional context coding can be allocated more freely (the allocation can be less restricted).

[0229] Furthermore, by turning off the management of the number of bits for context coding in a block (eg, TU) as described above, an increase in the amount of processing can be suppressed.

[0230] <3. Second Embodiment>

[0231] <Method #2>

[0232] Method #1 has been described in the first embodiment, but the present technology is not limited to this method. For example, the maximum number of bits that can be shared in the context coding within the non-zero sub-block is, Fig.18 As shown in the top row of the table.

[0233] That is, the number of bits for context encoding allocated to each non-zero subblock in the case where the upper limit value of the number of bits for context encoding is uniformly set for each subblock as in the method described in Non-Patent Document 1 is shared.

[0234] For example, Fig.18 As shown in the second row from the top of the table shown in , the sub-block coefficient flag (coded_sub_block_flag) is placed outside the existing loop and made independent. That is, the sub-block coefficient flags of all sub-blocks in the TU are parsed first.

[0235] Then, the sub-blocks with coded_sub_block_flag=1, that is, non-zero sub-blocks are counted. Then, the number of non-zero sub-blocks×the number of bits of context coding of one sub-block is shared.

[0236] In addition, the management of the number of bits for context coding is turned off within a TU (predetermined unit). That is, the number of bits for context coding as described above is allocated to each block.

[0237] So, like Fig.18 As shown in the third row from the top of the table shown in , the number of bits for additional context coding in a non-zero sub-block can be used as the number of bits for context coding in another non-zero sub-block, and the reduction in coding efficiency can be suppressed. In other words, the reduction in image quality of the decoded image can be suppressed.

[0238] For example, Fig.19 As shown in FIG. 1A , in the case where a limit of 30 bits is set for each subblock, even when only 24 bits are generated in the subblock 11-1 which is a non-zero subblock, 6 bits from reaching the upper limit value cannot be transferred to another non-zero subblock. Therefore, even in the subblock 11-4 which is a non-zero subblock, the number of bits of context coding cannot be increased by more than 30 bits which is the upper limit value.

[0239] On the other hand, in the case of method #2, if Fig.19 As shown in FIG. 1B , the number of bits (“6” in this example) of additional context coding in the sub-block 11-1 which is a non-zero sub-block can be allocated to the sub-block 11-4 which is a non-zero sub-block. That is, in those non-zero sub-blocks, the number of bits of context coding can be shared, and according to the allocation, 30 bits or more of syntax element values ​​can be generated. Therefore, the reduction in encoding efficiency can be suppressed.

[0240] <Flow of Syntax Element Value Derivation Process>

[0241] In this case as well, the configuration of the encoding device 100 is similar to that of the first embodiment ( Figure 4 ). In addition, the configuration of the CABAC 102 is similar to that of the first embodiment ( Figure 5 ). In addition, the flow of the encoding process performed by the encoding device 100 is similar to that of the first embodiment ( Figure 6 ). In addition, the flow of CABAC processing is similar to that of the first embodiment ( Fig.12 ).

[0242] Will refer to Fig. 20 The flowchart of FIG. 1 describes an example of the flow of the syntax element value derivation process in this case. Figure 21 to Figure 24 is a diagram showing an example of the syntax of residual coding. A description will be given in conjunction with these drawings when necessary. Note that Figure 21 to Figure 24 In the syntax of , the threshold values ​​(TH1 to TH3) can have values ​​such as those described below.

[0243] TH1=6, TH2=28, TH3=3.

[0244] Of course, the value of each threshold is arbitrary and is not limited to this example.

[0245] When the syntax element value derivation process is started, the sequencer 101 encodes the sub-block coefficient flag (coded_sub_block_flag) in the processing target block in step S301, for example Fig. 22 Shown in rows 1 to 10 from the top.

[0246] In step S302, the sequencer 101 counts the number of non-zero sub-blocks (numNonZeroSbk) based on the value of the sub-block coefficient flag, for example Fig. 22 As shown in the twelfth to fifteenth rows from the top. That is, the sequencer 101 counts the number of sub-blocks of which coded_sub_block_flag=1 of the block to be processed.

[0247] In step S303, the sequencer 101 sets a temporary upper limit value (remBinPass1Tmp) for the number of bits of context coding for each sub-block, for example Fig. 22 As shown in the seventeenth and eighteenth rows from the top. For example, in Fig.19 In the case of Figure 3 Similar to the case of , the maximum number of bits for context coding of each sub-block is 64 bits. The sequencer 101 sets the 64 bits to the temporary upper limit value (remBinPass1Tmp) mentioned above. That is, the temporary upper limit value (remBinPass1Tmp) can be set based on the size of the sub-block.

[0248] In step S304, the sequencer 101 sets the number of bits (remBinPass1) for additional context coding that can be allocated in all non-zero sub-blocks based on the number of non-zero sub-blocks (numNonZeroSbk), for example Fig. 22 As shown in the 19th and 20th rows from the top. The number of bits (remBinPass1) for additional context coding that can be allocated in all non-zero sub-blocks is derived from the product of the number of non-zero sub-blocks (numNonZeroSbk) counted in step S302 and the temporary upper limit value (remBinPass1Tmp) set in step S303, that is, the sum of the temporary upper limit values ​​(remBinPass1Tmp) allocated to each non-zero sub-block.

[0249] Next, the process proceeds to the process for each sub-block. That is, the process proceeds to the process in the loop of the for statement in the syntax for each sub-block. In step S305, the sequencer 101 selects a processing target sub-block (subSetId=i) from among the sub-blocks in the processing target block.

[0250] In step S306, the sequencer 101 derives the value of the syntax element about the processing target sub-block so that the sum of the number of bits generated in each sub-block of the processing target block does not exceed the number of bits (remBinPass1) of additional context coding that can be allocated in all non-zero sub-blocks derived in step S304. This process corresponds to Fig.23 and the syntax shown in subsequent figures.

[0251] In step S307, the sequencer 101 determines whether all sub-blocks have been processed. In the case where it is determined that there are unprocessed sub-blocks in the processing target block, the process returns to step S305, and the processes of step S305 and subsequent steps are repeated. That is, each process of step S305 to step S307 is performed for each sub-block in the processing target block. Then, in step S307, in the case where it is determined that all sub-blocks in the processing target block have been processed, the syntax element value derivation process ends, and the process returns to step S306. Figure 6 .

[0252] By performing the processing of each step as described above, the sequencer 101 can make the limit of the number of bits of context coding for each sub-block variable. For example, the number of bits of context coding can be distributed between non-zero sub-blocks in the processing target block. Therefore, the reduction in coding efficiency can be suppressed.

[0253] Note that by encoding the subblock coefficient flag (coded_sub_block_flag) before processing each subblock and by counting the number of zero subblocks and the number of non-zero subblocks, the number of bits for additional context coding can be allocated more freely (the allocation can be less restricted).

[0254] Furthermore, by turning off the management of the number of bits for context coding in a block (eg, TU) as described above, an increase in the amount of processing can be suppressed.

[0255] <Flow of coefficient data export processing>

[0256] In this case as well, the configuration of the decoding device 200 is similar to that of the first embodiment (the case of Fig.13 ). In addition, the configuration of CABAC 201 is similar to that of the first embodiment ( Fig.14 ). In addition, the flow of the decoding process performed by the decoding device 200 is similar to the case of the first embodiment ( Fig.15 ). In addition, the flow of CABAC processing is similar to that of the first embodiment ( Fig.16 ).

[0257] Will refer to Fig.25 The flowchart of FIG. 1 describes an example of the flow of the coefficient data derivation process in this case. The coefficient data derivation process is similar to the syntax element value derivation process ( Fig. 20 ) is basically executed in a similar process. That is, the coefficient data derivation process ( Fig.25 Each step of the process of steps S321 to S327) is processed in the same manner as the syntax element value derivation process ( Fig. 20 The processing of each process of steps S301 to S307) is basically performed in a similar manner.

[0258] However, the parser 202 decodes the sub-block coefficient flag (coded_sub_block_flag) in the processing target block in step S321, for example Fig. 22 As shown in the first row to the tenth row from the top. When the process of step S321 ends, the process proceeds to step S322.

[0259] Furthermore, in step S326, the parser 202 derives coefficient data by parsing the syntax element value about the processing target subblock so that the sum of the number of bits generated in each subblock of the processing target block does not exceed the number of bits (remBinPass1) for additional context coding that can be allocated in all non-zero subblocks derived in step S324. This process corresponds to Fig.23 and the syntax shown in the subsequent drawings. When the processing of step S326 ends, the processing proceeds to step S327.

[0260] By performing the processing of each step as described above, the parser 202 can make the limit of the number of bits of context coding for each sub-block variable. For example, the number of bits of context coding can be allocated between non-zero sub-blocks in the processing target block. Therefore, the reduction in coding efficiency can be suppressed. The number of bits of context coding assigned to the zero sub-block in the processing target block can be allocated to the non-zero sub-block. Therefore, the number of bits of context coding assigned to the zero sub-block can also be used. Therefore, the reduction in coding efficiency can be suppressed.

[0261] Note that by decoding the subblock coefficient flag (coded_sub_block_flag) before processing for each subblock and by counting the number of zero subblocks and the number of non-zero subblocks, the number of bits for additional context coding can be allocated more freely (the allocation can be less restricted).

[0262] Furthermore, by turning off the management of the number of bits for context coding in a block (eg, TU) as described above, an increase in the amount of processing can be suppressed.

[0263] <4. Third Embodiment>

[0264] <Method #3>

[0265] In addition, for example, the number of bits for the additional context coding in a non-zero sub-block can be used in another non-zero sub-block, such as Fig.26 Shown in the top row of the table.

[0266] For example, Fig.26 As shown in the second row from the top of the table shown in , in the case where there is a sub-block that does not reach the maximum value of the number of bits for context coding per sub-block, the number of additional context coding bits is pooled and used for the next and subsequent sub-blocks.

[0267] Thus, the sub-block coefficient flags can be applied without leaving the existing loop, such as in Fig.26 That is, the increase in the amount of changes in the grammar can be suppressed, and the present technology can be applied more easily.

[0268] In addition, the number of bits for additional context coding in a non-zero subblock can be used as the number of bits for context coding in another non-zero subblock, and a decrease in coding efficiency can be suppressed. In other words, a decrease in image quality of a decoded image can be suppressed.

[0269] For example, Fig. 27 As shown in FIG. 1A , when a 30-bit limit is set for each sub-block, even when only 16 bits are generated in sub-block 11-3, sub-block 11-4 is also subject to the 30-bit limit. That is, 14 bits of the difference upper limit value cannot be transferred to the next and subsequent sub-blocks.

[0270] On the other hand, in the case of method #3, if Fig. 27 As shown in FIG. 1B, the number of bits (in this example, "14") for the additional context coding in the sub-block 11-3 can be allocated to the sub-block 11-4 to be processed next. That is, in the sub-block 11-4, 30 bits or more of syntax element values ​​can be generated. Therefore, the reduction in coding efficiency can be suppressed.

[0271] <Flow of Syntax Element Value Derivation Process>

[0272] In this case as well, the configuration of the encoding device 100 is similar to that of the first embodiment ( Figure 4 ). In addition, the configuration of the CABAC 102 is similar to that of the first embodiment ( Figure 5 ). In addition, the flow of the encoding process performed by the encoding device 100 is similar to that of the first embodiment ( Figure 6 ). In addition, the flow of CABAC processing is similar to that of the first embodiment ( Fig.12 ).

[0273] Will refer to Fig.28 The flowchart describes an example of the flow of the syntax element value derivation process in this case. Figure 29 to Figure 32 is a diagram showing an example of the syntax of residual coding. A description will be given in conjunction with these drawings when necessary. Note that Figure 29 to Figure 32 In the syntax of , the threshold values ​​(TH1 to TH3) can have values ​​such as those described below.

[0274] TH1=6, TH2=28, TH3=3.

[0275] Of course, the value of each threshold is arbitrary and is not limited to this example.

[0276] When the syntax element value derivation process starts, in step S341, the sequencer 101 initializes the number of bits (remBinPass1Next) that can be allocated to the next sub-block for additional context coding (eg, remBinPass1Next=0), for example Fig.30 This is shown in the first and second rows from the top. That is, by default, the number of bits for the additional context coding is zero, indicating that the number of bits cannot be added to the next sub-block.

[0277] Next, the process proceeds to the process for each sub-block. That is, the process proceeds to the process in the loop of the for statement for each sub-block in the syntax. In step S342, the sequencer 101 selects a processing target sub-block (subSetId=i) from among the sub-blocks in the processing target block.

[0278] The sequencer 101 encodes the sub-block coefficient flag (coded_sub_block_flag) regarding the processing target sub-block in step S343, for example Fig.30 Shown in rows 3 through 11 from the top.

[0279] In step S344, the sequencer 101 sets a temporary upper limit value (remBinPass1Tmp) for the number of bits of context coding of the processing target sub-block, for example Fig.30 As shown in the fourteenth and fifteenth rows from the top. For example, in Fig. 27 In the case of Figure 3 Similar to the case of , the maximum number of bits for context coding of each sub-block is 64 bits. The sequencer 101 sets the 64 bits to the temporary upper limit value (remBinPass1Tmp) mentioned above. That is, the temporary upper limit value (remBinPass1Tmp) can be set based on the size of the sub-block.

[0280] In step S345, the sequencer 101 sets the upper limit value (remBinPass1) of the number of bits for context coding based on the temporary upper limit value (remBinPass1Tmp) set for the processing target sub-block in step S344 and the number of bits (remBinPass1Next) of the additional context coding of the immediately preceding processed sub-block that can be allocated to the next sub-block, e.g. Fig.30 As shown in the sixteenth and seventeenth rows from the top, for example, the upper limit value (remBinPass1) of the number of bits for context coding of the processing target subblock is derived from the sum of the temporary upper limit value (remBinPass1Tmp) and the number of bits for additional context coding that can be allocated to the next subblock (remBinPass1Next).

[0281] In step S346 , the sequencer 101 derives the syntax element value so as not to exceed the upper limit value ( remBinPass1 ) of the number of bits of context encoding set for the processing target subblock in step S345 .

[0282] In step S347, the sequencer 101 sets the number of bits for the additional context coding of the processing target sub-block (remBinPass1 after deriving the syntax element value) to the number of bits for the additional context coding that can be allocated to the next sub-block (remBinPass1Next), for example Fig.31 Shown in the twenty-first and twenty-second rows from the top.

[0283] In step S348, the sequencer 101 determines whether all sub-blocks have been processed. In the case where it is determined that there are unprocessed sub-blocks in the processing target block, the process returns to step S342, and the processes of step S342 and subsequent steps are repeated. That is, each process of step S342 to step S348 is performed for each sub-block in the processing target block. Then, in step S348, in the case where it is determined that all sub-blocks in the processing target block have been processed, the syntax element value derivation process ends, and the process returns to step S342. Figure 6 .

[0284] By performing the processing of each step as described above, the sequencer 101 can make the limit of the number of bits of context encoding of each sub-block variable. For example, the number of bits of additional context encoding of the processing target sub-block can be used as the number of bits of context encoding of the next and subsequent sub-blocks to be processed. Therefore, the reduction in encoding efficiency can be suppressed.

[0285] Note that by turning off the management of the number of bits for context coding in a block (eg, TU) as described above, an increase in the amount of processing can be suppressed.

[0286] <Flow of coefficient data export processing>

[0287] In this case as well, the configuration of the decoding device 200 is similar to that of the first embodiment ( Fig.13 ). In addition, the configuration of CABAC 201 is similar to that of the first embodiment ( Fig.14 ). In addition, the flow of the decoding process performed by the decoding device 200 is similar to the case of the first embodiment ( Fig.15 ). In addition, the flow of CABAC processing is similar to that of the first embodiment ( Fig.16 ).

[0288] Will refer to Fig.33 The flowchart of FIG. 1 describes an example of the flow of the coefficient data derivation process in this case. The coefficient data derivation process is similar to the syntax element value derivation process ( Fig.28 ) is basically executed in a similar process. That is, each step of the coefficient data derivation process ( Fig.33 The processing of steps S361 to S368) is performed in accordance with each processing of the syntax element value derivation processing ( Fig.28 The processing of steps S341 to S348) is performed in a basically similar manner.

[0289] However, the parser 202 decodes the sub-block coefficient flag (coded_sub_block_flag) in the processing target sub-block in step S363, for example Fig.30 As shown in the third row to the eleventh row from the top. When the processing of step S363 ends, the processing proceeds to step S364.

[0290] Furthermore, in step S366, the parser 202 derives coefficient data by parsing the syntax element value so as not to exceed the upper limit value (remBinPass1) of the number of bits of context coding set for the processing target subblock in step S365. When the processing of step S366 ends, the processing proceeds to step S367.

[0291] By performing the processing of each step as described above, the parser 202 can make the limit of the number of bits of context coding of each sub-block variable. For example, the number of bits of additional context coding of the processing target sub-block can be used as the number of bits of context coding of the next and subsequent sub-blocks to be processed. Therefore, it is possible to suppress the reduction of coding efficiency.

[0292] Note that by turning off the management of the number of bits for context coding in a block (eg, TU) as described above, an increase in the amount of processing can be suppressed.

[0293] <5. Fourth embodiment>

[0294] <Method #4>

[0295] The above methods can be combined arbitrarily as long as there is no contradiction. Fig.34 As shown in the top row of the table, the above methods #1 to #3 can be used in combination.

[0296] For example, Fig.34 As shown in the second row from the top of the table shown, the sub-block coefficient flag (coded_sub_block_flag) is placed outside the existing loop and made independent. That is, as in methods #1 and #2, the sub-block coefficient flags of all sub-blocks in the TU are first parsed.

[0297] Then, as in methods #1 and #2, sub-blocks with coded_sub_block_flag=0, that is, zero sub-blocks are counted.

[0298] Then, as in method #1, the number of zero sub-blocks × the number of context coded bits of one sub-block is transferred (added) to the non-zero sub-blocks. Also, as in method 2, the number of context coded bits allocated to each non-zero sub-block is shared. Also, as in method #3, the number of additional context coded bits generated in the processing target sub-block is pooled and used for the next and subsequent sub-blocks.

[0299] In addition, as in methods #1 to #3, the management of the number of bits for context coding is turned off within a TU (predetermined unit). That is, the number of bits for context coding as described above is allocated to each block.

[0300] So, like Fig.34 As shown in the third row from the top of the table shown, the effects of each of the above methods #1 to #3 can be obtained. Therefore, the coding efficiency can be suppressed from decreasing. In other words, the image quality of the decoded image can be suppressed from decreasing.

[0301] For example, Fig.35 As shown in A, in the case where a limit of 30 bits is set for each subblock, even when subblock 11-2 and subblock 11-3 are zero subblocks, the number of bits of their additional context coding cannot be used in subblock 11-1 or subblock 11-4, which is a non-zero subblock. In addition, the number of bits of additional context coding cannot be shared between non-zero subblocks. In addition, even when only 20 bits are generated in subblock 11-1, which is a non-zero subblock, 10 bits to the upper limit value cannot be transferred to subblock 11-4, which is another next and subsequent non-zero subblock to be processed. Therefore, subblock 11-4 is limited to 30 bits, and the number of bits of context coding cannot be increased to more than 30 bits as the upper limit value.

[0302] On the other hand, in the case of method #4, if Fig.35 As shown in FIG. 1B , the number of bits for the additional context coding of the subblocks 11-2 and 11-3 as zero subblocks can be allocated to the subblock 11-1 or the subblock 11-4 as a non-zero subblock. In addition, the number of bits for the context coding can be shared between the subblocks 11-1 and 11-4 as non-zero subblocks. In addition, the number of bits for the additional context coding of the subblock 11-1 (in Fig.35 In the case of A, 10 bits are allocated to the sub-block 11-4. That is, in the sub-block, a syntax element value of 30 bits or more can be generated. Therefore, the reduction in coding efficiency can be suppressed.

[0303] <Flow of Syntax Element Value Derivation Process>

[0304] In this case as well, the configuration of the encoding device 100 is similar to that of the first embodiment ( Figure 4 ). In addition, the configuration of the CABAC 102 is similar to that of the first embodiment ( Figure 5 ). In addition, the flow of the encoding process performed by the encoding device 100 is similar to that of the first embodiment ( Figure 6 ). In addition, the flow of CABAC processing is similar to that of the first embodiment ( Fig.12 ).

[0305] Will refer to Fig.36 and Fig.37 The flowchart describes an example of the flow of the syntax element value derivation process in this case. Figures 38 to 41 is a diagram showing an example of the syntax of residual coding. A description will be given in conjunction with these drawings when necessary. Note that Figures 38 to 41 In the syntax of , the threshold values ​​(TH1 to TH4) can have values ​​such as those described below.

[0306] TH1=6, TH2=28, TH3=2, TH4=4

[0307] Of course, the value of each threshold is arbitrary and is not limited to this example.

[0308] When the syntax element value derivation process is started, the sequencer 101 Fig.36 In step S381, the sub-block coefficient flag (coded_sub_block_flag) in the processing target block is encoded, for example Fig.39 Shown in rows 1 to 10 from the top.

[0309] In step S382, the sequencer 101 counts the number of non-zero sub-blocks (numNonZeroSbk) based on the value of the sub-block coefficient flag shown, for example Fig.39 As shown in the twelfth to fifteenth rows from the top. That is, the sequencer 101 counts the number of sub-blocks of which coded_sub_block_flag=1 of the block to be processed.

[0310] In step S383, the sequencer 101 sets a temporary upper limit value (remBinPass1Tmp) for the number of bits of context coding for each sub-block as shown, for example Fig.39 As shown in the seventeenth and eighteenth rows from the top. Similar to the case of method #1, the temporary upper limit value (remBinPass1Tmp) can be set based on the size of the sub-block.

[0311] In step S384, the sequencer 101 sets the number of bits for the additional context coding of all zero sub-blocks (remBinPass1InZeroSbk) based on the number of zero sub-blocks (numZeroSBk), for example Fig.39 As shown in the 19th and 20th rows from the top. Similar to the case of method #1, the number of zero subblocks (numZeroSBk) can be determined according to the number of non-zero subblocks (numNonZeroSbk) counted in step S382, and the number of bits (remBinPass1InZeroSbk) for the additional context coding of all zero subblocks can be derived by multiplying the number of zero subblocks (numZeroSBk) by the temporary upper limit value (remBinPass1Tmp).

[0312] In step S385, the sequencer 101 sets the average number of extra context coding bits that can be allocated to each non-zero sub-block (remBinPass1Extra), for example Fig.39 As shown in the 21st and 22nd rows from the top. The average number of bits for the extra context coding (remBinPass1Extra) can be derived by dividing the number of bits for the extra context coding of all zero sub-blocks (remBinPass1InZeroSbk) derived in step S384 by the number of zero sub-blocks (numZeroSBk). Note that in order to avoid the division, a lookup table can be used to derive the average number of bits for the extra context coding (remBinPass1Extra).

[0313] In step S386, the sequencer 101 initializes the number of additional context encoding bits (remBinPass1Next=0) that can be allocated to the next sub-block shown (eg, remBinPass1Next=0), for example Fig.39 As shown in the twenty-fourth and twenty-fifth rows from the top. That is, by default, the number of bits for the additional context encoding is zero, which means that the number of bits cannot be added to the next sub-block.

[0314] When the processing of step S386 is completed, the processing proceeds to Fig.37 .

[0315] Next, the process proceeds to the process for each sub-block. That is, the process proceeds to the process in the loop of the for statement for each sub-block in the syntax. In step S391, the sequencer 101 selects a processing target sub-block (subSetId=i) from among the sub-blocks in the processing target block.

[0316] In step S392, the sequencer 101 sets an upper limit value (remBinPass1) for the number of bits of context coding of the processing target sub-block shown based on the temporary upper limit value (remBinPass1Tmp), the average number of bits of additional context coding that can be additionally allocated from the zero sub-block (remBinPass1Extra), and the number of bits of additional context coding of the previous sub-block, e.g. Fig.40 Shown in the fifth and sixth rows from the top. Fig.40 In the example case, the upper limit value for the number of context coded bits (remBinPass1) is derived from the sum of the temporary upper limit value (remBinPass1Tmp), the average number of additional context coded bits that can be additionally allocated from the zero sub-block (remBinPass1Extra), and the sum of the number of additional context coded bits that can be allocated to the next sub-block (remBinPass1Next).

[0317] In step S393 , the sequencer 101 derives syntax element values ​​so as not to exceed an upper limit value ( remBinPass1 ) of the number of bits used to process the context encoding of the target sub-block.

[0318] In step S394, the sequencer 101 sets the number of bits for the additional context coding of the processing target sub-block (remBinPass1 after the derived syntax element value) to the number of bits for the additional context coding that can be allocated to the next sub-block shown (remBinPass1Next), for example Fig.40 Shown in rows 40 and 41 from the top.

[0319] In step S395, the sequencer 101 determines whether all sub-blocks have been processed. In the case where it is determined that there are unprocessed sub-blocks in the processing target block, the process returns to step S391, and the processes of step S391 and subsequent steps are repeated. That is, each process of step S391 to step S395 is performed for each sub-block in the processing target block. Then, in step S395, in the case where it is determined that all sub-blocks in the processing target block have been processed, the syntax element value derivation process ends, and the process returns to step S391. Figure 6 .

[0320] By performing the processing of each step as described above, the sequencer 101 can limit the number of bits of context encoding of each sub-block variable. That is, the effect of each method of the above methods #1 to #3 can be obtained. Therefore, the reduction in coding efficiency can be suppressed. In other words, the reduction in image quality of the decoded image can be suppressed.

[0321] <Flow of coefficient data export processing>

[0322] In this case as well, the configuration of the decoding device 200 is similar to that of the first embodiment ( Fig.13 ). In addition, the configuration of CABAC 201 is similar to that of the first embodiment ( Fig.14 ). In addition, the flow of the decoding process performed by the decoding device 200 is similar to the case of the first embodiment ( Fig.15 ). In addition, the flow of CABAC processing is similar to that of the first embodiment ( Fig.16 ).

[0323] Will refer to Fig.42 and Fig.43 The flowchart of FIG. 1 describes an example of the flow of the coefficient data derivation process in this case. The coefficient data derivation process is similar to the syntax element value derivation process ( Fig.36 and Fig.37 ) is executed in a process roughly similar to that of deriving the syntax element value ( Fig.36 Steps S381 to S386 and Fig.37 The coefficient data derivation processing ( Fig.42 Steps S411 to S416 and Fig.43 Processing of each step of steps S421 to S425).

[0324] However, the parser 202 Fig.42 In step S411, the sub-block coefficient flag (coded_sub_block_flag) in the processing target sub-block is decoded, for example Fig.39 As shown in the first row to the tenth row from the top. When the process of step S411 ends, the process proceeds to step S412.

[0325] In addition, Fig.43 In step S423, the parser 202 derives coefficient data by parsing the syntax element value so as not to exceed the upper limit value (remBinPass1) of the number of bits of context encoding set for the processing target subblock in step S422. When the processing of step S423 ends, the processing proceeds to step S424.

[0326] By performing the processing of each step as described above, the parser 202 can limit the number of bits of context encoding for each sub-block variable. That is, the effect of each method of the above methods #1 to #3 can be obtained. Therefore, the reduction in coding efficiency can be suppressed. In other words, the reduction in image quality of the decoded image can be suppressed.

[0327] <6. Fifth embodiment>

[0328] <Method #5>

[0329] In addition, for example, as Fig.44 By applying method #1 shown in the top row of the table, the priority of the number of bits used to allocate additional context coding can be set according to the position of the sub-block.

[0330] For example, Fig.44 As shown in the second row from the top of the table shown in , in the case of a low-range sub-block, a large number of bits for additional context coding can be allocated, while in the case of a high-range sub-block, a smaller number of bits for additional context coding can be allocated. In addition, conversely, in the case of a high-range sub-block, a large number of bits for additional context coding can be allocated, while in the case of a low-range sub-block, a smaller number of bits for additional context coding can be allocated.

[0331] So, like Fig.44 As shown in the third row from the top of the table shown in , the number of bits for additional context coding can be used as the number of bits for context coding in another subblock, and a decrease in coding efficiency can be suppressed. In other words, a decrease in image quality of a decoded image can be suppressed.

[0332] For example, in Fig.45 In the case of A, a larger number of bits for additional context encoding than that of the subblock 11-4 as the high range subblock is allocated to the subblock 11-1 as the low range subblock. Generally, coefficient data obtained by orthogonally transforming image data is collected in the low range rather than in the high range. Therefore, in this case, it is possible to Fig.45 As in the example of A, a large number of additional context coding bits are allocated to the low-range sub-blocks to further suppress the reduction in coding efficiency.

[0333] In addition, for example, in Fig.45 In the case of B, the number of bits of additional context encoding larger than the number of bits of additional context encoding of the subblock 11-1 as the low-range subblock is allocated to the subblock 11-4 as the high-range subblock. When the orthogonal transform is skipped, for example, as in the case of lossless encoding, the high-range coefficient data generally tends to be larger than in the case where the orthogonal transform is performed. In addition, for example, in the case of an image with many edge components, the high-range coefficient data tends to become larger. In this case, it is possible to Fig.45 As in the example of B, a large number of additional context coding bits are allocated to the high-range sub-blocks to further suppress the reduction in coding efficiency.

[0334] In this case as well, the configuration of the encoding device 100 is similar to that of the first embodiment ( Figure 4 ). In addition, the configuration of the CABAC 102 is similar to that of the first embodiment ( Figure 5 ). In addition, the flow of the encoding process performed by the encoding device 100 is similar to that of the first embodiment ( Figure 6 ). In addition, the flow of CABAC processing is similar to that of the first embodiment ( Fig.12 ).

[0335] In addition, the syntax element value derivation process is basically similar to the case of the first embodiment ( Figure 7 ). In step S127, as described above, when setting the upper limit value of the number of bits of context encoding for the processing target subblock, it is only necessary to set the upper limit value by considering the position of the processing target subblock.

[0336] Furthermore, also in this case, the configuration of the decoding device 200 is similar to that of the first embodiment ( Fig.13 ). In addition, the configuration of CABAC 201 is similar to that of the first embodiment ( Fig.14 ). In addition, the flow of the decoding process performed by the decoding device 200 is similar to the case of the first embodiment ( Fig.15 ). In addition, the flow of CABAC processing is similar to that of the first embodiment ( Fig.16 ).

[0337] In addition, the coefficient data derivation process is basically similar to the case of the first embodiment ( Fig.17 ). In step S257, as described above, when setting the upper limit value of the number of bits of context encoding for the processing target subblock, it is only necessary to set the upper limit value by considering the position of the processing target subblock.

[0338] <7. Sixth Implementation Method>

[0339] <Method #6>

[0340] In addition, for example, Fig.46 As shown in the top row of the table, the unit used to manage the number of bits for context coding can be extended to be larger than the size of the TU.

[0341] For example, from Fig.46 As shown in the second row from the top of the table shown in , the number of bits for additional context coding may be managed, for example, for each CU, VPDU, or CTU. That is, the number of bits may be allocated between non-zero sub-blocks in a CU, VPDU, or CTU to set an upper limit value for the number of bits allocated to a processing target sub-block.

[0342] In this way, the number of bits for the additional context coding can be shared over a larger range. Fig.46 As shown in the third row from the top of the table shown, the number of bits for context coding can be allocated with a higher degree of freedom. Therefore, it is possible to suppress a decrease in coding efficiency. In other words, it is possible to suppress a decrease in the image quality of the decoded image.

[0343] For example, Fig.47 As shown in A of FIG. 4 , it is assumed that CU 400 includes 4×4 sub-blocks 401 (sub-blocks 401 - 1 to 401 - 16). In addition, it is assumed that each sub-block is processed in a Morton code order as shown by an arrow in the figure.

[0344] In such Fig.47 In the case of the limitation of 30 bits per subblock shown in B, even when there is a zero subblock, the number of bits for additional context coding cannot be used in the next subblock and subsequent subblocks. Therefore, the number of bits for context coding is limited to 30 bits or less in all subblocks.

[0345] Therefore, method #6 is applied to share the number of bits for the additional context coding within CU 400. In this case, Fig.47 As shown in C, the number of bits of additional context coding generated in the processing target sub-block can be allocated to the next and subsequent sub-blocks to be processed. Therefore, in sub-block 401, a syntax element value of 30 or more bits can be generated. Therefore, the reduction in coding efficiency can be suppressed. In particular, in the case of method #6, since the range of the number of bits for sharing additional context coding is wider than the range of TU, ​​the number of bits for context coding can be allocated with a higher degree of freedom.

[0346] In this case as well, the configuration of the encoding device 100 is similar to that of the first embodiment ( Figure 4 ). In addition, the configuration of the CABAC 102 is similar to that of the first embodiment ( Figure 5 ). In addition, the flow of the encoding process performed by the encoding device 100 is similar to that of the first embodiment ( Figure 6 ). In addition, the flow of CABAC processing is similar to that of the first embodiment ( Fig.12 ).

[0347] In addition, the syntax element value derivation process is basically similar to the case of the first embodiment ( Figure 7 ). It is sufficient as long as the processing target block is a data unit larger than TU (such as CU, VPDU or CTU).

[0348] Furthermore, also in this case, the configuration of the decoding device 200 is similar to that of the first embodiment ( Fig.13 ). In addition, the configuration of CABAC 201 is similar to that of the first embodiment ( Fig.14 ). In addition, the flow of the decoding process performed by the decoding device 200 is similar to the case of the first embodiment ( Fig.15 ). In addition, the flow of CABAC processing is similar to that of the first embodiment ( Fig.16 ).

[0349] In addition, the coefficient data derivation process is basically similar to the case of the first embodiment ( Fig.17 ). It is sufficient as long as the processing target block is a data unit larger than TU (such as CU, VPDU or CTU).

[0350] <8. Seventh Implementation Method>

[0351] <Method #7>

[0352] In addition, for example, Fig.48 As shown in the top row of the table, the upper limit of the maximum number of bits for context coding in the syntax structure can be omitted (not set).

[0353] For example, Fig.48 As shown in the second row from the top of the table shown, the number of bits generated may be limited by level restriction, conditional expression, etc. In addition, instead of a sub-block unit, the amount of generation may be limited in a data unit larger than a sub-block (e.g., a TU unit, a CU unit, a VPDU unit, a CTU unit, a slice unit, a picture unit, etc.).

[0354] So, like Fig.48 As shown in the third row from the top of the table shown, when generating local coefficient data, many coefficients can be represented by flags. Therefore, it is possible to suppress the reduction in encoding efficiency. In other words, it is possible to suppress the reduction in image quality of the decoded image.

[0355] For example, Fig.49 As shown in A, in the case where a limit of 28 bits is set for each subblock, even when the number of bits for additional context coding is generated, the generated number of bits for additional context coding cannot be used in another subblock. Therefore, for example, even in subblock 11-4, the number of bits for context coding cannot be increased to exceed 28 bits as the upper limit value.

[0356] On the other hand, in the case of method #7, if Fig.49 As shown in B, the number of bits for context coding can be shared in the CU unit, for example. Therefore, 28 bits or more of context coding bits can be allocated to the sub-block. Therefore, the reduction in coding efficiency can be suppressed.

[0357] <Flow of Syntax Element Value Derivation Process>

[0358] In this case as well, the configuration of the encoding device 100 is similar to that of the first embodiment ( Figure 4 ). In addition, the configuration of the CABAC 102 is similar to that of the first embodiment ( Figure 5 ). In addition, the flow of the encoding process performed by the encoding device 100 is similar to that of the first embodiment ( Figure 6 ). In addition, the flow of CABAC processing is similar to that of the first embodiment ( Fig.12 ).

[0359] Will refer to Fig.50 The flowchart of describes an example of the flow of the syntax element value derivation process in this case. Note that here, the case where the number of bits of context encoding is shared in the CU unit will be described as an example. In addition, Figure 51 to Figure 53 1 and 2 are diagrams showing examples of syntax for residual coding. A description will be given in conjunction with these drawings when necessary.

[0360] When the syntax element value derivation process starts, in step S441, the sequencer 101 sets the upper limit value (remBinsPass1) of the number of bits of context coding that can be allocated in all CUs based on the number of bits of context coding that can be allocated for each coefficient ("*7)>>2") and the CU size (log2TbWidth, log2TbHeight), for example Fig.51 Shown in row 21 from the top.

[0361] Next, the process proceeds to the process for each sub-block. In step S442 , the sequencer 101 selects a processing target sub-block from among the sub-blocks in the processing target CU.

[0362] In step S443 , the sequencer 101 derives the syntax element value of the processing target subblock so as not to exceed the upper limit value ( remBinsPass1 ) of the number of bits for context encoding that can be allocated in all CUs set in step S441 .

[0363] In step S444, the sequencer 101 determines whether all sub-blocks have been processed. In the case where it is determined that there are unprocessed sub-blocks in the processing target CU, the process returns to step S442, and the processes of step S442 and subsequent steps are repeated. That is, each process of step S442 to step S444 is performed for each sub-block in the processing target CU. Then, in step S444, in the case where it is determined that all sub-blocks in the processing target CU have been processed, the syntax element value derivation process ends, and the process returns to step S442. Figure 6 .

[0364] By performing the processing of each step as described above, the sequencer 101 can limit the number of bits of context coding for each sub-block variable. For example, the number of bits of context coding can be shared in the CU unit. Therefore, the reduction in coding efficiency can be suppressed.

[0365] Note that, as described above, by turning off management of the number of bits for context encoding in the above-mentioned block (eg, CU), an increase in the amount of processing can be suppressed.

[0366] <Flow of coefficient data export processing>

[0367] In this case as well, the configuration of the decoding device 200 is similar to that of the first embodiment ( Fig.13 ). In addition, the configuration of CABAC 201 is similar to that of the first embodiment ( Fig.14 ). In addition, the flow of the decoding process performed by the decoding device 200 is similar to the case of the first embodiment ( Fig.15 ). In addition, the flow of CABAC processing is similar to that of the first embodiment ( Fig.16 ).

[0368] Will refer to Fig.54 The flowchart of FIG. 1 describes an example of the flow of the coefficient data derivation process in this case. The coefficient data derivation process is similar to the syntax element value derivation process ( Fig.50 ) is performed by a process that is basically similar to that of FIG. That is, the coefficient data derivation process ( Fig.54 Each step of the processing of steps S461 to S464) is performed in the same manner as the syntax element value derivation processing ( Fig.50 The processing of each process of steps S441 to S444) is performed in a basically similar manner.

[0369] However, in step S463, the parser 202 derives coefficient data by parsing the syntax element value for the processing target subblock so as not to exceed the upper limit value (remBinsPass1) of the number of bits for context coding that can be allocated in all CUs set in step S461. When the processing of step S463 ends, the processing proceeds to step S464.

[0370] By performing the processing of each step as described above, the parser 202 can limit the number of bits of context coding for each sub-block variable. For example, the number of bits of context coding can be shared in the CU unit. Therefore, the reduction in coding efficiency can be suppressed.

[0371] Note that, as described above, by turning off management of the number of bits for context encoding in the above-mentioned block (eg, CU), an increase in the amount of processing can be suppressed.

[0372] <9. Eighth Implementation Method>

[0373] <Method #8>

[0374] In addition, for example, Fig.55 As shown in the top row of the table, the upper limit of the maximum number of bits for context coding in the syntax structure can be omitted (not set).

[0375] At this time, for example, Fig.55 As shown in the second row from the top of the table shown, the maximum value of the number of bits that can be generated in the non-zero sub-block after the zero sub-block can be set to the upper limit value. In addition, in the case where the zero sub-blocks are continuous, the maximum value of the number of bits that can be generated in the same number of non-zero sub-blocks can be set to the upper limit value. That is, the maximum value of the number of bits that can be generated in the same number of non-zero sub-blocks as the zero sub-blocks can be set to the upper limit value.

[0376] So, like Fig.55 As shown in the third row from the top of the table shown in , when generating local coefficient data, many coefficients can be represented by flags. Therefore, it is possible to suppress the reduction in encoding efficiency. In other words, it is possible to suppress the reduction in image quality of the decoded image.

[0377] For example, in Fig.56 In the case of A, each subblock has a limit of 28 bits. Subblocks 11-1 and 11-3 are zero subblocks, and the number of bits for additional context coding (28 bits) is generated. However, in this case, the number of bits for additional context coding cannot be used in another subblock. That is, in subblocks 11-2 and 11-4, which are non-zero subblocks, bits greater than the upper limit value, i.e., 28 bits, cannot be allocated.

[0378] On the other hand, in the case of method #8, the maximum value of the number of bits that can be generated can be set to the upper limit value in the non-zero sub-block after the zero sub-block. Fig.56 As shown in FIG. 1B , since sub-block 11-1 is a zero sub-block, the maximum value of the number of bits (48 bits) generated in one sub-block can be assigned as an upper limit value to sub-block 11-2, which is the next non-zero sub-block to be processed. Similarly, since sub-block 11-3 is a zero sub-block, the maximum value of the number of bits (48 bits) generated in one sub-block can be assigned as an upper limit value to sub-block 11-4, which is the next non-zero sub-block to be processed.

[0379] Furthermore, with a limit of 28 bits per sub-block, even in Fig.56 In the case where the sub-blocks 11-1 and 11-2 shown in C are zero sub-blocks, the number of bits for additional context coding cannot be used in the sub-blocks 111-3 or 11-4 which are non-zero sub-blocks.

[0380] On the other hand, in the case of method #8, in the case of such an arrangement of sub-blocks, as Fig.56 As shown in D, sub-blocks 11-1 and 11-2 are consecutive zero sub-blocks. Therefore, in the two next and subsequent non-zero sub-blocks to be processed, the maximum value of the number of bits generated in one sub-block can be assigned as the upper limit value. That is, in sub-blocks 11-3 and 11-4, the maximum value of the number of bits generated in one sub-block (48 bits) can be assigned as the upper limit value.

[0381] <Flow of Syntax Element Value Derivation Process>

[0382] In this case as well, the configuration of the encoding device 100 is similar to that of the first embodiment ( Figure 4 ). In addition, the configuration of the CABAC 102 is similar to that of the first embodiment ( Figure 5 ). In addition, the flow of the encoding process performed by the encoding device 100 is similar to that of the first embodiment ( Figure 6 ). In addition, the flow of CABAC processing is similar to that of the first embodiment ( Fig.12 ).

[0383] Reference Fig.57 The flowchart of FIG. 1 describes an example of the flow of the syntax element value derivation process in this case.

[0384] When the syntax element value derivation process starts, the sequencer 101 initializes the count value of the zero subblock in step S481. For example, the sequencer 101 sets the count value of the zero subblock to "0".

[0385] Next, the process proceeds to processing for each sub-block. In step S482 , the sequencer 101 selects a processing target sub-block.

[0386] In step S483 , the sequencer 101 encodes the sub-block coefficient flag (coded_sub_block_flag) of the processing target sub-block.

[0387] In step S484, the sequencer 101 determines whether the processing target subblock is a non-zero block based on the value of the subblock coefficient flag. In the case where coded_sub_block_flag=1 and it is determined that the processing target subblock is a non-zero subblock, the process proceeds to step S485.

[0388] In step S485, the sequencer 101 determines whether the count value of the zero sub-block is “0.” In the case where it is determined that the count value is “0,” the process proceeds to step S486. In this case, the limitation of the number of bits is applied to the processing target sub-block.

[0389] Therefore, in step S486, the sequencer 101 sets the upper limit value (remBinPass1) of the number of bits of context encoding of the processing target subblock. That is, the upper limit value (remBinPass1) of the number of bits of context encoding is set based on the limitation of the number of bits of the processing target subblock.

[0390] In step S487, the sequencer 101 derives the syntax element value so as not to exceed the upper limit value (remBinPass1) of the number of bits of context encoding set for the processing target subblock in step S486. When the processing of step S487 ends, the processing proceeds to step S491.

[0391] Furthermore, in the event that it is determined in step S485 that the count value of the zero sub-block is "1" or greater, the process proceeds to step S488.

[0392] In step S488, the sequencer 101 derives the syntax element value for processing the target sub-block. In this case, the number of bits of the additional context coding from the zero sub-block may be added, and the limit on the number of bits is not applied to the processing target sub-block. That is, the syntax element value is derived without setting the upper limit value (remBinPass1) of the number of bits of the context coding.

[0393] In step S489, the sequencer 101 decrements the count value of the zero sub-block (for example, subtracts “1” from the count value). When the processing of step S489 ends, the processing proceeds to step S491.

[0394] In addition, in the case where it is determined in step S484 that the processing target block is a zero subblock, the process proceeds to step S490. In this case, the derivation of the syntax element value is omitted. In step S490, the sequencer 101 increments the count value of the zero subblock (for example, adds "1" to the count value). When the process of step S490 ends, the process proceeds to step S491.

[0395] In step S491, the sequencer 101 determines whether all sub-blocks have been processed. In the case where it is determined that there are unprocessed sub-blocks in the processing target block, the process returns to step S482, and the processes of step S482 and subsequent steps are repeated. That is, each process of step S482 to step S491 is performed for each sub-block in the processing target block. Then, in step S491, in the case where it is determined that all sub-blocks in the processing target block have been processed, the syntax element value derivation process ends, and the process returns to step S482. Figure 6 .

[0396] As described above, by using the count value of the zero sub-block, the setting of the upper limit value (remBinPass1) of the number of bits of the context encoding of the same number of non-zero sub-blocks as the zero sub-block can be omitted. That is, by performing the processing of each step as described above, the sequencer 101 can limit the number of bits of the context encoding of each sub-block variable. Therefore, the reduction in encoding efficiency can be suppressed. In other words, the reduction in image quality of the decoded image can be suppressed.

[0397] <Flow of coefficient data export processing>

[0398] In this case as well, the configuration of the decoding device 200 is similar to that of the first embodiment ( Fig.13 ). In addition, the configuration of CABAC 201 is similar to that of the first embodiment ( Fig.14 ). In addition, the flow of the decoding process performed by the decoding device 200 is similar to the case of the first embodiment ( Fig.15 ). In addition, the flow of CABAC processing is similar to that of the first embodiment ( Fig.16 ).

[0399] Will refer to Fig.58 The flowchart of FIG. 1 describes an example of the flow of the coefficient data derivation process in this case. Fig.57 ) is performed in a process basically similar to that of the syntax element value derivation process ( Fig.57 The coefficient data derivation processing ( Fig.58 Processing of each step of steps S521 to S531).

[0400] However, the parser 202 Fig.58 In step S523, the sub-block coefficient flag (coded_sub_block_flag) in the processing target sub-block is decoded. When the processing of step S523 ends, the processing proceeds to step S524.

[0401] In addition, Fig.58 In step S527, the parser 202 derives coefficient data by parsing the syntax element value so as not to exceed the upper limit value (remBinPass1) of the number of bits of context encoding set for the processing target subblock in step S526. When the processing of step S527 ends, the processing proceeds to step S531.

[0402] In addition, Fig.58 In step S528, the parser 202 derives coefficient data by parsing the syntax element value for the processing target subblock. When the process of step S528 ends, the process proceeds to step S529.

[0403] As described above, by using the count value of the zero sub-block, the setting of the upper limit value (remBinPass1) of the number of bits of the context encoding of the same number of non-zero sub-blocks as the zero sub-block can be omitted. That is, by performing the processing of each step as described above, the parser 202 can limit the number of bits of the context encoding of each sub-block variable. Therefore, the reduction in coding efficiency can be suppressed. In other words, the reduction in image quality of the decoded image can be suppressed.

[0404] <10. Ninth Implementation Method>

[0405] <Image Coding Device>

[0406] In the above, an example in which the present technology is applied to the encoding device 100 and the decoding device 200 has been described, but the present technology can be applied not only to the above technology but also to any device, apparatus, system, etc. For example, the present technology can be applied to an image encoding device that encodes image data.

[0407] Fig.59 : is a block diagram showing a configuration example of an image encoding device as one aspect of an image processing device to which the present technology is applied. Fig.59 The image encoding device 600 shown is a device that encodes image data of a moving image. For example, the image encoding device 600 encodes the image data of a moving image by the encoding method described in any one of non-patent documents 1 to 12.

[0408] Notice, Fig.59 The main contents such as processing units (blocks) and data flow are shown, and no Fig.59 That is, in the image encoding device 600, there may be Fig.59 The processing units shown as blocks in FIG. 1 may exist but are not shown in FIG. Fig.59 A process or data flow is shown as arrows or the like.

[0409] like Fig.59 As shown, the image encoding device 600 includes a control unit 601, a sorting buffer 611, an operation unit 612, an orthogonal transform unit 613, a quantization unit 614, an encoding unit 615, a storage buffer 616, an inverse quantization unit 617, an inverse orthogonal transform unit 618, an operation unit 619, a loop filter unit 620, a frame memory 621, a prediction unit 622 and a rate control unit 623.

[0410] <Control Unit>

[0411] The control unit 601 divides the moving image data held by the sorting buffer 611 into blocks (CU, PU, ​​transform block, etc.) in the processing unit based on the block size in the external or predetermined processing unit. In addition, the control unit 601 determines the encoding parameters (header information Hinfo, prediction mode information Pinfo, transform information Tinfo, filter information Finfo, etc.) provided to each block based on, for example, rate-distortion optimization (RDO).

[0412] The details of these encoding parameters will be described later. When the control unit 601 determines the above encoding parameters, the control unit 601 provides them to each block. Specifically, it is described as follows.

[0413] Header information Hinfo is provided to each block.

[0414] The prediction mode information Pinfo is provided to the encoding unit 615 and the prediction unit 622 .

[0415] The transformation information Tinfo is provided to the encoding unit 615 , the orthogonal transformation unit 613 , the quantization unit 614 , the inverse quantization unit 617 , and the inverse orthogonal transformation unit 618 .

[0416] The filter information Finfo is provided to the loop filter unit 620 .

[0417] <Sort Buffer>

[0418] Each field (input image) of the moving image data is input to the image encoding device 600 in a reproduction order (display order). The sorting buffer 611 acquires and holds (stores) each input image in a reproduction order (display order). The sorting buffer 611 sorts the input images in a coding order (decoding order) or divides the input images into blocks in the processing unit based on the control of the control unit 601. The sorting buffer 611 provides each processed input image to the operation unit 612. In addition, the sorting buffer 611 also provides each input image (original image) to the prediction unit 622 and the loop filter unit 620.

[0419] <Calculation Unit>

[0420] The operation unit 612 takes the image I corresponding to the block in the processing unit and the predicted image P provided from the prediction unit 622 as input, subtracts the predicted image P from the image I as shown in the following formula, derives the prediction residual D and provides it to the orthogonal transformation unit 613.

[0421] D=IP

[0422] <Orthogonal Transformation Unit>

[0423] The orthogonal transform unit 613 uses the prediction residual D provided from the operation unit 612 and the transform information Tinfo provided from the control unit 601 as input, and performs an orthogonal transform on the prediction residual D based on the transform information Tinfo to derive a transform coefficient Coeff. Note that the orthogonal transform unit 613 may perform an adaptive orthogonal transform (AMT) to adaptively select the type of orthogonal transform (transform coefficient). The orthogonal transform unit 613 provides the obtained transform coefficient Coeff to the quantization unit 614.

[0424] <Quantization Unit>

[0425] The quantization unit 614 takes the transform coefficient Coeff provided from the orthogonal transform unit 613 and the transform information Tinfo provided from the control unit 601 as input, and scales (quantizes) the transform coefficient Coeff based on the transform information Tinfo. Note that the rate of quantization is controlled by the rate control unit 623. The quantization unit 614 provides the quantized transform coefficient obtained by such quantization, that is, the quantized transform coefficient level, to the encoding unit 615 and the inverse quantization unit 617.

[0426] <Coding unit>

[0427] The encoding unit 615 takes as input the quantized transform coefficient level scale supplied from the quantization unit 614, various encoding parameters (header information Hinfo, prediction mode information Pinfo, transform information Tinfo, filter information Finfo, etc.) supplied from the control unit 601, information related to the filter such as the filter coefficient supplied from the loop filter unit 620, and information related to the optimal prediction mode supplied from the prediction unit 622. The encoding unit 615 performs variable length encoding (e.g., arithmetic encoding) of the quantized transform coefficient level scale to generate a bit string (encoded data).

[0428] Furthermore, the encoding unit 615 derives residual information Rinfo from the quantized transform coefficient level, encodes the residual information Rinfo, and generates a bit string.

[0429] In addition, the encoding unit 615 includes information about the filter provided from the loop filter unit 620 in the filter information Finfo, and includes information about the optimal prediction mode provided from the prediction unit 622 in the prediction mode information Pinfo. Then, the encoding unit 615 encodes the above-mentioned various encoding parameters (header information Hinfo, prediction mode information Pinfo, transformation information Tinfo, filter information Finfo, etc.) to generate a bit string.

[0430] Furthermore, the encoding unit 615 multiplexes the bit strings of various information generated as described above to generate encoded data. The encoding unit 615 supplies the encoded data to the storage buffer 616.

[0431] <Storage Buffer>

[0432] The storage buffer 616 temporarily holds the coded data obtained by the coding unit 615. The storage buffer 616 outputs the held coded data as, for example, a bit stream to the outside of the image coding device 600 at a predetermined timing. For example, the coded data is transmitted to the decoding side via any recording medium, any transmission medium, any information processing device, etc. That is, the storage buffer 616 is also a transmission unit that transmits the coded data (bit stream).

[0433] <Inverse Quantization Unit>

[0434] The inverse quantization unit 617 performs processing related to inverse quantization. For example, the inverse quantization unit 617 takes the quantized transform coefficient level supplied from the quantization unit 614 and the transform information Tinfo supplied from the control unit 601 as inputs, and scales (inverse quantizes) the value of the quantized transform coefficient level based on the transform information Tinfo. Note that this inverse quantization is an inverse process of the quantization performed in the quantization unit 614. The inverse quantization unit 617 supplies the transform coefficient Coeff_IQ obtained by this inverse quantization to the inverse orthogonal transform unit 618.

[0435] <Inverse Orthogonal Transformation Unit>

[0436] The inverse orthogonal transform unit 618 performs processing related to inverse orthogonal transform. For example, the inverse orthogonal transform unit 618 takes the transform coefficient Coeff_IQ provided from the inverse quantization unit 617 and the transform information Tinfo provided from the control unit 601 as input, and performs an inverse orthogonal transform of the transform coefficient Coeff_IQ based on the transform information Tinfo to derive the prediction residual D'. Note that this inverse orthogonal transform is an inverse process of the orthogonal transform performed in the orthogonal transform unit 613. That is, the inverse orthogonal transform unit 618 can perform an adaptive inverse orthogonal transform (AMT) to adaptively select the type (transform coefficient) of the inverse orthogonal transform.

[0437] The inverse orthogonal transform unit 618 provides the prediction residual D' obtained by this inverse orthogonal transform to the operation unit 619. Note that since the inverse orthogonal transform unit 618 is similar to the inverse orthogonal transform unit on the decoding side (described later), the description given on the decoding side (described later) can be applied to the inverse orthogonal transform unit 618.

[0438] <Calculation Unit>

[0439] The operation unit 619 takes as input the prediction residual D' provided from the inverse orthogonal transform unit 618 and the prediction image P provided from the prediction unit 622. The operation unit 619 adds the prediction residual D' and the prediction image P corresponding to the prediction residual D' to derive a local decoded image Rlocal. The operation unit 619 provides the derived local decoded image Rlocal to the loop filter unit 620 and the frame memory 621.

[0440] <Loop filter unit>

[0441] The loop filter unit 620 performs processing related to the loop filtering process. For example, the loop filter unit 620 uses the local decoded image Rlocal provided from the operation unit 619, the filter information Finfo provided from the control unit 601, and the input image (original image) provided from the sorting buffer 611 as input. Note that the information input to the loop filter unit 620 is arbitrary, and information other than such information can be input. For example, the prediction mode, motion information, code amount target value, quantization parameter QP, picture type, block information (CU, CTU, etc.), etc. can be input to the loop filter unit 620 as needed.

[0442] The loop filter unit 620 appropriately performs a filtering process on the local decoded image Rlocal based on the filter information Finfo. The loop filter unit 620 also uses an input image (original image) and other input information for filtering processing as necessary.

[0443] For example, the loop filter unit 620 applies four loop filters: a bidirectional filter, a deblocking filter (DBF), an adaptive offset filter (sample adaptive offset (SAO)), and an adaptive loop filter (ALF) in the order described in non-patent document 11. Note that which filter is applied and in what order is arbitrary and can be appropriately selected.

[0444] Of course, the filtering process performed by the loop filter unit 620 is arbitrary and not limited to the above example. For example, the loop filter unit 620 may apply Wiener filtering or the like.

[0445] The loop filter unit 620 supplies the local decoded image Rlocal that has been subjected to filtering to the frame memory 621. Note that the loop filter unit 620 supplies the information on the filter to the encoding unit 615 in the case where information on the filter such as the filter coefficient is supplied to the decoding side.

[0446] <Frame Memory>

[0447] The frame memory 621 performs processing related to storing data related to an image. For example, the frame memory 621 takes the local decoded image Rlocal provided from the operation unit 619 or the local decoded image Rlocal provided from the loop filter unit 620 after filtering as input, and holds (stores) the decoded image Rlocal. In addition, the frame memory 621 reconstructs the decoded image R for each picture unit using the local decoded image Rlocal, and holds the decoded image R (stores it in a buffer in the frame memory 621). The frame memory 621 provides the decoded image R (or a part thereof) to the prediction unit 622 in response to a request of the prediction unit 622.

[0448] <Prediction Unit>

[0449] The prediction unit 622 performs processing related to the generation of a predicted image. For example, the prediction unit 622 takes the prediction mode information Pinfo provided from the control unit 601, the input image (original image) provided from the sorting buffer 611, and the decoded image R (or a part thereof) read from the frame memory 621 as input. The prediction unit 622 performs prediction processing such as inter-frame prediction and intra-frame prediction using the prediction mode information Pinfo or the input image (original image), performs prediction by using the decoded image R as a reference image, performs motion compensation processing based on the prediction result, and generates a predicted image P. The prediction unit 622 provides the generated predicted image P to the operation unit 612 and the operation unit 619. In addition, the prediction unit 622 provides information related to the prediction mode selected by the above-mentioned processing, that is, the optimal prediction mode, to the encoding unit 615 as needed.

[0450] <Rate Control Unit>

[0451] The rate control unit 623 performs processing related to rate control. For example, the rate control unit 623 controls the rate of the quantization operation of the quantization unit 614 based on the code amount of the encoded data stored in the storage buffer 616 so that overflow or underflow does not occur.

[0452] Note that these processing units (control unit 601, sorting buffer 611 to rate control unit 623) have arbitrary configurations. For example, each processing unit may include a logic circuit that implements the above-mentioned processing. In addition, each processing unit may include, for example, a CPU, ROM, RAM, etc., and use them to execute a program to implement the above-mentioned processing. Of course, each processing unit can have two configurations, and part of the above-mentioned processing can be implemented by a logic circuit, and the rest can be implemented by executing a program. The configurations of the various processing units can be independent of each other. For example, some processing units can implement part of the above-mentioned processing by a logic circuit, and some other processing units can execute a program to implement the above-mentioned processing, while other processing units can implement the above-mentioned processing by both a logic circuit and a program.

[0453] In the image encoding device 600 having the above configuration, the present technology is applied to the encoding unit 615. That is, the encoding unit 615 has Figure 4 The image encoding device 600 is configured similarly to the configuration of the encoding device 100 shown in FIG. 1 and performs similar processing. That is, the encoding unit 615 performs one of the above-mentioned methods #1 to #8. In this way, the image encoding device 600 can obtain an effect similar to the effect described in any one of the first embodiment to the eighth embodiment above. Therefore, the image encoding device 600 can suppress the reduction of encoding efficiency (improve encoding efficiency). In other words, the image encoding device 600 can suppress the reduction of the image quality of the decoded image.

[0454] <Flow of Image Coding Process>

[0455] Next, we will refer to Fig.60 The flowchart describes an example of the flow of an image encoding process performed by the image encoding device 600 having the above configuration.

[0456] When the image encoding process is started, in step S601 , the sorting buffer 611 is controlled by the control unit 601 to sort the frame order of the input moving image data from the display order to the encoding order.

[0457] In step S602 , the control unit 601 sets a processing unit (performs block division) on the input image held by the sorting buffer 611 .

[0458] In step S603 , the control unit 601 determines (sets) encoding parameters of the input image held by the sorting buffer 611 .

[0459] In step S604, the prediction unit 622 performs prediction processing and generates a predicted image of the optimal prediction mode, etc. For example, in the prediction processing, the prediction unit 622 performs intra prediction to generate a predicted image of the optimal intra prediction mode, etc., performs inter prediction to generate a predicted image of the optimal inter prediction mode, etc., and selects the optimal prediction mode therefrom based on a cost function value, etc.

[0460] In step S605, the operation unit 612 arithmetically calculates the difference between the input image and the predicted image of the optimal mode selected by the prediction process in step S604. That is, the operation unit 612 generates a prediction residual D between the input image and the predicted image. The data amount of the prediction residual D obtained in this way is smaller than the data amount of the original image data. Therefore, compared with the case where the image is encoded as it is, the data amount can be compressed.

[0461] In step S606 , the orthogonal transform unit 613 performs an orthogonal transform process on the prediction residual D generated by the process of step S605 , and derives a transform coefficient Coeff.

[0462] In step S607 , the quantization unit 614 quantizes the transform coefficient Coeff obtained by the process of step S606 , for example, by using the quantization parameter calculated by the control unit 601 , and derives the quantized transform coefficient level.

[0463] In step S608 , the inverse quantization unit 617 inversely quantizes the quantized transform coefficient level generated by the process of step S607 with a feature corresponding to the feature of the quantization in step S607 , and derives a transform coefficient Coeff_IQ.

[0464] In step S609, the inverse orthogonal transform unit 618 performs inverse orthogonal transform on the transform coefficient Coeff_IQ obtained by the process of step S608 using a method corresponding to the orthogonal transform process of step S606, and derives the prediction residual D'. Note that since the inverse orthogonal transform process is similar to the inverse orthogonal transform process (described later) performed on the decoding side, the description given on the decoding side (described later) can be applied to the inverse orthogonal transform process of step S609.

[0465] In step S610 , the operation unit 619 adds the predicted image obtained by the prediction process of step S604 and the prediction residual D′ derived by the process of step S609 to generate a locally decoded decoded image.

[0466] In step S611 , the loop filter unit 620 performs a loop filtering process on the locally decoded decoded image obtained through the process of step S610 .

[0467] In step S612 , the frame memory 621 stores the locally decoded decoded image derived through the process of step S610 or the locally decoded decoded image subjected to the filtering process in step S611 .

[0468] In step S613, the encoding unit 615 encodes the quantized transform coefficient level obtained by the process of step S607. For example, the encoding unit 615 encodes the quantized transform coefficient level as information about the image by arithmetic coding or the like to generate encoded data. In addition, at this time, the encoding unit 615 encodes various encoding parameters (header information Hinfo, prediction mode information Pinfo, transform information Tinfo). In addition, the encoding unit 615 obtains residual information RInfo from the quantized transform coefficient level, and encodes the residual information RInfo.

[0469] In step S614, the storage buffer 616 stores the obtained coded data and outputs it as a bit stream to the outside of the image coding device 600, for example. The bit stream is transmitted to the decoding side via a transmission path or a recording medium, for example. In addition, the rate control unit 623 performs rate control as needed.

[0470] When the process of step S614 ends, the image encoding process ends.

[0471] In the above-described image encoding process, the present technology is applied to the encoding process of step S613. That is, in step S613, a process similar to Figure 6 That is, the encoding unit 615 performs the encoding process using one of the above-mentioned methods #1 to #8. In this way, the image encoding device 600 can obtain an effect similar to the effect described in any one of the first to eighth embodiments above. Therefore, the image encoding device 600 can suppress the reduction in encoding efficiency (improve encoding efficiency). In other words, the image encoding device 600 can suppress the reduction in image quality of the decoded image.

[0472] <Image decoding device>

[0473] Furthermore, for example, the present technology can be applied to an image decoding device that decodes encoded data of image data. Fig.61 : is a block diagram showing a configuration example of an image decoding device as one aspect of an image processing device to which the present technology is applied. Fig.61 The image decoding device 700 shown is a device for decoding the coded data of the moving image. For example, the image decoding device 700 decodes the coded data using the decoding method described in any one of non-patent documents 1 to 12. For example, the image decoding device 700 decodes the coded data (bit stream) generated by the image coding device 600 described above.

[0474] Notice, Fig.61 The main contents such as processing units and data flow are shown, and no Fig.61 That is, in the image decoding apparatus 700, there may be Fig.61 The processing units shown as blocks in FIG. 1 may exist but are not shown in FIG. Fig.61 A process or data flow is shown as arrows or the like.

[0475] exist Fig.61 , the image decoding device 700 includes a storage buffer 711, a decoding unit 712, an inverse quantization unit 713, an inverse orthogonal transformation unit 714, an operation unit 715, a loop filter unit 716, a sorting buffer 717, a frame memory 718, and a prediction unit 719. Note that the prediction unit 719 includes an intra-frame prediction unit and an inter-frame prediction unit not shown. The image decoding device 700 is a device for generating moving image data by decoding encoded data (bit stream).

[0476] <Storage Buffer>

[0477] The storage buffer 711 acquires and holds (stores) a bit stream input to the image decoding device 700. The storage buffer 711 supplies the stored bit stream to the decoding unit 712 at a predetermined timing or, for example, when a predetermined condition is satisfied.

[0478] <Decoding Unit>

[0479] The decoding unit 712 performs processing related to image decoding. For example, the decoding unit 712 takes the bit stream supplied from the storage buffer 711 as input, performs variable length decoding on the syntax value of each syntax element from the bit string according to the definition of the syntax table, and derives parameters.

[0480] The syntax elements and the parameters derived from the syntax values ​​of the syntax elements include, for example, information such as header information Hinfo, prediction mode information Pinfo, transform information Tinfo, residual information Rinfo, and filter information Finfo. That is, the decoding unit 712 parses (analyzes and obtains) this information from the bitstream. This information will be described below.

[0481] <Header Information Hinfo>

[0482] The header information Hinfo includes, for example, header information such as video parameter set (VPS) / sequence parameter set (SPS) / picture parameter set (PPS) / slice header (SH). The header information Hinfo includes, for example, specified image size (width PicWidth, height PicHeight), bit depth (luminance bitDepthY, chrominance bitDepthC), chrominance array type ChromaArrayType, maximum value MaxCUSize / minimum value MinCUSize of CU size, maximum depth MaxQTDepth / minimum depth MinQTDepth of quadtree division (also called Quad-tree division), maximum depth MaxBTDepth / minimum depth MinBTDepth of binarytree division (Binary-tree division), maximum value MaxTSSize (also called maximum transform skip block size) of transform skip block, on / off flag (also called enable flag) of each encoding tool.

[0483] For example, the on / off flag of the coding tool included in the header information Hinfo includes an on / off flag related to the transformation and quantization processing shown below. Note that the on / off flag of the coding tool can also be interpreted as a flag indicating whether there is a syntax related to the coding tool in the coded data. In addition, when the value of the on / off flag is 1 (true), it means that the coding tool can be used, and when the value of the on / off flag is 0 (false), it means that the coding tool cannot be used. Note that the interpretation of the flag value can be reversed.

[0484] Cross-component prediction enabled flag (ccp_enabled_flag): Flag information indicating whether cross-component prediction (CCP) (also called CC prediction) can be used. For example, if the flag information is "1" (true), it means that it can be used, and if the flag information is "0" (false), it means that it cannot be used.

[0485] Note that this CCP is also called Cross-Component Linear Prediction (CCLM or CCLMP).

[0486] <Prediction mode information Pinfo>

[0487] The prediction mode information Pinfo includes, for example, information such as size information PBSize (prediction block size) of a processing target PB (prediction block), intra prediction mode information Ipinfo, motion prediction information MVinfo, and the like.

[0488] The intra prediction mode information IPinfo includes, for example, prev_intra_luma_pred_flag, mpm_idx, rem_intra_pred_mode in JCTVC-W1005, 7.3.8.5 coding unit syntax, and the luma intra prediction mode IntraPredModeY derived from the syntax.

[0489] In addition, the intra-frame prediction mode information IPinfo includes, for example, a cross-component prediction flag (ccp_flag(cclmp_flag)), a multi-class linear prediction mode flag (mclm_flag), a chroma sample location type identifier (chroma_sample_loc_type_idx), a chroma MPM identifier (chroma_mpm_idx), and a luma intra-frame prediction mode (IntraPredModeC) derived from these syntaxes.

[0490] The cross-component prediction flag (ccp_flag (cclmp_flag)) is flag information indicating whether cross-component linear prediction is applied. For example, when ccp_flag==1, it indicates that cross-component prediction is applied; when ccp_flag==0, it indicates that cross-component prediction is not applied.

[0491] The multi-class linear prediction mode flag (mclm_flag) is information about the mode of linear prediction (linear prediction mode information). More specifically, the multi-class linear prediction mode flag (mclm_flag) is flag information indicating whether the multi-class linear prediction mode is set. For example, in the case of "0", it indicates a 1-class mode (single-class mode) (e.g., CCLMP), and in the case of "1", it indicates a 2-class mode (multi-class mode) (e.g., MCLMP).

[0492] The chroma sample position type identifier (chroma_sample_loc_type_idx) is an identifier for identifying the type of the pixel position of the chroma component (also referred to as the chroma sample position type). For example, in the case where the chroma array type (ChromaArrayType) as information related to the color format indicates the 420 format, the chroma sample position type identifier is assigned as shown in the following formula.

[0493] chroma_sample_loc_type_idx==0: Type2

[0494] chroma_sample_loc_type_idx==1:Type3

[0495] chroma_sample_loc_type_idx==2:Type0

[0496] chroma_sample_loc_type_idx==3: Type1

[0497] Note that a color difference sample position type identifier (chroma_sample_loc_type_idx) is transmitted (stored) as information (chroma_sample_loc_info()) on the pixel position of the color difference component.

[0498] The chroma MPM identifier (chroma_mpm_idx) is an identifier indicating which prediction mode candidate in the chroma intra prediction mode candidate list (intraPredModeCandListC) is designated as the chroma intra prediction mode.

[0499] The motion prediction information MVinfo includes, for example, information such as merge_idx, merge_flag, inter_pred_idc, ref_idx_LX, mvp_lX_flag, X={0,1}, mvd (for example, see JCTVC-W1005, 7.3.8.6 Prediction Unit Syntax).

[0500] Of course, the information included in the prediction mode information Pinfo is arbitrary, and may include information other than these information.

[0501] <Transformation information Tinfo>

[0502] The transformation information Tinfo includes, for example, information described below. Of course, the information included in the transformation information Tinfo is arbitrary, and information other than these information may be included.

[0503] The width size TBWSize and the height TBHSize of the processing target transform block (or may be the logarithmic values ​​log2TBWSize and log2TBHSize of the corresponding TBWSize and TBHSize with base 2).

[0504] Transform skip flag (ts_flag): A flag indicating whether to skip (inverse) primary transform and (inverse) secondary transform.

[0505] Scan Identifier (scanIdx)

[0506] Quantization parameter (qp)

[0507] Quantization matrix (scaling_matrix (e.g., JCTVC-W1005, 7.3.4 Scaling List Data Syntax))

[0508] <Residual information Rinfo>

[0509] The residual information Rinfo (for example, see 7.3.8.11 Residual Coding Syntax of JCTVC-W1005) includes, for example, the syntax described below.

[0510] cbf(coded_block_flag): residual data presence / absence flag

[0511] last_sig_coeff_x_pos: the last non-zero coefficient X coordinate

[0512] last_sig_coeff_y_pos: the last non-zero coefficient Y coordinate

[0513] coded_sub_block_flag: Sub-block non-zero coefficient presence / absence flag

[0514] sig_coeff_flag: Non-zero coefficient presence / absence flag

[0515] gr1_flag: A flag indicating whether the non-zero coefficient level is greater than 1 (also called the GR1 flag)

[0516] gr2_flag: A flag indicating whether the non-zero coefficient level is greater than 2 (also called the GR2 flag)

[0517] sign_flag: A code indicating whether a non-zero coefficient is positive or negative (also called a sign code)

[0518] coeff_abs_level_remaining: The remaining level of non-zero coefficients (also called the non-zero coefficient remaining level)

[0519] Of course, the information included in the residual information Rinfo is arbitrary, and may include information other than these information.

[0520] <Filter information Finfo>

[0521] The filter information Finfo includes, for example, control information related to each filtering process described below.

[0522] Control information related to the deblocking filter (DBF)

[0523] Control information related to pixel adaptive offset (SAO)

[0524] Control information related to the adaptive loop filter (ALF)

[0525] Control information related to other linear and nonlinear filters

[0526] More specifically, for example, information for specifying a picture to which each filter is applied and an area in the picture, filter on / off control information in a CU unit, filter on / off control information related to slice and tile boundaries, etc. are included. Of course, the information included in the filter information Finfo is arbitrary, and information other than these information may be included.

[0527] Referring back to the description of the decoding unit 712 , the decoding unit 712 refers to the residual information Rinfo to derive the quantized transform coefficient level for each coefficient position in each transform block. The decoding unit 712 provides the quantized transform coefficient level to the inverse quantization unit 713 .

[0528] In addition, the decoding unit 712 provides the parsed header information Hinfo, prediction mode information Pinfo, quantization transform coefficient level, transform information Tinfo, and filter information Finfo to each block. Specifically, it is as follows.

[0529] The header information Hinfo is provided to the inverse quantization unit 713 , the inverse orthogonal transform unit 714 , the prediction unit 719 , and the loop filter unit 716 .

[0530] The prediction mode information Pinfo is provided to the inverse quantization unit 713 and the prediction unit 719 .

[0531] The transform information Tinfo is provided to the inverse quantization unit 713 and the inverse orthogonal transform unit 714 .

[0532] The filter information Finfo is provided to the loop filter unit 716 .

[0533] Of course, the above example is an example and is not limited to this example. For example, each encoding parameter can be provided to any processing unit. In addition, other information can be provided to any processing unit.

[0534] <Inverse Quantization Unit>

[0535] The inverse quantization unit 713 has at least a configuration necessary to perform processing related to inverse quantization. For example, the inverse quantization unit 713 takes the transform information Tinfo and the quantized transform coefficient level provided from the decoding unit 712 as input, scales (inverse quantizes) the value of the quantized transform coefficient level based on the transform information Tinfo, and derives the transform coefficient Coeff_IQ after inverse quantization.

[0536] Note that this inverse quantization is performed as an inverse process of the quantization performed by the quantization unit 614. Furthermore, this inverse quantization is a process similar to the inverse quantization of the inverse quantization unit 617. That is, the inverse quantization unit 617 performs a process similar to that of the inverse quantization unit 713 (inverse quantization).

[0537] The inverse quantization unit 713 provides the derived transform coefficient Coeff_IQ to the inverse orthogonal transform unit 714 .

[0538] <Inverse Orthogonal Transformation Unit>

[0539] The inverse orthogonal transform unit 714 performs inverse orthogonal transform-related processing. For example, the inverse orthogonal transform unit 714 takes the transform coefficient Coeff_IQ provided from the inverse quantization unit 713 and the transform information Tinfo provided from the decoding unit 712 as input, and performs inverse orthogonal transform processing on the transform coefficient Coeff_IQ based on the transform information Tinfo to derive the prediction residual D'.

[0540] Note that the inverse orthogonal transform is performed as an inverse process of the orthogonal transform performed by the orthogonal transform unit 613. In addition, the inverse orthogonal transform is a process similar to the inverse orthogonal transform performed by the inverse orthogonal transform unit 618. That is, the inverse orthogonal transform unit 618 performs a process similar to that of the inverse orthogonal transform unit 714 (inverse orthogonal transform).

[0541] The inverse orthogonal transform unit 714 provides the derived prediction residual D′ to the operation unit 715 .

[0542] <Calculation Unit>

[0543] The operation unit 715 performs processing related to the addition of information about the image. For example, the operation unit 715 takes the prediction residual D' provided from the inverse orthogonal transform unit 714 and the predicted image P provided from the prediction unit 719 as input. As shown in the following formula, the operation unit 715 adds the prediction residual D' and the predicted image P (prediction signal) corresponding to the prediction residual D' to derive the local decoded image R local .

[0544] R local =D'+P

[0545] The operation unit 715 converts the derived local decoded image R local Provided to the loop filter unit 716 and the frame memory 718.

[0546] <Loop filter unit>

[0547] The loop filter unit 716 performs processing related to the loop filtering process. For example, the loop filter unit 716 uses the local decoded image R provided from the operation unit 715. local And the filter information Finfo supplied from the decoding unit 712 is input. Note that the information input to the loop filter unit 716 is arbitrary, and information other than such information may be input.

[0548] The loop filter unit 716 performs filtering on the local decoded image R based on the filter information Finfo. local The filtering process is performed appropriately.

[0549] For example, the loop filter unit 716 applies four loop filters: a bidirectional filter, a deblocking filter (DBF), an adaptive offset filter (sample adaptive offset (SAO)), and an adaptive loop filter (ALF) in the order described in non-patent document 11. Note that which filter is applied and in what order is arbitrary and can be appropriately selected.

[0550] The loop filter unit 716 performs a filtering process corresponding to the filtering process performed by the encoding side (e.g., the loop filter unit 620 of the image encoding device 600). Of course, the filtering process performed by the loop filter unit 716 is arbitrary and is not limited to the above example. For example, the loop filter unit 716 can apply a Wiener filter or the like.

[0551] The loop filter unit 716 converts the local decoded image R that has been filtered into local Provided to the sorting buffer 717 and the frame memory 718.

[0552] <Sort Buffer>

[0553] The sorting buffer 717 uses the local decoded image R provided from the loop filter unit 716 local As input and hold (store) the local decoded image R local The sorting buffer 717 uses the local decoded image R local The decoded image R is reconstructed for each picture unit, and the reconstructed decoded image R is held (the reconstructed decoded image R is stored in a buffer). The sorting buffer 717 sorts the obtained decoded images R in the decoding order to the reproduction order. The sorting buffer 717 outputs the sorted decoded image R group as moving image data to the outside of the image decoding device 700.

[0554] <Frame Memory>

[0555] The frame memory 718 performs processing related to storing data related to the image. For example, the frame memory 718 uses the local decoded image R provided by the operation unit 715 local As input, a decoded image R is reconstructed for each picture unit and stored in a buffer of the frame memory 718 .

[0556] In addition, the frame memory 718 uses the local decoded image Rlocal that has been subjected to the loop filtering process and is provided from the loop filtering unit 716 as an input, reconstructs the decoded image R for each picture unit, and stores the image in a buffer in the frame memory 718. The frame memory 718 appropriately provides the stored decoded image R (or a portion thereof) to the prediction unit 719 as a reference image.

[0557] Note that the frame memory 718 may store header information Hinfo, prediction mode information Pinfo, transform information Tinfo, filter information Finfo, and the like related to generation of a decoded image.

[0558] <Prediction Unit>

[0559] The prediction unit 719 performs processing related to the generation of a predicted image. For example, the prediction unit 719 takes the prediction mode information Pinfo supplied from the decoding unit 712 as input, performs prediction using the prediction method specified by the prediction mode information Pinfo, and derives a predicted image P. When deriving, the prediction unit 719 uses the decoded image R (or a portion thereof) specified by the prediction mode information Pinfo stored in the frame memory 718 before or after filtering as a reference image. The prediction unit 719 supplies the derived predicted image P to the operation unit 715.

[0560] Note that these processing units (storage buffer 711 to prediction unit 719) have arbitrary configurations. For example, each processing unit may include a logic circuit that implements the above-mentioned processing. In addition, each processing unit may include, for example, a CPU, ROM, RAM, etc., and use them to execute a program to implement the above-mentioned processing. Of course, each processing unit may have two configurations, and part of the above-mentioned processing may be implemented by a logic circuit, and the rest may be implemented by executing a program. The configurations of the various processing units may be independent of each other. For example, some processing units may implement part of the above-mentioned processing by a logic circuit, and some other processing units may execute a program to implement the above-mentioned processing, while other processing units may implement the above-mentioned processing by both a logic circuit and an execution program.

[0561] In the image decoding device 700 having the above configuration, the present technology is applied to the decoding unit 712. That is, the decoding unit 712 has Fig.13 The image decoding device 700 is similar to the configuration of the decoding device 200 shown in FIG. 1 and performs similar processing. That is, the decoding unit 712 executes one of the above-mentioned methods #1 to #8. In this way, the image decoding device 700 can obtain an effect similar to the effect described in any one of the first embodiment to the eighth embodiment above. Therefore, the image decoding device 700 can suppress the reduction of coding efficiency (improve coding efficiency). In other words, the image decoding device 700 can suppress the reduction of the image quality of the decoded image.

[0562] <Flow of Image Decoding Process>

[0563] Next, the flow of each process performed by the image decoding device 700 having the above configuration will be described. Fig.62 The flowchart describes an example of the flow of image encoding processing.

[0564] When the image decoding process is started, in step S701 , the storage buffer 711 acquires encoded data (bit stream) supplied from the outside of the image decoding device 700 , and holds (stores) it.

[0565] In step S702, the decoding unit 712 decodes the encoded data (bit stream) to obtain the quantized transform coefficient level scale. In addition, the decoding unit 712 parses (analyzes and obtains) various encoding parameters from the encoded data (bit stream) through the decoding.

[0566] In step S703 , the inverse quantization unit 713 performs inverse quantization, which is an inverse process of quantization performed on the encoding side, with respect to the quantized transform coefficient level obtained by the process of step S702 , to obtain a transform coefficient Coeff_IQ.

[0567] In step S704 , the inverse orthogonal transform unit 714 performs an inverse orthogonal transform process that is an inverse process of the orthogonal transform process performed on the encoding side with respect to the transform coefficient Coeff_IQ obtained in step S703 , and obtains a prediction residual D′.

[0568] In step S705 , the prediction unit 719 performs prediction processing using the prediction method specified on the encoding side based on the information parsed in step S702 , refers to the reference image stored in the frame memory 718 , and generates a predicted image P.

[0569] In step S706, the operation unit 715 adds the prediction residual D' obtained in step S704 to the predicted image P obtained in step S705 to derive the local decoded image R local .

[0570] In step S707, the loop filter unit 716 processes the local decoded image R obtained by the process of step S706. local Perform loop filtering processing.

[0571] In step S708, the sorting buffer 717 uses the local decoded image R that has been subjected to the filtering process and obtained through the process of step S707. local The decoded images R are derived, and the decoded image groups R are sorted from the decoding order to the reproduction order. The decoded image groups R sorted in the reproduction order are output to the outside of the image decoding device 700 as moving images.

[0572] Furthermore, in step S709, the frame memory 718 stores the local decoded image R obtained by the process of step S706. local or the local decoded image R after filtering obtained by the process of step S707 local at least one of .

[0573] When the process of step S709 ends, the image decoding process ends.

[0574] In the above-described image decoding process, the present technology is applied to the decoding process of step S703. That is, in step S703, the same Fig.15 The decoding process of the similar process. That is, the decoding unit 712 adopts one of the above-mentioned methods #1 to #8 to perform the decoding process. In this way, the image decoding device 700 can obtain an effect similar to the effect described in any one of the first embodiment to the eighth embodiment above. Therefore, the image decoding device 700 can suppress the reduction of coding efficiency (improve coding efficiency). In other words, the image decoding device 700 can suppress the reduction of the image quality of the decoded image.

[0575] <11. Appendix>

[0576] <Computer>

[0577] The above series of processing can be performed by hardware or by software. In the case of performing a series of processing by software, the program constituting the software is installed in a computer. Here, the computer includes a computer installed in dedicated hardware, such as a general-purpose personal computer that can perform various functions by installing various programs.

[0578] Fig.63 : is a block diagram showing a configuration example of hardware of a computer that executes the above-described series of processes by a program.

[0579] exist Fig.63 In the computer 800 shown, a central processing unit (CPU) 801 , a read-only memory (ROM) 802 , and a random access memory (RAM) 803 are interconnected via a bus 804 .

[0580] An input / output interface 810 is also connected to the bus 804. An input unit 811, an output unit 812, a storage unit 813, a communication unit 814, and a drive 815 are connected to the input / output interface 810.

[0581] The input unit 811 includes, for example, a keyboard, a mouse, a microphone, a touch panel, an input terminal, etc. The output unit 812 includes, for example, a display, a speaker, an output terminal, etc. The storage unit 813 includes, for example, a hard disk, a RAM disk, a nonvolatile memory, etc. The communication unit 814 includes, for example, a network interface, etc. The drive 815 drives a removable medium 821 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

[0582] For example, in the computer configured in the above manner, the CPU 801 loads the program stored in the storage unit 813 onto the RAM 803 via the input / output interface 810 and the bus 804, and executes the program, thereby performing the above-mentioned series of operations. Data and the like required for the CPU 801 to perform various processes are also stored in the RAM 803 as appropriate.

[0583] The program to be executed by the computer can be applied by being recorded on the removable medium 821 as a package medium, etc. In this case, the program can be installed on the storage unit 813 via the input / output interface 810 with the removable medium 821 mounted on the drive 815 .

[0584] In addition, the program can also be provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital satellite broadcasting. In this case, the program can be received by the communication unit 814 and installed in the storage unit 813.

[0585] In addition, the program may be preinstalled in the ROM 802 or the storage unit 813 .

[0586] <Information Processing Unit>

[0587] The data units in which the above-mentioned various information are set and the data units targeted for various processing are arbitrary and are not limited to the above-mentioned examples. For example, this information and processing can be set for each transform unit (TU), transform block (TB), prediction unit (PU), prediction block (PB), coding unit (CU), maximum coding unit (LCU), and the sub-block, block, tile, slice, picture, sequence or component or the data in these data units can be the target. Of course, the data unit can be set for each information or processing, and it is not necessary to make the data units of all information or processing consistent. Note that the storage location of this information is arbitrary, and the information can be stored in the header, parameter set, etc. of the above-mentioned data unit. In addition, the information can be stored in multiple locations.

[0588] <Control Information>

[0589] The control information related to the present technology described in each of the above embodiments may be sent from the encoding side to the decoding side. For example, control information (e.g., enabled_flag) that controls whether to allow (or prohibit) the application of the above-mentioned present technology may be sent. In addition, for example, control information (e.g., present_flag) indicating that the target of the above-mentioned present technology is applied (or the target of the above-mentioned present technology is not applied) may be sent. For example, control information that specifies the block size (upper limit and lower limit, or both), frame, component, layer, etc. to which the present technology is to be applied (or application is allowed or prohibited) may be sent.

[0590] <Targets of application of this technology>

[0591] The present technology can be applied to any image coding method. That is, as long as it does not contradict the above-mentioned present technology, the specifications of various processes related to image coding, such as transformation (inverse transformation), quantization (inverse quantization), encoding (decoding), prediction, etc., are arbitrary and are not limited to the above-mentioned examples. In addition, as long as it does not contradict the above-mentioned present technology, part of the processing can be omitted.

[0592] In addition, the present technology can be applied to a multi-view image encoding and decoding system that encodes and decodes multi-view images including images of multiple viewpoints (views). In this case, it is only necessary to apply the present technology to encoding and decoding of each viewpoint (view).

[0593] In addition, the present technology can be applied to a hierarchical image encoding (scalable encoding) and decoding system, which encodes and decodes a layered hierarchical image to have a scalability function for a predetermined parameter. In this case, it is only necessary to apply the present technology to the encoding and decoding of each hierarchy (layer).

[0594] Furthermore, in the above, the encoding device 100 , the decoding device 200 , the image encoding device 600 , and the image decoding device 700 have been described as application examples of the present technology, but the present technology can be applied to any configuration.

[0595] For example, the present technology can be applied to various electronic devices, such as: transmitters or receivers for satellite broadcasting (e.g., television receivers or mobile phones), wired broadcasting such as cable television, distribution on the Internet, and distribution to terminals by cellular communications, or devices that record images on media such as optical disks, magnetic disks, and flash memories and reproduce images from these storage media (e.g., hard disk recorders and video cameras).

[0596] In addition, for example, the present technology can be implemented as a configuration of a part of a device such as a processor of a system large-scale integration (LSI) or the like (e.g., a video processor), a module using multiple processors or the like (e.g., a video module), a unit using multiple modules or the like (e.g., a video unit), or a device that adds other functions to a unit (e.g., a video device).

[0597] In addition, for example, the present technology can also be applied to a network system including a plurality of devices. For example, the present technology can be applied as cloud computing shared and processed together by a plurality of devices via a network. For example, the present technology can be applied to a cloud service that provides services regarding images (moving images) to any terminal such as a computer, an audio-visual (AV) device, a portable information processing terminal, or an Internet of Things (IoT) device.

[0598] Note that in this specification, a system refers to a group of multiple constituent elements (devices, modules (components), etc.), and it is not important whether all the constituent elements are in the same housing. Therefore, multiple devices housed in different housings and connected via a network, and a device in which multiple modules are housed in one housing are both systems.

[0599] <Fields and uses where this technology can be applied>

[0600] Systems, devices, processing units, etc. to which this technology is applied can be used in any field such as transportation, medical care, crime prevention, agriculture, animal husbandry, mining, beauty, factories, home appliances, weather and nature monitoring. In addition, the use is also arbitrary.

[0601] For example, the present technology can be applied to systems and devices provided for providing content for enjoyment, etc. In addition, for example, the present technology can also be applied to systems and devices for traffic such as traffic condition monitoring and automatic driving control. In addition, for example, the present technology can also be applied to systems and devices provided for safety purposes. In addition, for example, the present technology can be applied to systems and devices provided for automatic control of machines, etc. In addition, for example, the present technology can also be applied to systems and devices provided for agriculture or animal husbandry. In addition, the present technology can also be applied to systems and devices for monitoring natural conditions such as volcanoes, forests, and oceans, wild animals, etc. In addition, for example, the present technology can also be applied to systems and devices provided for sports purposes.

[0602] <Others>

[0603] Note that in this specification, a "flag" is information for identifying multiple states, and includes not only information for identifying two states, true (1) and false (0), but also information capable of identifying three or more states. Therefore, the value that a "flag" can take can be, for example, a binary value 1 / 0 or can be a ternary value or more. That is, the number of bits constituting a "flag" is arbitrary, and can be 1 bit or more. In addition, identification information (including a flag) is assumed to include not only identification information in a bit stream, but also difference information between the identification information in the bit stream and specific reference information. Therefore, in this specification, "flag" and "identification information" include not only the information itself, but also difference information relative to the reference information.

[0604] In addition, various types of information (metadata, etc.) about the coded data (bitstream) can be sent or recorded in any form, as long as the various types of information are associated with the coded data. Here, the term "association" means, for example, making another data available (linkable) when processing one data. That is, data associated with each other can be combined into one data or can be separate data. For example, information associated with the coded data (image) can be sent on a transmission path different from the transmission path of the coded data (image). In addition, for example, information associated with the coded data (image) can be recorded on a recording medium different from the recording medium of the coded data (image) (or recorded in another recording area of ​​the same recording medium). Note that the "association" can be a part of the data, not all of the data. For example, an image and information corresponding to the image can be associated with each other in arbitrary units such as multiple frames, one frame, or a part of a frame.

[0605] Note that in this specification, terms such as "synthesis", "multiplexing", "addition", "integration", "inclusion", "storage", "submission", "connection" and "insertion" refer to putting multiple things into one, such as putting encoded data and metadata into one data, and refer to a method of the above-mentioned "association".

[0606] Furthermore, the embodiments of the present technology are not limited to the aforementioned embodiments, but various changes can be made within the scope not departing from the gist of the present technology.

[0607] For example, a configuration described as one device (or processing unit) may be divided and configured into a plurality of devices (or processing units). Conversely, a configuration described above as a plurality of devices (or processing units) may be configured together as one device (or processing unit). In addition, a configuration other than the above configuration may be added to the configuration of each device (or each processing unit). In addition, when the configuration and operation of the entire system are substantially the same, a portion of the configuration of a certain device (or processing unit) may be included in the configuration of another device (or another processing unit).

[0608] In addition, for example, the above-mentioned program can be executed in an arbitrary device. In that case, it is sufficient as long as the device has necessary functions (functional blocks, etc.) so that necessary information can be obtained.

[0609] In addition, for example, each step in a flowchart may be performed by one device, or may be shared and performed by multiple devices. In addition, in the case where a step includes multiple processes, the multiple processes may be performed by one device, or may be shared and performed by multiple devices. In other words, the multiple processes included in one step may be performed as processes of multiple steps. Conversely, the processes described as multiple steps may be performed together as one step.

[0610] In addition, for example, with respect to a program executed by a computer, the processing of the steps describing the program may be performed in chronological order according to the order described in this specification, or may be performed separately in parallel or at necessary timings such as when a call is made. That is, the processing of each step may be performed in an order different from the above order as long as no contradiction occurs. In addition, the processing of the steps describing the program may be performed in parallel with the processing of another program, or may be performed in combination with the processing of another program.

[0611] In addition, for example, multiple technologies related to the present technology can be independently performed as long as there is no contradiction. Of course, any number of the present technologies can be used in combination. For example, part or all of the present technology described in any embodiment can be performed in combination with part or all of the present technology described in another embodiment. In addition, another technology not described above can also be combined to perform part or all of any of the above-mentioned present technologies.

[0612] Note that the present technology can be configured as follows.

[0613] (1) An image processing device comprising:

[0614] a syntax element value deriving unit that sets an upper limit value of the number of bits allocated to a processing target subblock by distributing the number of bits among non-zero subblocks, and derives a syntax element value regarding the processing target subblock by using coefficient data derived from the image data so that the number of bits does not exceed the upper limit value; and

[0615] An encoding unit encodes the derived syntax element value and generates encoded data.

[0616] (2) The image processing device according to (1), wherein:

[0617] The syntax element value deriving unit sets an upper limit value of the number of bits allocated to the processing target subblock based on the extra bit number of the immediately preceding non-zero subblock and a temporary upper limit value of the number of bits allocated to the processing target subblock.

[0618] (3) The image processing device according to (1) or (2), wherein

[0619] The syntax element value deriving unit sets an upper limit value of the number of bits allocated to the processing target sub-block by distributing the number of bits among non-zero sub-blocks in a CU.

[0620] (4) The image processing device according to any one of (1) to (3), wherein:

[0621] The syntax element value deriving unit derives a syntax element value regarding the processing target subblock so that a sum of the numbers of bits allocated to all non-zero subblocks in a block does not exceed a sum of temporary upper limit values ​​of the numbers of bits allocated to each non-zero subblock in the block.

[0622] (5) The image processing device according to (4), wherein:

[0623] The syntax element value derivation unit also allocates an extra number of bits for zero sub-blocks in the block.

[0624] (6) The image processing device according to any one of (1) to (5), wherein

[0625] The syntax element value deriving unit sets an upper limit value of the number of bits allocated to a block, and derives a syntax element value regarding the block by using coefficient data derived from the image data so that the number of bits does not exceed the upper limit value.

[0626] (7) An image processing method, comprising:

[0627] setting an upper limit value of the number of bits allocated to a processing target subblock by distributing the number of bits among non-zero subblocks, and deriving a syntax element value with respect to the processing target subblock by using coefficient data derived from image data so that the number of bits does not exceed the upper limit value; and

[0628] The derived syntax element value is encoded and encoded data is generated.

[0629] (8) An image processing device comprising:

[0630] a decoding unit that decodes the encoded data and generates syntax element values; and

[0631] A coefficient data exporting unit that sets an upper limit value for the number of bits allocated to a processing target subblock by distributing the number of bits among non-zero subblocks, and exports coefficient data corresponding to image data about the processing target subblock by using the generated syntax element value so that the number of bits does not exceed the upper limit value.

[0632] (9) An image processing method, comprising:

[0633] Decoding the encoded data and generating syntax element values; and

[0634] An upper limit value of the number of bits allocated to a processing target subblock is set by distributing the number of bits among non-zero subblocks, and coefficient data corresponding to image data regarding the processing target subblock is derived by using the generated syntax element value so that the number of bits does not exceed the upper limit value.

[0635] (10) An image processing device comprising:

[0636] a syntax element value deriving unit that sets an upper limit value of the number of bits allocated to a processing target subblock by allocating the number of bits allocated to a zero subblock to a non-zero subblock, and derives a syntax element value with respect to the processing target subblock by using coefficient data derived from the image data so that the number of bits does not exceed the upper limit value; and

[0637] An encoding unit encodes the derived syntax element value and generates encoded data.

[0638] (11) The image processing device according to (10), wherein:

[0639] The syntax element value deriving unit sets an upper limit value of the number of bits allocated to the processing target subblock based on an average extra bit number of zero subblocks and a temporary upper limit value of the number of bits allocated to the processing target subblock.

[0640] (12) The image processing device according to (11), wherein:

[0641] The syntax element value deriving unit further sets the upper limit value of the number of bits allocated to the processing target sub-block according to a priority based on a position of the processing target sub-block.

[0642] (13) An image processing method, comprising:

[0643] setting an upper limit value of the number of bits allocated to a processing target subblock by allocating the number of bits allocated to a zero subblock to a non-zero subblock, and deriving a syntax element value with respect to the processing target subblock by using coefficient data derived from image data so that the number of bits does not exceed the upper limit value; and

[0644] The derived syntax element value is encoded and encoded data is generated.

[0645] (14) An image processing device comprising:

[0646] a decoding unit that decodes the encoded data and generates syntax element values; and

[0647] A coefficient data exporting unit that sets an upper limit value for the number of bits allocated to a processing target subblock by allocating the number of bits allocated to a zero subblock to a non-zero subblock, and exports coefficient data corresponding to image data about the processing target subblock by using the generated syntax element value so that the number of bits does not exceed the upper limit value.

[0648] (15) An image processing method, comprising:

[0649] Decoding the encoded data and generating syntax element values; and

[0650] An upper limit value of the number of bits allocated to a processing target subblock is set by allocating the number of bits allocated to a zero subblock to a non-zero subblock, and coefficient data corresponding to image data regarding the processing target subblock is derived by using the generated syntax element value so that the number of bits does not exceed the upper limit value.

[0651] Reference numerals list

[0652] 100 Encoding device

[0653] 101 Sequencer

[0654] 102 CABAC

[0655] 121 Binarization unit

[0656] 122 Select Unit

[0657] 123 Context Model

[0658] 124 Arithmetic Coding Unit

[0659] 125 Arithmetic Coding Unit

[0660] 126 Select Unit

[0661] 200 Decoding device

[0662] 201 CABAC

[0663] 202 Parser

[0664] 221 Select Unit

[0665] 222 Context Model

[0666] 223 Arithmetic decoding unit

[0667] 224 Arithmetic decoding unit

[0668] 225 Select Unit

[0669] 226 Multi-value conversion unit

[0670] 600 Image encoding device

[0671] 615 Coding Unit

[0672] 700 Image decoding device

[0673] 712 Decoding Unit< / cabac> < / cabac>

Claims

1. An image processing device, comprising: a syntax element value deriving unit that sets an upper limit value of the number of bits allocated to a processing target subblock by distributing the number of bits among non-zero subblocks and derives a syntax element value regarding the processing target subblock by using coefficient data derived from the image data so that the number of bits does not exceed the upper limit value; as well as An encoding unit encodes the derived syntax element value and generates encoded data.

2. The image processing device according to claim 1, wherein: The syntax element value deriving unit sets an upper limit value of the number of bits allocated to the processing target subblock based on the extra bit number of the immediately preceding non-zero subblock and a temporary upper limit value of the number of bits allocated to the processing target subblock.

3. The image processing device according to claim 1, wherein: The syntax element value deriving unit sets an upper limit value of the number of bits allocated to the processing target subblock by distributing the number of bits among non-zero subblocks in a coding unit.

4. The image processing device according to claim 1, wherein: The syntax element value deriving unit derives a syntax element value regarding the processing target subblock so that a sum of the numbers of bits allocated to all non-zero subblocks in a block does not exceed a sum of temporary upper limit values ​​of the numbers of bits allocated to each non-zero subblock in the block.

5. The image processing device according to claim 4, wherein: The syntax element value derivation unit also allocates an extra number of bits for zero sub-blocks in the block.

6. The image processing device according to claim 1, wherein: The syntax element value deriving unit sets an upper limit value of the number of bits allocated to a block, and derives a syntax element value regarding the block by using coefficient data derived from the image data so that the number of bits does not exceed the upper limit value.

7. An image processing method, comprising: setting an upper limit value of the number of bits allocated to a processing target subblock by distributing the number of bits among non-zero subblocks, and deriving a syntax element value with respect to the processing target subblock by using coefficient data derived from image data so that the number of bits does not exceed the upper limit value; as well as The derived syntax element value is encoded and encoded data is generated.

8. An image processing device, comprising: a decoding unit that decodes the encoded data and generates syntax element values; as well as A coefficient data exporting unit that sets an upper limit value of the number of bits allocated to a processing target subblock by distributing the number of bits among non-zero subblocks, and exports coefficient data corresponding to image data about the processing target subblock by using the generated syntax element value so that the number of bits does not exceed the upper limit value.

9. An image processing method, comprising: Decoding the encoded data and generating syntax element values; as well as An upper limit value of the number of bits allocated to a processing target subblock is set by distributing the number of bits among non-zero subblocks, and coefficient data corresponding to image data regarding the processing target subblock is derived by using the generated syntax element value so that the number of bits does not exceed the upper limit value.

10. An image processing device, comprising: a syntax element value deriving unit that sets an upper limit value of the number of bits allocated to a processing target subblock by allocating the number of bits allocated to a zero subblock to a non-zero subblock, and derives a syntax element value with respect to the processing target subblock by using coefficient data derived from the image data so that the number of bits does not exceed the upper limit value; as well as An encoding unit encodes the derived syntax element value and generates encoded data.

11. The image processing apparatus according to claim 10, wherein: The syntax element value deriving unit sets the upper limit value of the number of bits allocated to the processing target subblock based on an average extra bit number of zero subblocks and a temporary upper limit value of the number of bits allocated to the processing target subblock.

12. The image processing apparatus according to claim 11, wherein The syntax element value deriving unit further sets the upper limit value of the number of bits allocated to the processing target sub-block according to a priority based on a position of the processing target sub-block.

13. An image processing method, comprising: setting an upper limit value of the number of bits allocated to a processing target subblock by allocating the number of bits allocated to a zero subblock to a non-zero subblock, and deriving a syntax element value with respect to the processing target subblock by using coefficient data derived from image data so that the number of bits does not exceed the upper limit value; as well as The derived syntax element value is encoded and encoded data is generated.

14. An image processing device, comprising: a decoding unit that decodes the encoded data and generates syntax element values; as well as A coefficient data exporting unit that sets an upper limit value for the number of bits allocated to a processing target subblock by allocating the number of bits allocated to a zero subblock to a non-zero subblock, and exports coefficient data corresponding to image data about the processing target subblock by using the generated syntax element value so that the number of bits does not exceed the upper limit value.

15. An image processing method, comprising: Decoding the encoded data and generating syntax element values; as well as An upper limit value of the number of bits allocated to a processing target subblock is set by allocating the number of bits allocated to a zero subblock to a non-zero subblock, and coefficient data corresponding to image data regarding the processing target subblock is derived by using the generated syntax element value so that the number of bits does not exceed the upper limit value.

Images