Image processing apparatus and method

Abstract

This disclosure relates to an image processing apparatus and method capable of suppressing the increase in the load of encoding / decoding processing. A fixed context variable is set for the first binary number of the binary sequence of control information related to secondary conversion, and each binary number of the secondary conversion control information is arithmetically encoded with reference to the set fixed context variable. This disclosure is applicable to, for example, image processing apparatuses, image encoding apparatuses, image decoding apparatuses, transmission apparatuses, receiving apparatuses, transmission and reception apparatuses, information processing apparatuses, imaging apparatuses, playback apparatuses, electronic machines, image processing methods, or information processing methods.

Description

Technical Field

[0001] This disclosure relates to image processing apparatus and methods, and in particular to image processing apparatus and methods that can suppress the increase in the load of encoding / decoding processing. Prior Technology

[0002] Previously, a coding method has been proposed that derives the prediction residuals of moving images, performs coefficient transformation, quantizes, and then encodes them (e.g., Non-Patent Document 1). In the VVC WD6 described in Non-Patent Document 1, a coding tool that performs a low-frequency secondary transformation (LFST) on the transformation coefficients after the primary transformation and also improves energy compaction already exists. As pattern information regarding this low-frequency secondary transformation, there is an ST identifier st_idx (also called lfnst_idx).

[0003] When encoding the ST identifier lfnst_idx, firstly, the ST identifier lfnst_idx is binarized, and then arithmetic encoding is performed with reference to the context variable ctx corresponding to each binIdx of the obtained binary sequence (bins). For example, for the binary number where binIdx=0, the context variable ctx is assigned according to the tree type treType and the MTS identifier mts_idx. [Previous Technical Documents] [Non-patent literature]

[0004] [Non-Patent Document 1] Benjamin Bross, Jianle Chen, Shan Liu, "Versatile Video Coding (Draft 6)", JVET-O2001-vE, m49908, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11 15th Meeting: Gothenburg, SE, 3-12 July 2019 Summary of the Invention

[0005] [The problem that the invention aims to solve]

[0006] However, since LFSNT is only applicable when tu_mts_idx == 0 (DCT2xDCT2), the condition "tu_mts_idx == 0" is always true. Therefore, the divergence of "tu_mts_idx == 0" is redundant, which may lead to an increase in the load of encoding / decoding processing.

[0007] This disclosure was developed in view of this situation, with the aim of suppressing the increase in the load of encoding / decoding processing. [Methods used to solve problems]

[0008] One aspect of this technology is an image processing device, which includes an encoding unit that sets a fixed context variable for the first binary number of the binary sequence of the control information for secondary conversion, that is, the control information for secondary conversion, and performs arithmetic encoding on each binary number of the preceding secondary conversion control information by referring to the fixed context variable.

[0009] One aspect of this technology is an image processing method that sets a fixed context variable for the first binary number of the binary sequence of the secondary transformation control information, which is the control information of the secondary transformation, and performs arithmetic encoding on each binary number of the preceding secondary transformation control information by referring to the fixed context variable.

[0010] Another aspect of this technology is an image processing device that includes a decoding unit that performs arithmetic decoding with reference to a fixed context variable set for the first binary number of the binary sequence of control information regarding secondary transformation, i.e., secondary transformation control information.

[0011] Another aspect of this technology is an image processing method that performs arithmetic decoding by referring to a fixed context variable set for the first binary number of the binary sequence of the control information regarding the secondary transformation, i.e., the control information of the secondary transformation, which is the control information of the secondary transformation.

[0012] In one aspect of the image processing apparatus and method of this technology, a fixed context variable is set for the first binary number of the binary sequence of the control information for secondary conversion, that is, the control information for secondary conversion. The fixed context variable is referenced, and each binary number of the secondary conversion control information is arithmetically encoded.

[0013] In another aspect of this technology, in an image processing apparatus and method, a fixed context variable is set for the first binary number of the binary sequence of the control information regarding the secondary transformation, that is, the secondary transformation control information, and arithmetic decoding is performed by reference. Simple Explanation of the Diagram

[0014] [Figure 1] Explanation of ST identifier. [Figure 2] An illustration of an example of the allocation method of context variables ctx corresponding to each binIdx of the binary sequence of ST identifiers. [Figure 3] Block diagram of the main components of an encoding device. [Figure 4] Flowchart of an example of the encoding process. [Figure 5] Block diagram of the main components of a decoding device. [Figure 6] Flowchart of an example of the decoding process. [Figure 7] Block diagram of the main components of an image encoding device. [Figure 8] A flowchart of an example of the image encoding process. [Figure 9] Block diagram of the main components of an image decoding device. [Figure 10] A flowchart of an example of image decoding processing. [Figure 11] Block diagram of the main components of a computer. Implementation

[0015] The following describes the form in which this disclosure is implemented (hereinafter referred to as the implementation form). Furthermore, the description is presented in the following order. 1. ST identification element encoding 2. First Embodiment (Encoding Device) 3. Second Embodiment (Decoding Device) 4. Third embodiment (image decoding device) 5. Fourth Embodiment (Image Encoding Device) 6. Postscript

[0016] <1. ST Identifier Encoding> Supporting documents containing technical content and terminology. The scope of this technology includes not only the content described in the embodiments, but also the content described in the following non-patent documents that were known at the time of application, or the content of other documents referenced in the following non-patent documents.

[0017] Non-patent literature 1: (as mentioned above) Non-patent literature 2: Jianle Chen, Yan Ye, Seung Hwan Kim, "Algorithm description for Versatile Video Coding and Test Model 6 (VTM 6)", JVET-O2002-v2, m49914, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11 15th Meeting:Gothenburg, SE, 3-12 July 2019 Non-patent document 3: Recommendation ITU-T H.264 (04 / 2017) "Advanced video coding for generic audiovisual services", April 2017 Non-patent document 4: Recommendation ITU-T H.265 (02 / 18) "High efficiency video coding", February 2018

[0018] In other words, the content described in the aforementioned non-patent literature is also a basis for determining the supporting requirements. For example, the Quad-Tree Block Structure and QTBT (Quad Tree Plus Binary Tree) Block Structure described in the aforementioned non-patent literature, even if not directly described in the embodiments, still fall within the scope of this technology and are considered to meet the supporting requirements of the patent application. Similarly, for example, technical terms such as parsing, syntax, and semantics, even if not directly described in the embodiments, still fall within the scope of this technology and are considered to meet the supporting requirements of the patent application.

[0019] Also, in this specification, a "block" (not the block representing a processing unit) used to describe a partial area or a processing unit of an image (picture) represents an arbitrary partial area within the image, without special mention, and its size, shape, characteristics, etc. are not limited. For example, the "block" may include: TB (Transform Block), TU (Transform Unit), PB (Prediction Block), PU (Prediction Unit), SCU (Smallest Coding Unit), CU (Coding Unit), LCU (Largest Coding Unit), CTB (Coding Tree Block), CTU (Coding Tree Unit), sub-block, macro-block, tile, or slice, etc., in the non-patent literature described above, any partial area (processing unit).

[0020] Also, when specifying the size of such a block, not only can the block size be directly specified, but the block size can also be indirectly specified. For example, identification information used to identify the size can also be used to specify the block size. Also, for example, the block size can be specified by the ratio or difference from the size of a reference block (such as LCU or SCU, etc.). For example, when transmitting information used to specify the block size as a syntax element, etc., as this information, the indirectly specified size information as described above can also be used. By designing like this, the amount of information of this information can be reduced, and sometimes the coding efficiency can be improved. Also, the specification of the block size includes the specification of the range of the block size (such as the specification of the allowable range of the block size, etc.).

[0021] Also, in this specification, coding not only includes the overall process of converting an image into a bitstream, but also includes partial processes. For example, it not only includes processes that include prediction processing, orthogonal transformation, quantization, arithmetic coding, etc., but also includes processes that collectively refer to quantization and arithmetic coding, processes that include prediction processing, quantization, and arithmetic coding, etc. Similarly, decoding not only includes the overall process of converting a bitstream into an image, but also includes partial processes. For example, it not only includes processes that include inverse arithmetic decoding, inverse quantization, inverse orthogonal transformation, prediction processing, etc., but also includes processes that include inverse arithmetic decoding and inverse quantization, processes that include inverse arithmetic decoding, inverse quantization, and prediction processing, etc.

[0022] <ST identification element> In the VVC WD6 described in Non-Patent Document 1, an encoding tool exists that performs a low-frequency secondary transformation (LFST) on the transformation coefficients after the primary transformation and also improves energy compaction. As pattern information (control information regarding the secondary transformation) concerning this low-frequency secondary transformation, there is an ST identifier st_idx (also called lfnst_idx) indicating the type of the secondary transformation. The encoding of this ST identifier is performed according to the following procedure.

[0023] First, the ST identifier lfnst_idx is binarized to obtain the binary sequence bins. The ST identifier lfnst_idx takes the values ​​shown in Table A of Figure 1, where each value represents the transformation type shown in the table. Furthermore, by binarizing the ST identifier lfnst_idx in this way, the binary sequence (binarization) shown in the table can be obtained.

[0024] Furthermore, the binarization of the ST identifier lfnst_idx is achieved by binarizing the truncated unary code (TU) to obtain the binary sequence bins. Moreover, the TU code, as shown in table B of Figure 1, is equivalent to the truncated Rice code (TR) with the Rice parameter cRiceParam set to 0.

[0025] Next, arithmetic encoding is performed by referring to the context variable ctx corresponding to each binIdx of the obtained binary sequence bins. The index used to identify the context variable ctx is also called ctxInc (or ctxIdx).

[0026] Specifically, for binary numbers where binIdx=0, as shown in table C of Figure 1, the context variable ctx is assigned according to the tree type treType and the MTS identifier mts_idx. In the example of this table, ctxInc corresponding to binIdx=0 is derived with the value ctxInc=(mts_idx==0 && treeType !=SINGLE_TREE). Furthermore, for binary numbers bins where binIdx=1, bypass encoding is applied.

[0027] Furthermore, the bins of the binary sequence bins of the ST identifier lfnst_idx can also be interpreted as flags corresponding to the transformation type. In this case, the value of the binary number binIdx=0 corresponds to the flag indicating whether the secondary transformation is possible (0: Yes, 1: No), and the value of the binary number binIdx=1 corresponds to the flag indicating whether the first secondary transformation is possible (0: Yes, 1: No).

[0028] During decoding, the binary numbers are arithmetically decoded with reference to the context variable ctx set above. Then, the binary sequence is multivalued to obtain the ST identifier lfnst_idx.

[0029] However, since LFSNT is only applicable when tu_mts_idx == 0 (DCT2xDCT2), the condition "tu_mts_idx == 0" is always true in the table shown in C of Figure 1. Therefore, the divergence of "tu_mts_idx == 0" is redundant, which may lead to an increase in the load of encoding / decoding processing. In order to reduce implementation costs, it is required to maintain encoding efficiency while reducing context variables or simplifying the derivation of context variables.

[0030] <Fixed Context Allocation> Therefore, a fixed context variable is set for the first binary number of the binary sequence of the control information regarding the secondary transformation, which is the control information of the secondary transformation control information.

[0031] For example, in image processing (especially image encoding), a fixed context variable is set for the first binary number of the binary sequence of the control information for secondary transformation, that is, the control information for secondary transformation, and each binary number of the secondary transformation control information is arithmetically encoded by referring to the fixed context variable.

[0032] For example, in an image processing apparatus (especially an image encoding apparatus that encodes images), an encoding unit is provided that sets a fixed context variable for the first binary number of the binary sequence of the control information for secondary conversion, that is, the secondary conversion control information, and performs arithmetic encoding on each binary number of the secondary conversion control information by referring to the fixed context variable.

[0033] This design allows for maintaining coding efficiency while simplifying the derivation of context variables for ST identifiers. Furthermore, it maintains coding efficiency while reducing the memory size used to hold the context variables of ST identifiers. In other words, it suppresses the increase in coding processing load.

[0034] For example, in image processing (decoding of encoded image data), arithmetic decoding is performed with reference to a fixed context variable set for the first binary number of the binary sequence of the control information for the secondary transformation, which is the control information for the secondary transformation.

[0035] For example, an image processing device (especially an image decoding device that decodes encoded data of an image) is equipped with a decoding unit that performs arithmetic decoding by referring to a fixed context variable set for the first binary number of the binary sequence of control information for secondary transformation, that is, secondary transformation control information.

[0036] This design allows for maintaining coding efficiency while simplifying the derivation of context variables for ST identifiers. Furthermore, it maintains coding efficiency while reducing the memory size used to hold the context variables of ST identifiers. In other words, it suppresses the increase in decoding processing load.

[0037] For example, as shown in the second row from the top of the table in Figure 2A (Method 1), the binary number (the first binary number) with binIdx=0 in the binary sequence can be assigned "0" as a fixed context variable ctx. That is, the first binary number of the ST identifier's binary sequence can also be set to "0" as the context variable ctx, and arithmetic encoding can be performed with reference to this value. In other words, arithmetic decoding can also be performed with reference to the "0" set as the first binary number of the ST identifier's binary sequence as the context variable ctx. With this design, the divergence processing related to the selection of context variables caused by treeType (and tu_mts_idx==0) can be omitted in the derivation of the context variable in the first binary number of the ST identifier's binary sequence. This can suppress the increase in the encoding / decoding processing load. Furthermore, as shown in the table in Figure 2B, Method 1 can further reduce the number of context variables compared to the VTM case. Therefore, the memory size used to hold the context variables of the ST identifier can be reduced.

[0038] Alternatively, as shown in Table A of Figure 2 (Method 1), a bypass flag can be assigned to the binary number (the second binary number) where binIdx=1 in the binary sequence. That is, a bypass flag can also be set for the first binary number of the ST identifier's binary sequence, and arithmetic encoding can be performed based on this bypass flag. In other words, arithmetic decoding can also be performed based on the bypass flag set for the first binary number of the ST identifier's binary sequence.

[0039] Furthermore, for example, as shown in the third row from the top of the table in Figure 2A (Method 2), a common context variable ctx can be assigned to the binary numbers (first binary number) with binIdx=0 and the binary numbers (second binary number) with binIdx=1 in the binary sequence. In the case of the table shown in Figure 2A, both the binary numbers (first binary number) with binIdx=0 and the binary numbers (second binary number) with binIdx=1 in the binary sequence are assigned "0". That is, a common context variable ctx (e.g., "0") can be set for the first and second binary numbers of the ST identifier's binary sequence, and arithmetic encoding of each binary number can be performed with reference to this value. In other words, arithmetic decoding of each binary number can be performed with reference to the common context variable ctx (e.g., "0") set for the first and second binary numbers of the ST identifier's binary sequence. With this design, as shown in Table B of Figure 2, Method 2 can maintain the number of contexts while reducing the number of bypass coding bins, compared to Method 1. Therefore, the coding amount of ST identifiers can be reduced.

[0040] <2. First Implementation Form> <Encoding device> The technique described in <1. Encoding of ST identifier> can be applied to any device. Examples of its application are described below. Figure 3 is a block diagram illustrating an example of the configuration of an encoding device for an image processing apparatus to which this technique is applied. The encoding device 100 shown in Figure 3 is a device for encoding the ST identifier lfnst_idx. The encoding device 100 performs encoding, for example, using CABAC (Context-based Adaptive Binary Arithmetic Code).

[0041] Furthermore, Figure 3 mainly shows the flow of processing units or data, and what is shown in Figure 3 is not all. That is, in the encoding device 100, there may also be processing units that are not shown as blocks in Figure 3, and there may also be processing or data flow that is not indicated by arrows in Figure 3.

[0042] As shown in Figure 3, the encoding device 100 includes: a binarization unit 121, a selection unit 122, a context model 123, an arithmetic encoding unit 124, an arithmetic encoding unit 125, and a selection unit 126.

[0043] The binarization unit 121 acquires the syntax element values ​​supplied to the encoding device 100, binarizes them using the method defined for each syntax element, and generates a binarized bit array. The binarization unit 121 then supplies the binarized bit array to the selection unit 122.

[0044] The selection unit 122 obtains the binary bit array and the flag information isBypass supplied by the binary encoding unit 121. The selection unit 122 selects the target for supplying the binary bit array based on the value of isBypass. For example, if isBypass=0, the selection unit 122 determines it is in normal mode and supplies the binary bit array to the context model 123. If isBypass=1, the selection unit 122 determines it is in bypass mode and supplies the binary bit array to the arithmetic encoding unit 125.

[0045] Context model 123 dynamically switches the applicable context model according to the encoded object and its surrounding conditions. For example, context model 123 holds the context variable ctx, and once the binary bit column is obtained from the selection unit 122, it reads out the context variable ctx corresponding to each binary position (binIdx) of the bin column defined for each syntax element. Context model 123 then supplies the binary bit column and the read context variable ctx to the arithmetic encoding unit 124.

[0046] The arithmetic coding unit 124, upon receiving the binary bit array and context variable ctx supplied from the context model 123, performs arithmetic coding (context coding) on ​​the binary value of the binary bit array located at binIdx using the normal mode of CABAC, referring to the probability state of the context variable ctx. The arithmetic coding unit 124 then supplies the encoded data generated by this context coding to the selection unit 126. Furthermore, the arithmetic coding unit 124 supplies and holds the context variable ctx after context coding processing to the context model 123.

[0047] The arithmetic coding unit 125 performs arithmetic coding (bypass coding) on ​​the binary bit array supplied from the selection unit 122 using the CABAC bypass mode. The arithmetic coding unit 125 then supplies the coded data generated by the bypass coding to the selection unit 126.

[0048] The selection unit 126 acquires the flag information isBypass and selects the encoded data to be output based on the value of isBypass. For example, when isBypass=0, the selection unit 126 determines that it is in normal mode, acquires the encoded data supplied by the arithmetic encoding unit 124, and outputs it to the outside of CABAC102 (encoding device 100). When isBypass=1, the selection unit 126 determines that it is in bypass mode, acquires the encoded data supplied by the arithmetic encoding unit 125, and outputs it to the outside of the encoding device 100.

[0049] Furthermore, these processing units (binarization unit 121 and even selection unit 126) can have arbitrary configurations. For example, each processing unit can be configured by implementing the above-mentioned processing through logic circuits. Alternatively, each processing unit can have, for example, a CPU (Central Processing Unit), ROM (Read Only Memory), RAM (Random Access Memory), etc., and use them to execute programs to implement the above-mentioned processing. Of course, each processing unit can also have both of these configurations, where part of the above-mentioned processing is implemented by logic circuits, and the rest is implemented by executing programs. The configurations of each processing unit can also be independent of each other. For example, one processing unit can implement a part of the above-mentioned processing through logic circuits, another processing unit can implement the above-mentioned processing by executing programs, and yet another processing unit can implement the above-mentioned processing through both logic circuits and program execution.

[0050] In this encoding device 100, the technique described in <1. Encoding of ST identifier> is applied. That is, the binarization unit 121 binarizes the secondary conversion control information (ST identifier lfnst_idx) to generate a binary sequence. The selection unit 122 sets a fixed context variable ctx (e.g., "0") for the first binary number of the binary sequence. The arithmetic encoding unit 124 performs arithmetic encoding by referring to the fixed context variable ctx (e.g., "0") using the normal CABAC pattern.

[0051] By designing in this way, the encoding device 100 can maintain encoding efficiency while simplifying the derivation of the context variables of the ST identifier. Furthermore, the encoding device 100 can maintain encoding efficiency while reducing the memory size used to hold the context variables of the ST identifier. In other words, the encoding device 100 can suppress the increase in the encoding processing load.

[0052] At this time, the selection unit 122 can also set a bypass flag (method 1) for the second binary number of the binary sequence. In this case, the arithmetic encoding unit 125 performs arithmetic encoding on the second binary number using the CABAC bypass mode.

[0053] Furthermore, the selection unit 122 can also set a context variable ctx (e.g., "0") that is common to the first binary number for the second binary number of the binary sequence (method 2). In this case, the arithmetic encoding unit 124 performs arithmetic encoding by referring to the fixed context variable ctx (e.g., "0") using the normal pattern of CABAC.

[0054] <Encoding Process> Next, an example of the image encoding process performed by the encoding device 100 will be described with reference to the flowchart in FIG4.

[0055] Once the encoding process is started, the binarization unit 121 of the encoding device 100 inputs the syntax element value (syncVal) of the object to be processed in step S101.

[0056] In step S102, the binarization unit 121 performs binarization processing on each syntax element as defined, and derives a binary sequence (synBins) of the syntax element value (syncVal).

[0057] In step S103, the selection unit 122 reads out the context variable ctx corresponding to each binary position (binIdx) of the bin column defined for each syntax element, and the flag isBypass indicating whether it is the bypass mode.

[0058] In step S104, the selection unit 122 determines whether it is in bypass mode. If isBypass=0 and is determined to be in normal mode, the processing system proceeds to step S105.

[0059] In step S105, the arithmetic encoding unit 124 performs context encoding. That is, the arithmetic encoding unit 124 encodes the binary values ​​of the bin column (synBins) at the binary positions (binIdx) using the normal CABAC pattern, referring to the probability state of the context variable ctx. Once the processing in step S105 is completed, the process proceeds to step S107.

[0060] Furthermore, in step S104, isBypass=1, and if it is determined to be in bypass mode, the process proceeds to step S106.

[0061] In step S106, the arithmetic encoding unit 125 performs bypass encoding. That is, the arithmetic encoding unit 125 encodes the binary value of the bin column (synBins) located at the binary position (binIdx) by using the CABAC bypass mode. Once the processing of step S106 is completed, the process proceeds to step S107.

[0062] In step S107, the selection unit 126 determines whether the predetermined rest condition A is met. The rest condition A is defined based on the values ​​of the binary sequence from binIdx=0 to binIdx=k (the current position k of binIdx) or the binarization method of each syntax element.

[0063] If the rest condition A is not met, the processing system returns to step S103 and performs subsequent processing on the next binary position (binIdx). That is, each binary position (binIdx) will be processed through steps S103 and up to S107.

[0064] Then, in step S107, if the rest condition A is determined to be met, the CABAC process ends.

[0065] In this encoding process, the technique described in <1. Encoding of ST identifier> is applied. That is, the binarization unit 121 binarizes the secondary conversion control information (ST identifier lfnst_idx) in step S102 to generate a binary sequence. The selection unit 122 sets a fixed context variable ctx (e.g., "0") for the first binary number of the binary sequence in step S103. The arithmetic encoding unit 124 performs arithmetic encoding in step S105 by referring to the fixed context variable ctx (e.g., "0") using the normal CABAC pattern.

[0066] By designing in this way, the encoding device 100 can maintain encoding efficiency while simplifying the derivation of the context variables of the ST identifier. Furthermore, the encoding device 100 can maintain encoding efficiency while reducing the memory size used to hold the context variables of the ST identifier. In other words, the encoding device 100 can suppress the increase in the encoding processing load.

[0067] Furthermore, in step S103, the selection unit 122 can also set a bypass flag (method 1) for the second binary number of the binary sequence. In this case, the arithmetic encoding unit 125 performs arithmetic encoding on the second binary number in step S106 using the CABAC bypass mode.

[0068] Furthermore, in step S103, the selection unit 122 can also set a context variable ctx (e.g., "0") that is common to the first binary number of the binary sequence (method 2). In this case, the arithmetic encoding unit 124 performs arithmetic encoding in step S105 by referring to the fixed context variable ctx (e.g., "0") using the normal mode of CABAC.

[0069] <3. Second Implementation> <Decoding device> Figure 5 is a block diagram illustrating an example of the configuration of a decoding device in an image processing apparatus to which this technology is applied. The decoding device 200 shown in Figure 5 is an apparatus for decoding the encoded data of the ST identifier lfnst_idx. The decoding device 200 performs decoding using a decoding method (e.g., CABAC) corresponding to the encoding method of the encoding device 100. For example, the decoding device 200 decodes the encoded data of the ST identifier lfnst_idx generated by the encoding device 100.

[0070] Furthermore, Figure 5 mainly shows the processing unit or the flow of data, and what is shown in Figure 5 is not all. That is, in the decoding device 200, there may also be processing units that are not shown as blocks in Figure 5, and there may also be processing or data flow that is not indicated by arrows in Figure 5.

[0071] As shown in Figure 5, the decoding device 200 includes: a selection unit 221, a context model 222, an arithmetic decoding unit 223, an arithmetic decoding unit 224, a selection unit 225, and a multi-valued unit 226.

[0072] The selection unit 221 acquires the encoded data and the flag information isBypass input to the decoding device 200. Based on the value of isBypass, the selection unit 221 selects the target to which the encoded data is supplied. For example, if isBypass=0, the selection unit 221 determines it is in normal mode and supplies the encoded data to the context model 222. Conversely, if isBypass=1, the selection unit 221 determines it is in bypass mode and supplies the binary sequence to the arithmetic decoding unit 224.

[0073] The context model 222 dynamically switches between applicable context models based on the decoding object and its surrounding conditions. For example, the context model 222 stores the context variable ctx, and once the encoded data is obtained from the selection unit 221, it reads out the context variable ctx corresponding to each binary position (binIdx) of the bin column defined for each syntax element. The context model 222 then supplies the encoded data and the read context variable ctx to the arithmetic decoding unit 223.

[0074] The arithmetic decoding unit 223, upon receiving the encoded data and context variable ctx supplied from the context model 222, performs arithmetic decoding (context decoding) on ​​the binary value of the binIdx in the binarized bit column using the normal CABAC mode, referring to the probability state of the context variable ctx. The arithmetic decoding unit 223 then supplies the binarized bit column generated by the context decoding to the selection unit 225. Furthermore, the arithmetic decoding unit 223 supplies and holds the context variable ctx after context decoding processing to the context model 222.

[0075] The arithmetic decoding unit 224 performs arithmetic decoding (bypass decoding) on ​​the encoded data supplied from the selection unit 221 using the CABAC bypass mode. The arithmetic decoding unit 224 then supplies the binary bit array generated by the bypass decoding to the selection unit 225.

[0076] Selection unit 225 acquires the flag information isBypass and, based on the value of isBypass, selects the binary bit column to be supplied to multi-value unit 226. For example, when isBypass=0, selection unit 225 determines that it is in normal mode and acquires the binary bit column supplied from arithmetic decoding unit 223 and supplies it to multi-value unit 226. When isBypass=1, selection unit 225 determines that it is in bypass mode and acquires the binary bit column supplied from arithmetic decoding unit 224 and supplies it to multi-value unit 226.

[0077] The multi-valued unit 226 obtains the binary bit array supplied by the selection unit 225, multi-values ​​it according to the method defined for each syntax element, and generates a syntax element value. The multi-valued unit 226 outputs the syntax element value to the outside of the decoding device 200.

[0078] Furthermore, these processing units (selection unit 221 and even multi-valued unit 226) can have arbitrary configurations. For example, each processing unit can be configured by implementing the above-mentioned processing using logic circuits. Alternatively, each processing unit can have, for example, a CPU, ROM, RAM, etc., and use them to execute programs to implement the above-mentioned processing. Of course, each processing unit can also have both of these configurations, where part of the above-mentioned processing is implemented by logic circuits, and the rest is implemented by executing programs. The configurations of each processing unit can also be independent of each other. For example, one processing unit can implement a part of the above-mentioned processing using logic circuits, another processing unit can implement the above-mentioned processing by executing programs, and yet another processing unit can implement the above-mentioned processing by combining both logic circuits and program execution.

[0079] In such a decoding device 200, the technique described in <1. Encoding of ST identifier> is applied. That is, the arithmetic decoding unit 223 performs arithmetic decoding by referring to a fixed context variable ctx (e.g., "0") set for the first binary number of the binary sequence of the secondary conversion control information (ST identifier lfnst_idx), using the normal mode of CABAC.

[0080] With this design, the decoding device 200 can maintain encoding efficiency while simplifying the derivation of the context variables of the ST identifier. Furthermore, the decoding device 200 can maintain encoding efficiency while reducing the memory size used to hold the context variables of the ST identifier. In other words, the decoding device 200 can suppress the increase in the decoding processing load.

[0081] At this time, the arithmetic decoding unit 224 can also perform arithmetic decoding on the second binary number by using the CABAC bypass mode based on the bypass flag set on the second binary number of the binary sequence.

[0082] Furthermore, the arithmetic decoding unit 223 can also perform arithmetic decoding by referring to the context variable ctx (e.g., "0") that is common to the first binary number and is set to the second binary number of the binary sequence, using the normal mode of CABAC.

[0083] <Decoding Process> Next, an example of the decoding process performed by the decoding device 200 will be described with reference to the flowchart in FIG6.

[0084] Once the decoding process is started, the selection unit 221 of the decoding device 200 reads out the context variable ctx corresponding to each binary position (binIdx) of the bin column defined for each syntax element and the flag isBypass indicating whether it is a bypass mode in step S201.

[0085] In step S202, the selection unit 221 determines whether it is in bypass mode. If isBypass=0 and is determined to be in normal mode, the processing system proceeds to step S203.

[0086] In step S203, the arithmetic decoding unit 223 performs context decoding. That is, the arithmetic decoding unit 223 decodes the encoded data using the normal CABAC pattern, referring to the probability state of the context variable ctx, to generate the binary value of the bin column (synBins) located at the binary position (binIdx). Once the processing in step S203 is completed, the process proceeds to step S205.

[0087] Furthermore, in step S202, isBypass=1, and if it is determined to be in bypass mode, the processing system proceeds to step S204.

[0088] In step S204, the arithmetic decoding unit 224 performs bypass decoding. That is, the arithmetic decoding unit 224 decodes the encoded data using the CABAC bypass mode, generating the binary value of the bin column (synBins) located at the binary position (binIdx). Once the processing in step S204 is completed, the process proceeds to step S205.

[0089] In step S205, the selection unit 225 determines whether the predetermined rest condition A is met. The rest condition A is defined based on the values ​​of the binary sequence from binIdx=0 to binIdx=k (the current position k of binIdx) or the binarization method of each syntax element.

[0090] If the rest condition A is not met, the process returns to step S201 to generate the next binary position (binIdx) value and execute subsequent processing. That is, each binary position (binIdx) will be processed through steps S201 to S205.

[0091] Then, in step S205, if it is determined that rest condition A is met, the processing system proceeds to step S206.

[0092] In step S206, the multi-valued unit 226 derives the syntax element value (syncVal) from the binary sequence (synBins) by performing multi-valued processing defined for each syntax element.

[0093] In step S207, the multi-valued unit 226 outputs the exported syntax element value (syncVal) to the outside of the decoding device 200.

[0094] In such decoding processing, the technique described in <1. Encoding of ST identifier> is applied. That is, the arithmetic decoding unit 223, in step S203, refers to the fixed context variable ctx (e.g., "0") set for the first binary number of the binary sequence of the secondary conversion control information (ST identifier lfnst_idx), and performs arithmetic decoding in the normal mode of CABAC.

[0095] With this design, the decoding device 200 can maintain encoding efficiency while simplifying the derivation of the context variables of the ST identifier. Furthermore, the decoding device 200 can maintain encoding efficiency while reducing the memory size used to hold the context variables of the ST identifier. In other words, the decoding device 200 can suppress the increase in the decoding processing load.

[0096] Furthermore, the arithmetic decoding unit 224 can also perform arithmetic decoding of the second binary number in step S204 based on the bypass flag set for the second binary number of the binary sequence using the CABAC bypass mode (method 1).

[0097] Furthermore, the arithmetic decoding unit 223 can also perform arithmetic decoding in step S203 by referring to the context variable ctx (e.g., "0") that is common to the first binary number and set for the second binary number of the binary sequence, using the normal mode of CABAC (method 2).

[0098] <4. Third Implementation Form> <Image Encoding Device> Figure 7 is a block diagram illustrating an example of the configuration of an image encoding device, one type of image processing apparatus to which this technology is applied. The image encoding device 300 shown in Figure 7 is a device for encoding image data of moving images. For example, the image encoding device 300 can encode the image data of moving images using any of the encoding methods described in the aforementioned non-patent documents.

[0099] Furthermore, Figure 7 mainly shows the processing unit (block) or the flow of data, and what is shown in Figure 7 is not all. That is, in the image encoding device 300, there may also be processing units that are not shown as blocks in Figure 7, and there may also be processing or data flows that are not indicated by arrows in Figure 7.

[0100] As shown in Figure 7, the image encoding device 300 includes: a control unit 301, a sorting buffer 311, a calculation unit 312, an orthogonal conversion unit 313, a quantization unit 314, an encoding unit 315, an accumulation buffer 316, an inverse quantization unit 317, an inverse orthogonal conversion unit 318, a calculation unit 319, an in-loop filter unit 320, a frame memory 321, a prediction unit 322, and a rate control unit 323.

[0101] <Control Department> The control unit 301 divides the dynamic image data held by the sorting buffer 311 into blocks of processing units (CU, PU, ​​conversion blocks, etc.) based on the block size of the processing unit, either externally or pre-specified. Furthermore, the control unit 301 determines the encoding parameters (header information Hinfo, prediction mode information Pinfo, conversion information Tinfo, filter information Finfo, etc.) supplied to each block, for example, based on RDO (Rate-Distortion Optimization).

[0102] Details regarding these encoding parameters will be described later. Once the control unit 301 has determined the encoding parameters as described above, it supplies them to each block. Specifically, this is as follows.

[0103] Hinfo header information is provided to each block. The prediction pattern information Pinfo is supplied to the encoding unit 315 and the prediction unit 322. The conversion information Tinfo is supplied to the encoding unit 315, the orthogonal conversion unit 313, the quantization unit 314, the inverse quantization unit 317, and the inverse orthogonal conversion unit 318. The filter information Finfo is supplied to the loop filter section 320.

[0104] <Sort Buffer> In the image encoding device 300, each field (input image) of the moving image data is input according to its reproduction order (display order). The sorting buffer 311 acquires and stores (memorizes) each input image according to its reproduction order (display order). Based on the control of the control unit 301, the sorting buffer 311 sorts the input images into an encoding order (decoding order) or divides them into blocks of processing units, etc. The sorting buffer 311 supplies the processed input images to the calculation unit 312. Furthermore, the sorting buffer 311 also supplies each input image (original image) to the prediction unit 322 or the loop filter unit 320.

[0105] <Calculation Department> The calculation unit 312 takes the image I corresponding to the block of the processing unit and the prediction image P supplied by the prediction unit 322 as input, and performs subtraction on the prediction image P from the image I as shown in the following formula to derive the prediction residual D, and supplies it to the orthogonal conversion unit 313.

[0106] D=I - P

[0107] <Orthogonal Transition Unit> The orthogonal transformation unit 313 takes the predicted residual D supplied from the calculation unit 312 and the transformation information Tinfo supplied from the control unit 301 as inputs, and performs an orthogonal transformation on the predicted residual D based on the transformation information Tinfo to derive the transformation coefficient Coeff. For example, the orthogonal transformation unit 313 performs a primary transformation on the predicted residual D to generate primary transformation coefficients, and performs a secondary transformation on the primary transformation coefficients based on the ST identifier to generate secondary transformation coefficients. The orthogonal transformation unit 313 supplies the obtained secondary transformation coefficients as the transformation coefficient Coeff to the quantization unit 314.

[0108] Quantitative Department The quantization unit 314 takes the conversion coefficient Coeff supplied from the quadrature conversion unit 313 and the conversion information Tinfo supplied from the control unit 301 as inputs, and scales (quantizes) the conversion coefficient Coeff based on the conversion information Tinfo. Furthermore, the quantization rate is controlled by the rate control unit 323. The quantization unit 314 supplies the quantized conversion coefficients, i.e., the quantized conversion coefficient level, obtained through this quantization to the encoding unit 315 and the inverse quantization unit 317.

[0109] <Code Department> The encoding unit 315 takes as input the quantization conversion coefficient level supplied by the quantization unit 314, various encoding parameters (header information Hinfo, prediction mode information Pinfo, conversion information Tinfo, filter information Finfo, etc.) supplied by the control unit 301, filter-related information such as filter coefficients supplied by the in-loop filter unit 320, and the optimal prediction mode-related information supplied by the prediction unit 322. The encoding unit 315 performs variable-length encoding (e.g., arithmetic encoding) on ​​the quantization conversion coefficient level to generate a bit column (encoded data).

[0110] Furthermore, the encoding unit 315 derives the residual information Rinfo from the quantization conversion coefficient level, encodes the residual information Rinfo, and generates a bit column.

[0111] Then, the encoding unit 315 incorporates the filter-related information supplied from the loop filter unit 320 into the filter information Finfo, and incorporates the optimal prediction mode-related information supplied from the prediction unit 322 into the prediction mode information Pinfo. Then, the encoding unit 315 encodes the various encoding parameters (header information Hinfo, prediction mode information Pinfo, transformation information Tinfo, filter information Finfo, etc.) to generate a bit array.

[0112] Furthermore, the encoding unit 315 multiplexes the bit arrays of the various information generated as described above to generate encoded data. The encoding unit 315 then supplies the encoded data to the accumulation buffer 316.

[0113] <Accumulation Buffer> The accumulation buffer 316 temporarily stores the encoded data obtained in the encoding unit 315. The accumulation buffer 316 then outputs the stored encoded data, for example as a bitstream, to the outside of the image encoding apparatus 300 at a predetermined timing. For example, the encoded data can be transmitted to the decoding side via any recording medium, any transmission medium, or any information processing device. In other words, the accumulation buffer 316 is also a transmission unit that transmits the encoded data (bitstream).

[0114] <Inverse Quantization Department> The inverse quantization unit 317 performs inverse quantization processing. For example, the inverse quantization unit 317 takes the quantization conversion coefficient level (level) supplied from the quantization unit 314 and the conversion information (Tinfo) supplied from the control unit 301 as inputs, and scales the value of the quantization conversion coefficient level (level) proportionally based on the conversion information (Tinfo). Furthermore, this inverse quantization is the reverse processing of the quantization performed in the quantization unit 314. The inverse quantization unit 317 supplies the conversion coefficient Coeff_IQ obtained by such inverse quantization to the inverse quadrature conversion unit 318.

[0115] <Inverse-Orthogonal Transition Section> The inverse quadrature transformation unit 318 performs related processing for inverse quadrature transformation. For example, the inverse quadrature transformation unit 318 takes the transformation coefficient Coeff_IQ supplied from the inverse quantization unit 317 and the transformation information Tinfo supplied from the control unit 301 as inputs, and performs inverse quadrature transformation on the transformation coefficient Coeff_IQ based on the transformation information Tinfo to derive the prediction residual D'. Furthermore, this inverse quadrature transformation is the inverse processing of the quadrature transformation performed in the quadrature transformation unit 313. That is, the inverse quadrature transformation unit 318 can perform adaptive inverse quadrature transformation (AMT) by adaptively selecting the type (transformation coefficient) of the inverse quadrature transformation.

[0116] The inverse quadrature conversion unit 318 supplies the prediction residual D' obtained by such inverse quadrature conversion to the calculation unit 319. Furthermore, the inverse quadrature conversion unit 318 is the same as the inverse quadrature conversion unit on the decoding side (described later), so the description of the inverse quadrature conversion unit 318 is applicable to the decoding side (described later).

[0117] <Calculation Department> The calculation unit 319 takes the prediction residual D' supplied from the inverse quadrature conversion unit 318 and the prediction image P supplied from the prediction unit 322 as inputs. The calculation unit 319 adds the prediction residual D' and the prediction image P corresponding to the prediction residual D' to derive a local decoded image Rlocal. The calculation unit 319 supplies the derived local decoded image Rlocal to the loop filter unit 320 and the frame memory 321.

[0118] <Intraloop Filter Section> The in-loop filter unit 320 performs related processing for in-loop filter processing. For example, the in-loop filter unit 320 takes as input the local decoded image Rlocal supplied from the calculation unit 319, the filter information Finfo supplied from the control unit 301, and the input image (original image) supplied from the sorting buffer 311. In addition, the information input to the in-loop filter unit 320 is arbitrary, and other information can also be input. For example, prediction mode, motion information, target value of coding amount, quantization parameter QP, image type, block (CU, CTU, etc.) information can also be input to the in-loop filter unit 320 as needed.

[0119] The loop filter unit 320 performs appropriate filter processing on the local decoded image Rlocal based on the filter information Finfo. The loop filter unit 320 also uses the input image (original image) or other input information for filter processing as needed.

[0120] For example, the in-loop filter section 320, as described in Non-Patent Document 11, applies four in-loop filters—a bilateral filter, a deblocking filter (DBF), a sample adaptive offset filter (SAO), and an adaptive loop filter (ALF)—in that order. Furthermore, the choice of which filter to apply and in what order is arbitrary and can be appropriately selected.

[0121] Of course, the filter processing performed by the in-loop filter section 320 is arbitrary and not limited to the examples described above. For example, the in-loop filter section 320 can also be equipped with Wiener filters, etc.

[0122] The loop filter unit 320 supplies the locally decoded image Rlocal, which has been processed by the filter, to the frame memory 321. Furthermore, if filter-related information, such as filter coefficients, is transmitted to the decoding side, the loop filter unit 320 supplies this filter-related information to the encoding unit 315.

[0123] <Paint Frame Memory> Frame memory 321 performs related processing for remembering image data. For example, frame memory 321 takes the local decoded image Rlocal supplied from the calculation unit 319, or the locally decoded image Rlocal that has been filtered and supplied from the loop filter unit 320, as input and stores it (memorizes it). Furthermore, frame memory 321 uses the local decoded image Rlocal to reconstruct the decoded image R of each image unit and stores it (stored in the buffer within frame memory 321). Frame memory 321 supplies the decoded image R (or a portion thereof) to prediction unit 322 according to the request of prediction unit 322.

[0124] <Forecasting Department> The prediction unit 322 performs related processing for generating the predicted image. For example, the prediction unit 322 takes the prediction mode information Pinfo supplied by the control unit 301, the input image (original image) supplied by the sorting buffer 311, and the decoded image R (or a portion thereof) read from the frame memory 321 as input. The prediction unit 322 uses the prediction mode information Pinfo or the input image (original image) to perform prediction processing such as inter-frame prediction or intra-frame prediction, uses the decoded image R as a reference image for prediction, performs motion compensation processing based on the prediction result, and generates the predicted image P. The prediction unit 322 supplies the generated predicted image P to the calculation unit 312 and the calculation unit 319. Furthermore, the prediction unit 322 supplies the information related to the prediction mode selected through the above processing, i.e., the optimal prediction mode, to the encoding unit 315 as needed.

[0125] <Rate Control Department> The rate control unit 323 performs rate control-related processing. For example, the rate control unit 323 controls the rate of quantization operation of the quantization unit 314 in a manner that does not cause overflow or underflow, based on the amount of encoded data accumulated in the accumulation buffer 316.

[0126] Furthermore, these processing units (control unit 301, sorting buffer 311, and even rate control unit 323) can have arbitrary configurations. For example, each processing unit can be configured by implementing the above-mentioned processing using logic circuits. Alternatively, each processing unit can have, for example, a CPU, ROM, RAM, etc., and use them to execute programs to implement the above-mentioned processing. Of course, each processing unit can also have both of these configurations, where part of the above-mentioned processing is implemented using logic circuits, and the rest is implemented by executing programs. The configurations of each processing unit can also be independent of each other. For example, one processing unit can implement a part of the above-mentioned processing using logic circuits, another processing unit can implement the above-mentioned processing by executing programs, and yet another processing unit can implement the above-mentioned processing by combining both logic circuits and program execution.

[0127] In the image encoding apparatus 300 configured as described above, the technology described in <1.ST Encoding of Identifier Elements> can be applied to the encoding unit 315. That is, the encoding unit 315 has the same configuration as the encoding apparatus 100 shown in FIG3 and performs the same processing.

[0128] By designing in this way, the image encoding device 300 can maintain encoding efficiency while simplifying the derivation of context variables for ST identifiers. Furthermore, the image encoding device 300 can maintain encoding efficiency while reducing the memory size used to hold the context variables for ST identifiers. In other words, the image encoding device 300 can suppress the increase in the encoding processing load.

[0129] <Image Encoding Process> Next, an example of the image encoding process performed by the image encoding apparatus 300 configured as described above will be explained with reference to the flowchart in FIG8.

[0130] Once the image encoding process begins, in step S301, the sorting buffer 311, controlled by the control unit 301, sorts the order of the frames of the input dynamic image data from the display order to the encoding order.

[0131] In step S302, the control unit 301 sets a processing unit (performs block segmentation) for the input image held in the sorting buffer 311.

[0132] In step S303, the control unit 301 determines (sets) the encoding parameters for the input image held in the sorting buffer 311.

[0133] In step S304, the prediction unit 322 performs prediction processing to generate a prediction image of the best prediction mode. For example, in this prediction processing, the prediction unit 322 performs intra-frame prediction to generate a prediction image of the best intra-frame prediction mode, and performs inter-frame prediction to generate a prediction image of the best inter-frame prediction mode. From these, the best prediction mode is selected based on the cost function value, etc.

[0134] In step S305, the calculation unit 312 calculates the difference between the input image and the predicted image of the optimal mode selected through the prediction processing in step S304. That is, the calculation unit 312 generates the prediction residual D between the input image and the predicted image. The resulting prediction residual D has a lower data volume compared to the original image data. Therefore, compared to directly encoding the image, the data volume can be compressed.

[0135] In step S306, the orthogonal transformation unit 313 performs orthogonal transformation on the prediction residual D generated by the processing in step S305 to derive the transformation coefficient Coeff.

[0136] In step S307, the quantization unit 314 uses the quantization parameters calculated by the control unit 301 to quantize the conversion coefficient Coeff obtained in step S306 and derive the quantization conversion coefficient level.

[0137] In step S308, the inverse quantization unit 317 performs inverse quantization on the quantization conversion coefficient level generated in step S307, according to the characteristics corresponding to the quantization characteristics in step S307, to derive the conversion coefficient Coeff_IQ.

[0138] In step S309, the inverse quadrature transformation unit 318 performs an inverse quadrature transformation on the transformation coefficients Coeff_IQ obtained in step S308 using the method corresponding to the quadrature transformation process in step S306, thereby deriving the prediction residual D'. Furthermore, this inverse quadrature transformation process is the same as the inverse quadrature transformation process performed on the decoding side (described later), therefore, the explanation regarding the inverse quadrature transformation process in step S309 is applicable to the decoding side (described later).

[0139] In step S310, the calculation unit 319 adds the prediction image obtained by the prediction processing in step S304 to the prediction residual D' derived by the processing in step S309, so as to generate a decoded image that has been locally decoded.

[0140] In step S311, the loop filter unit 320 performs loop filter processing on the decoded image that has been locally decoded and exported by the processing in step S310.

[0141] In step S312, the frame memory 321 stores the decoded image that has been locally decoded and exported by the processing in step S310, or the decoded image that has been locally decoded and processed by the filter in step S311.

[0142] In step S313, the encoding unit 315 encodes the quantization conversion coefficient level obtained from the processing in step S307. For example, the encoding unit 315 encodes the image's related information, i.e., the quantization conversion coefficient level, using arithmetic coding or similar methods to generate encoded data. Furthermore, at this time, the encoding unit 315 encodes various encoding parameters (header information Hinfo, prediction mode information Pinfo, and conversion information Tinfo). Then, the encoding unit 315 derives residual information RInfo from the quantization conversion coefficient level and encodes this residual information RInfo.

[0143] In step S314, the accumulation buffer 316 accumulates the encoded data thus obtained, for example, as a bitstream, and outputs it to the outside of the image encoding device 300. This bitstream is transmitted to the decoding side, for example, through a transmission path or recording medium. Furthermore, the rate control unit 323 performs rate control as needed.

[0144] Once step S314 is completed, the image encoding process ends.

[0145] In the image encoding process described above, the encoding process in step S313 is subject to this technology. That is, in step S313, the encoding process is performed in the same manner as in FIG4. In other words, the encoding unit 315 performs the encoding process using the technology described in <1. ST Encoding of Identifier>.

[0146] By designing in this way, the image encoding device 300 can maintain encoding efficiency while simplifying the derivation of context variables for ST identifiers. Furthermore, the image encoding device 300 can maintain encoding efficiency while reducing the memory size used to hold the context variables for ST identifiers. In other words, the image encoding device 300 can suppress the increase in the encoding processing load.

[0147] <5. Fourth Implementation Form> <Image Decoding Device> Figure 9 is a block diagram illustrating an example of the configuration of an image decoding apparatus, one type of image processing apparatus to which this technology is applied. The image decoding apparatus 400 shown in Figure 9 is an apparatus for encoding encoded data of moving images. For example, the image decoding apparatus 400 can decode the encoded data using any of the decoding methods described in the aforementioned non-patent documents. For example, the image decoding apparatus 400 decodes the encoded data (bit stream) generated by the aforementioned image encoding apparatus 300.

[0148] Furthermore, Figure 9 mainly shows the processing unit (block) or the flow of data, and what is shown in Figure 9 is not all. That is, in the image decoding device 400, there may also be processing units that are not shown as blocks in Figure 9, and there may also be processing or data flows that are not indicated by arrows in Figure 9.

[0149] As shown in Figure 9, the image decoding apparatus 400 includes: an accumulation buffer 411, a decoding unit 412, an inverse quantization unit 413, an inverse quadrature conversion unit 414, an calculation unit 415, an in-loop filter unit 416, a sorting buffer 417, a frame memory 418, and a prediction unit 419. Furthermore, the prediction unit 419 includes an in-frame prediction unit and an inter-frame prediction unit (not shown). The image decoding apparatus 400 is a device required to generate moving image data by decoding encoded data (bitstream).

[0150] <Accumulation Buffer> The accumulation buffer 411 acquires and stores (memorizes) the bitstream that has been input to the image decoding device 400. The accumulation buffer 411 supplies the accumulated bitstream to the decoding unit 412 when a predetermined timing or predetermined conditions are met.

[0151] <Decoding Department> The decoding unit 412 performs related processing for decoding the image. For example, the decoding unit 412 takes the bit stream supplied from the accumulation buffer 411 as input, performs variable-length decoding of the syntax values ​​of each syntax element from the bit column according to the definition of the syntax table, and derives the parameters.

[0152] The parameters derived from the grammatical elements and their grammatical values ​​include information such as header information (Hinfo), prediction mode information (Pinfo), transformation information (Tinfo), residual information (Rinfo), and filter information (Finfo). That is, the decoding unit 412 parses (analyzes and obtains) this information from the bitstream. This information is explained below.

[0153] <Header Information Hinfo> The header information (Hinfo) contains header information such as VPS (Video Parameter Set), SPS (Sequence Parameter Set), PPS (Picture Parameter Set), and SH (slice header). The Hinfo header information includes information specifying image dimensions (PicWidth, PicHeight), bit depth (bitDepthY, bitDepthC), chroma array type (ChromaArrayType), maximum / minimum CU size (MaxCUSize / MinCUSize), maximum / minimum depth (MaxQTDepth / MinQTDepth) for quad-tree segmentation, maximum / minimum depth (MaxBTDepth / MinBTDepth) for binary-tree segmentation, maximum TSSize (MaxTSSize, also known as maximum skipped block size), and enable / disable flags (also known as valid flags) for various encoding tools.

[0154] For example, the enable / disable flags for encoding tools included in the header information Hinfo include those related to conversion and quantization processing, as shown below. Furthermore, the enable / disable flags for encoding tools can also be interpreted as indicating whether the syntax involved in the encoding tool exists in the encoded data. Also, a value of 1 (true) for the enable / disable flag indicates that the encoding tool is usable, while a value of 0 (false) indicates that the encoding tool is unusable. Moreover, the interpretation of the flag values ​​can also be reversed.

[0155] Cross-Component Prediction Enabled Flag (ccp_enabled_flag): This flag indicates whether Cross-Component Prediction (CCP, also known as CC prediction) is usable. For example, a value of "1" (true) indicates that it is usable, while a value of "0" (false) indicates that it is not usable.

[0156] Furthermore, this CCP is also known as inter-component linear prediction (CCLM or CCLMP).

[0157] <Predictive Pattern Information Pinfo> The prediction mode information Pinfo contains information such as: the size information of the processing object PB (prediction block) PBSize (prediction block size), the in-screen prediction mode information IPinfo, and the motion prediction information MVinfo.

[0158] The prediction mode information in the IPinfo file contains, for example: JCTVC-W1005, 7.3.8.5 Coding Unit syntax. prev_intra_luma_pred_flag, mpm_idx, rem_intra_pred_mode, and the in-image luminance prediction mode IntraPredModeY derived from this syntax.

[0159] Furthermore, the in-screen prediction mode information IPinfo contains, for example: inter-component prediction flag (ccp_flag (cclmp_flag)), multi-level linear prediction mode flag (mclm_flag), chroma sample location type identifier (chroma_sample_loc_type_idx), chroma MPM identifier (chroma_mpm_idx), and the luminance in-screen prediction mode (IntraPredModeC) derived from these syntaxes.

[0160] The inter-component prediction flag (ccp_flag(cclmp_flag)) is a flag indicating whether inter-component linear prediction is applicable. For example, ccp_flag==1 indicates that inter-component prediction should be applied, and ccp_flag==0 indicates that inter-component prediction should not be applied.

[0161] The multi-level linear forecasting mode flag (mclm_flag) contains information related to the linear forecasting mode (linear forecasting mode information). More specifically, the multi-level linear forecasting mode flag (mclm_flag) indicates whether a multi-level linear forecasting mode is set. For example, "0" indicates a level 1 mode (single-level mode) (e.g., CCLMP), and "1" indicates a level 2 mode (multi-level mode) (e.g., MCLMP).

[0162] The chroma_sample_loc_type_idx is an identifier used to identify the type of pixel position of a chroma component (also known as the chroma_sample_loc_type_idx). For example, if the chroma format information, namely the chroma array type (ChromaArrayType), indicates a 420 format, the chroma_sample_loc_type_idx is assigned in the following manner.

[0163] chroma_sample_loc_type_idx==0:Type2 chroma_sample_loc_type_idx==1:Type3 chroma_sample_loc_type_idx==2:Type0 chroma_sample_loc_type_idx==3:Type1

[0164] In addition, the color difference sample location type identifier (chroma_sample_loc_type_idx) is transmitted as information related to the pixel location of the color difference component (chroma_sample_loc_info()) (which is stored therein).

[0165] The color difference MPM identifier (chroma_mpm_idx) indicates which prediction mode candidate in the color difference image intraPredModeCandListC should be designated as the identifier for the color difference image intraPredModeCandListC.

[0166] The motion prediction information MVinfo contains information such as merge_idx, merge_flag, inter_pred_idc, ref_idx_LX, mvp_lX_flag, X={0,1}, mvd, etc. (for example, refer to JCTVC-W1005, 7.3.8.6 Prediction Unit Syntax).

[0167] Of course, the information contained in the prediction model information Pinfo is arbitrary and may also contain information other than this.

[0168] <Conversion Information Tinfo> The transformation information Tinfo contains information such as the following. Of course, the information contained in the transformation information Tinfo is arbitrary and may also contain information other than these.

[0169] Process the banner size TBWSize and vertical size TBHSize of the object conversion block (or the logarithmic values ​​of each TBWSize and TBHSize, base 2, log2TBWSize and log2TBHSize). Transformation Skip Flag (ts_flag): A flag indicating whether to skip (inverse) primary transformations and (inverse) secondary transformations. Scan Idx Quantization parameter (qp) Quantization matrix (e.g., JCTVC-W1005, 7.3.4 Scaling list data syntax)

[0170] Residual Information (Rinfo) The residual information Rinfo (for example, refer to JCTVC-W1005 7.3.8.11 Residual Coding syntax) contains syntax such as the following.

[0171] cbf(coded_block_flag): Whether the residual data has a flag. last_sig_coeff_x_pos: The X coordinate of the last non-zero coefficient. last_sig_coeff_y_pos: The Y-coordinate of the last non-zero coefficient. coded_sub_block_flag: Whether the non-zero coefficients of the sub-block have a flag. sig_coeff_flag: Whether non-zero coefficients are flagged gr1_flag: A flag indicating that the level of a non-zero coefficient is greater than 1 (also known as the GR1 flag). gr2_flag: A flag indicating that the level of a non-zero coefficient is greater than 2 (also known as the GR2 flag). sign_flag: Indicates the sign of a non-zero coefficient (also known as a symbol). coeff_abs_level_remaining: The residual level of non-zero coefficients (also known as the residual level of non-zero coefficients), etc.

[0172] Of course, the information contained in the residual information Rinfo is arbitrary and may also contain information other than this information.

[0173] <Filter Information Finfo> The filter information Finfo contains, for example, control information related to each filter process, as shown below.

[0174] Control information related to the deblocking filter (DBF) Control information related to Pixel Adaptive Offset (SAO) Control information related to the adaptive loop filter (ALF) Control information related to other linear and nonlinear filters

[0175] More specifically, it contains, for example, information about the regions within the image applicable to each filter, or the region within the instruction image, or filter on / off control information for CU units, and filter on / off control information regarding the boundaries of slices and tiles. Of course, the information contained in the filter information Finfo is arbitrary and may also contain information other than these.

[0176] Returning to the description of the decoding unit 412, the decoding unit 412 derives the quantization conversion coefficient level (level) for each coefficient position within each conversion block by referring to the residual information Rinfo. The decoding unit 412 then supplies this quantization conversion coefficient level (level) to the inverse quantization unit 413.

[0177] Furthermore, the decoding unit 412 supplies the analyzed header information Hinfo, prediction mode information Pinfo, quantization conversion coefficient level, conversion information Tinfo, and filter information Finfo to each block. Specifically, it is as follows.

[0178] The header information Hinfo is supplied to the inverse quantization unit 413, the inverse quadrature conversion unit 414, the prediction unit 419, and the loop filter unit 416. The prediction model information Pinfo is supplied to the inverse quantization unit 413 and the prediction unit 419. The conversion information Tinfo is supplied to the inverse quantization unit 413 and the inverse orthogonal conversion unit 414. The filter information Finfo is supplied to the loop filter section 416.

[0179] Of course, the above example is just one example and is not limited to this one. For example, each encoding parameter can also be supplied to any processing unit. Furthermore, other information can also be supplied to any processing unit.

[0180] <Inverse Quantization Department> The inverse quantization unit 413 has at least the necessary components for performing inverse quantization related processing. For example, the inverse quantization unit 413 takes the conversion information Tinfo and the quantization conversion coefficient level supplied by the decoding unit 412 as input, scales the value of the quantization conversion coefficient level (inverse quantization) based on the conversion information Tinfo, and derives the inverse quantization conversion coefficient Coeff_IQ.

[0181] Furthermore, this inverse quantization is performed as the reverse process of the quantization caused by the quantization unit 314. Also, this inverse quantization is the same process as the inverse quantization caused by the inverse quantization unit 317. That is, the inverse quantization unit 317 performs the same process (inverse quantization) as the inverse quantization unit 413.

[0182] The inverse quantization unit 413 supplies the derived conversion coefficients Coeff_IQ to the inverse quadrature conversion unit 414.

[0183] <Inverse-Orthogonal Transition Section> The inverse quadrature transformation unit 414 performs related processing for inverse quadrature transformation. For example, the inverse quadrature transformation unit 414 takes the transformation coefficient Coeff_IQ supplied by the inverse quantization unit 413 and the transformation information Tinfo supplied by the decoding unit 412 as inputs, and performs inverse quadrature transformation processing on the transformation coefficient Coeff_IQ based on the transformation information Tinfo to derive the prediction residual D'. For example, the inverse quadrature transformation unit 414 performs inverse secondary transformation on the transformation coefficient Coeff_IQ based on the ST identifier to generate primary transformation coefficients, and performs primary transformation on the primary transformation coefficients to generate the prediction residual D'.

[0184] Furthermore, this inverse orthogonal conversion is performed as the reverse of the orthogonal conversion caused by orthogonal conversion unit 313. Also, this inverse orthogonal conversion is the same process as the inverse orthogonal conversion caused by inverse orthogonal conversion unit 318. That is, inverse orthogonal conversion unit 318 performs the same process (inverse orthogonal conversion) as inverse orthogonal conversion unit 414.

[0185] The inverse orthogonal transformation unit 414 supplies the derived prediction residual D' to the calculation unit 415.

[0186] <Calculation Department> The calculation unit 415 performs related processing, such as adding relevant information about the image. For example, the calculation unit 415 takes the prediction residual D' supplied from the inverse quadrature conversion unit 414 and the prediction image P supplied from the prediction unit 419 as input. The calculation unit 415 adds the prediction residual D' to the prediction image P (prediction signal) corresponding to the prediction residual D' as shown in the following formula, and then derives the local decoded image Rlocal.

[0187] Rlocal=D' + P

[0188] The calculation unit 415 supplies the exported local decoded image Rlocal to the loop filter unit 416 and the frame memory 418.

[0189] <Intraloop Filter Section> The loop filter unit 416 performs related processing for loop filter processing. For example, the loop filter unit 416 takes the local decoded image Rlocal supplied from the calculation unit 415 and the filter information Finfo supplied from the decoding unit 412 as inputs. In addition, the information input to the loop filter unit 416 is arbitrary, and information other than this information can also be input.

[0190] The loop filter unit 416 performs appropriate filter processing on the local decoded image Rlocal based on the filter information Finfo.

[0191] For example, the in-loop filter section 416, as described in Non-Patent Document 11, applies four in-loop filters—a bilateral filter, a deblocking filter (DBF), a sample adaptive offset filter (SAO), and an adaptive loop filter (ALF)—in that order. Furthermore, the choice of which filter to apply and in what order is arbitrary and can be appropriately selected.

[0192] The in-loop filter unit 416 performs filter processing corresponding to the filter processing performed on the encoding side (e.g., the in-loop filter unit 320 of the image encoding device 300). Of course, the filter processing performed by the in-loop filter unit 416 is arbitrary and not limited to the examples described above. For example, the in-loop filter unit 416 may also use Wiener filters, etc.

[0193] The loop filter section 416 supplies the locally decoded image Rlocal, which has been processed by the filter, to the sorting buffer 417 and the frame memory 418.

[0194] <Sort Buffer> The sorting buffer 417 takes the local decoded image Rlocal supplied from the loop filter unit 416 as input and stores it (memorizes it). The sorting buffer 417 reconstructs the decoded image R of each image unit using the local decoded image Rlocal and stores it (stores it in the buffer). The sorting buffer 417 sorts the obtained decoded images R from the decoding order to the playback order. The sorting buffer 417 outputs the sorted group of decoded images R as dynamic image data to the outside of the image decoding device 400.

[0195] <Paint Frame Memory> Frame memory 418 is used for memory-related processing of image data. For example, frame memory 418 takes the local decoded image Rlocal supplied by the calculation unit 415 as input, reconstructs the decoded image R of each image unit, and stores it in the buffer in frame memory 418.

[0196] Furthermore, the grid memory 418 takes the local decoded image Rlocal, which has been processed by the in-loop filter and supplied by the in-loop filter unit 416, as input, reconstructs the decoded image R of each image unit, and stores it in the buffer within the grid memory 418. The grid memory 418 is adapted to supply the decoded image R (or a portion thereof) stored therein to the prediction unit 419 as a reference image.

[0197] In addition, the frame memory 418 can also store header information Hinfo, prediction mode information Pinfo, transformation information Tinfo, filter information Finfo, etc., involved in the generation of the decoded image.

[0198] <Forecasting Department> The prediction unit 419 performs related processing for generating the predicted image. For example, the prediction unit 419 takes the prediction mode information Pinfo supplied by the decoding unit 412 as input, performs prediction using the prediction method specified by the prediction mode information Pinfo, and derives the predicted image P. During this deriver, the prediction unit 419 uses the decoded image R (or a portion thereof) stored in the frame memory 418, specified by the prediction mode information Pinfo, as a reference image. The prediction unit 419 then supplies the derived predicted image P to the calculation unit 415.

[0199] Furthermore, these processing units (accumulation buffer 411 and even prediction unit 419) can have arbitrary configurations. For example, each processing unit can be configured by implementing the above-mentioned processing using logic circuits. Alternatively, each processing unit can have, for example, a CPU, ROM, RAM, etc., and use them to execute programs to implement the above-mentioned processing. Of course, each processing unit can also have both of these configurations, where part of the above-mentioned processing is implemented by logic circuits, and the rest is implemented by executing programs. The configurations of each processing unit can also be independent of each other. For example, one processing unit can implement a part of the above-mentioned processing using logic circuits, another processing unit can implement the above-mentioned processing by executing programs, and yet another processing unit can implement the above-mentioned processing by combining both logic circuits and program execution.

[0200] In the image decoding apparatus 400 configured as described above, the technology described in <1.ST Encoding of Identifier Elements> can be applied to the decoding unit 412. That is, the decoding unit 412 has the same configuration as the decoding apparatus 200 shown in FIG. 5, and performs the same processing.

[0201] With this design, the image decoding device 400 can maintain encoding efficiency while simplifying the derivation of context variables of the ST identifier. Furthermore, the image decoding device 400 can maintain encoding efficiency while reducing the memory size used to hold the context variables of the ST identifier. In other words, the image decoding device 400 can suppress the increase in the decoding processing load.

[0202] <Image Encoding Process> Next, an example of the image decoding process performed by the image decoding device 400 configured as described above will be explained with reference to the flowchart in FIG10.

[0203] Once the image decoding process begins, the accumulation buffer 411, in step S401, acquires and stores (accumulates) the encoded data (bit stream) supplied from outside the image decoding device 400.

[0204] In step S402, the decoding unit 412 decodes the encoded data (bit stream) to obtain the quantization conversion coefficient level. Furthermore, the decoding unit 412 uses this decoding to analyze (parse and obtain) various encoding parameters from the encoded data (bit stream).

[0205] In step S403, the inverse quantization unit 413 performs inverse quantization on the quantization conversion coefficient level obtained by the processing in step S402, which is the inverse processing of the quantization performed on the encoding side, to obtain the conversion coefficient Coeff_IQ.

[0206] In step S404, the inverse orthogonal transformation unit 414 performs the inverse orthogonal transformation process on the transformation coefficients Coeff_IQ obtained in step S403, which is the inverse orthogonal transformation process, to obtain the prediction residual D'.

[0207] In step S405, the prediction unit 419 performs prediction processing based on the information parsed in step S402, using the prediction method specified by the encoding side, and generates a prediction image P by referring to the reference image stored in the frame memory 418.

[0208] In step S406, the calculation unit 415 adds the prediction residual D' obtained in step S404 with the prediction image P obtained in step S405 to derive the local decoded image Rlocal.

[0209] In step S407, the loop filter unit 416 performs loop filter processing on the local decoded image Rlocal obtained by the processing in step S406.

[0210] In step S408, the sorting buffer 417 uses the locally decoded image Rlocal, which has been processed by the filter obtained in step S407, to derive the decoded image R, and sorts the order of the decoded image R group from the decoding order to the playback order. The decoded image R group, which has been sorted into the playback order, is output as a moving image to the outside of the image decoding device 400.

[0211] Furthermore, in step S409, the grid memory 418 stores at least one of the local decoded image Rlocal obtained by the processing in step S406 and the local decoded image Rlocal after filter processing obtained by the processing in step S407.

[0212] Once step S409 is completed, the image decoding process is finished.

[0213] In the image decoding process described above, this technology is applied to the decoding process in step S402. That is, in step S402, the decoding process is performed in the same manner as in FIG6. That is, the decoding unit 412 performs the decoding process using the technology described in <1.ST Encoding of Identifier>.

[0214] With this design, the image decoding device 400 can maintain encoding efficiency while simplifying the derivation of context variables of the ST identifier. Furthermore, the image decoding device 400 can maintain encoding efficiency while reducing the memory size used to hold the context variables of the ST identifier. In other words, the image decoding device 400 can suppress the increase in the decoding processing load.

[0215] <6. Postscript> Computers The aforementioned series of processes can be executed by hardware or software. When the series of processes is executed by software, the program constituting the software can be installed on a computer. Here, "computer" includes: a computer assembled in dedicated hardware, or a general-purpose personal computer that can perform various functions by installing various programs.

[0216] Figure 11 is a block diagram of an example of the hardware configuration of a computer that uses a program to execute the above series of processes.

[0217] In the computer 800 shown in Figure 11, the CPU (Central Processing Unit) 801, ROM (Read Only Memory) 802, and RAM (Random Access Memory) 803 are connected to each other through bus 804.

[0218] Bus 804 is also connected to input / output interface 810. Input / output interface 810 is connected to: input section 811, output section 812, memory section 813, communication section 814, and driver 815.

[0219] The input unit 811 is composed of, for example, a keyboard, mouse, microphone, touch panel, and input terminals. The output unit 812 is composed of, for example, a display, speaker, and output terminals. The memory unit 813 is composed of, for example, a hard drive, RAM disk, or non-volatile memory. The communication unit 814 is composed of, for example, a network interface. The drive unit 815 drives removable media 821 such as magnetic disks, optical disks, optical discs, or semiconductor memory.

[0220] In a computer configured as described above, the CPU 801 loads, for example, the program stored in the memory unit 813 into the RAM 803 via the input / output interface 810 and bus 804 and executes it, thus performing the aforementioned series of processes. The RAM 803 also appropriately stores the data necessary for the CPU 801 to perform various processes.

[0221] The program executed by the computer can be recorded in a removable medium 821, such as a packaging medium, and is applicable thereto. In this case, the program can be installed into the memory unit 813 through the input / output interface 810 by attaching the removable medium 821 to the drive 815.

[0222] Furthermore, the program can be provided via wired or wireless transmission media such as local area networks, the Internet, or digital satellite broadcasting. In this case, the program can be received by the communication unit 814 and installed into the memory unit 813.

[0223] In addition, the program can be pre-installed in ROM 802 or memory unit 813.

[0224] <Applicable Objects of This Technology> This technology is applicable to any image encoding and decoding method. That is, as long as it does not contradict the above-described technology, the specifications of various image encoding and decoding processes such as conversion (inverse conversion), quantization (inverse quantization), encoding (decoding), and prediction are arbitrary and not limited to the examples mentioned above. Furthermore, as long as it does not contradict the above-described technology, some of these processes can be omitted.

[0225] Furthermore, this technology is applicable to multi-view image encoding and decoding systems that encode and decode multi-view images containing multiple views. In this case, this technology can be applied only to the encoding and decoding of each view.

[0226] Furthermore, this technology can be applied to layered image encoding (adjustable encoding) and decoding systems that encode and decode layered images that are multi-layered (hierarchical) with adjustable parameters. In this case, this technology can be applied to the encoding and decoding of each layer.

[0227] Furthermore, although the above examples illustrate the application of this technology, including encoding device 100, decoding device 200, image encoding device 300, and image decoding device 400, this technology is applicable to any configuration.

[0228] For example, this technology can be applied to various electronic machines, such as satellite broadcasting, cable TV, internet distribution, and distribution to terminals via cellular base station communication, including transmitters or receivers (e.g., television receivers or mobile phones), devices that record images on media such as optical discs, magnetic disks, and flash memory, or devices that reproduce images from these memory media (e.g., hard disk recorders or camcorders).

[0229] For example, this technology can also be implemented as a processor (e.g., a video processor) of a system LSI (Large Scale Integration), a module (e.g., a video module) using multiple processors, a unit (e.g., a video unit) using multiple modules, or a suite formed by adding other functions to a unit (e.g., a video suite), as part of a device.

[0230] For example, this technology can also be applied to network systems composed of multiple devices. For instance, it can be implemented through cloud computing, where the workload is distributed among multiple devices via a network for collaborative processing. Alternatively, it can be implemented in cloud services that provide image (moving video) related services to any terminal such as computers, AV (Audio Visual) machines, portable information processing terminals, and IoT (Internet of Things) devices.

[0231] Furthermore, in this specification, the term "system" means a collection of multiple constituent elements (devices, modules (parts), etc.), regardless of whether all constituent elements are located in the same enclosure. Therefore, multiple devices housed in individual enclosures and connected via a network, and a single device containing multiple modules in a single enclosure, are both systems.

[0232] <Applicable fields and uses of this technology> Systems, devices, and processing units utilizing this technology can be applied to any field, such as transportation, medical care, security, agriculture, animal husbandry, mining, beauty, factories, home appliances, meteorology, and nature monitoring. Furthermore, their uses are also arbitrary.

[0233] For example, this technology can be applied to systems or devices used for providing content for viewing. It can also be applied to systems or devices used for traffic monitoring or automatic driving control. Furthermore, it can be applied to systems or devices used for safety. It can also be applied to systems or devices used for the automatic control of machinery. Furthermore, it can be applied to systems or devices used in agriculture or animal husbandry. It can also be applied to systems or devices used for monitoring the natural state of volcanoes, forests, oceans, or wildlife. Finally, it can be applied to systems or devices used for sports.

[0234] <Other> Furthermore, the term "flag" in this specification refers to information required to identify multiple states. It includes not only information used to identify the two states of true (1) or false (0), but also information that can identify three or more states. Therefore, the value that the "flag" can take can be, for example, two values ​​of 1 / 0, or three or more values. That is, the number of bits constituting the "flag" is arbitrary, and can be 1 bit or multiple bits. Also, the identification information (including the flag) not only includes the identification information in the form of a bit stream, but also assumes that the difference information of the identification information relative to a certain reference information is included in the form of a bit stream. Therefore, in this specification, "flag" or "identification information" not only includes the information itself, but also includes the difference information relative to the reference information.

[0235] Furthermore, various information (metadata, etc.) related to encoded data (bit stream) can be transmitted or recorded in any form as long as a connection is established with the encoded data. Here, "establishing a connection" means, for example, a situation where data from another party can be used while data from one party is being processed (a connection can be established). That is, data that are linked to each other can be considered as a whole or as separate data. For example, information linked to encoded data (image) can be transmitted on a different transmission path than that encoded data (image). Also, for example, information linked to encoded data (image) can be recorded on a different recording medium (or another recording area of ​​the same recording medium) than that encoded data (image). Moreover, this "establishing a connection" may not be the entire data, but a part of it. For example, an image and the information corresponding to that image can be linked using any unit such as multiple frames, one frame, or a portion of a frame.

[0236] Furthermore, in this specification, the terms "synthesis", "multifunctionalization", "addition", "integration", "inclusion", "storage", "placement", "insertion", "plugging", etc., mean, for example, summarizing coded data and meta data into one data, or binding multiple items into one, which means a method of "establishing a relationship" as mentioned above.

[0237] Furthermore, the implementation of this technology is not limited to the above-mentioned implementation forms, and various changes can be made within the scope of the main idea of ​​this technology.

[0238] For example, the configuration described as a single device (or processing unit) can be divided into multiple devices (or processing units). Conversely, the configuration described as multiple devices (or processing units) in the above description can be summarized into a single device (or processing unit). Furthermore, configurations other than those mentioned above can certainly be added to the configurations of each device (or processing unit). Moreover, if the configuration or operation of the entire system is substantially the same, a part of the configuration of a certain device (or processing unit) can be included in the configuration of other devices (or other processing units).

[0239] Furthermore, for example, the aforementioned program can be executed on any device. In this case, it is sufficient as long as the device has the necessary functions (functional blocks, etc.) and can obtain the necessary information.

[0240] Furthermore, for example, each step of a flowchart can be executed by a single device or by multiple devices. Even if a single step contains multiple processes, those multiple processes can also be executed by a single device or by multiple devices. In other words, multiple processes within a single step can be executed as multiple steps. Conversely, processes described as multiple steps can be integrated into a single step for execution.

[0241] Furthermore, for example, the steps of a program executed by a computer can be executed sequentially according to the order described in this specification, or they can be executed individually in parallel, or as necessary, such as during a call. That is, as long as there is no contradiction, the steps can be executed in a different order than described above. Moreover, the steps describing the program can be executed in parallel with the processes of other programs, or in combination with the processes of other programs.

[0242] Furthermore, for example, the plurality of technologies related to this technology can be implemented independently as individual units, provided that no contradiction arises. Of course, any plurality of these technologies can also be implemented in combination. For example, part or all of the technology described in any embodiment can be combined with part or all of the technology described in other embodiments. Also, any part or all of the aforementioned technologies can be implemented in combination with other technologies not mentioned above.

[0243] In addition, this technology can also be configured as follows. (1) An image processing device comprising: The encoding unit sets a fixed context variable for the first binary number of the binary sequence of the secondary conversion control information, which is the control information for secondary conversion. Referring to the fixed context variable, it performs arithmetic encoding on each binary number of the aforementioned secondary conversion control information. (2) The image processing apparatus as described in (1), wherein, The context variable is fixed as "0". (3) The image processing apparatus as described in (1), wherein, The preceding encoding section also sets a bypass flag for the second binary number in the preceding binary sequence. (4) The image processing apparatus as described in (1), wherein, The preceding encoding section also sets a context variable common to the preceding binary number for the second binary number of the preceding binary sequence, which is the same as that for the preceding first binary number. (5) The image processing apparatus as described in (4), wherein, The common context for the preceding text is "0". (6) The image processing apparatus as described in (1), wherein, The aforementioned secondary conversion control information contains a secondary conversion identifier element that indicates the type of the aforementioned secondary conversion. (7) The image processing apparatus as described in (1), wherein, The preceding encoding unit converts the preceding secondary conversion control information into binary to generate a preceding binary sequence, and sets a preceding fixed context variable for the first binary number of the generated preceding binary sequence. (8) The image processing apparatus as described in (1), wherein, It also includes: a coefficient conversion unit, which performs a prior secondary conversion on the primary conversion coefficients of the conversion block corresponding to the prior secondary conversion control information to generate secondary conversion coefficients. (9) The image processing apparatus as described in (8), wherein, It also has a quantization unit, which quantizes the previous secondary conversion coefficients generated by the previous coefficient conversion unit to generate quantized coefficients. (10) An image processing method, A fixed context variable is set for the first binary number of the binary sequence of the secondary transition control information. The binary numbers of the aforementioned secondary transition control information are arithmetically encoded by referring to the fixed context variable.

[0244] (11) An image processing apparatus comprising: The decoding unit performs arithmetic decoding by referring to a fixed context variable set for the first binary number of the binary sequence of the control information regarding the secondary transformation, which is the control information regarding the secondary transformation. (12) The image processing apparatus as described in (11), wherein, The context variable is fixed as "0". (13) The image processing apparatus as described in (11), wherein, The preceding decoding unit also performs arithmetic decoding based on the bypass flag set for the second binary number of the preceding binary sequence. (14) The image processing apparatus as described in (11), wherein, The aforementioned decoding unit also performs arithmetic decoding by referring to the context variables that are common to the first binary number in the aforementioned binary sequence and are set for the second binary number in the aforementioned binary sequence. (15) The image processing apparatus as described in (14), wherein, The common context for the preceding text is "0". (16) The image processing apparatus as described in (11), wherein, The aforementioned secondary conversion control information contains a secondary conversion identifier element that indicates the type of the aforementioned secondary conversion. (17) The image processing apparatus as described in (11), wherein, The preceding decoding unit performs multi-valued transformation on the preceding binary sequence obtained by arithmetic decoding to generate preceding secondary conversion control information. (18) The image processing apparatus as described in (11), wherein, It also has: an inverse coefficient conversion unit, which performs inverse secondary conversion on the secondary conversion coefficients of the conversion block corresponding to the previous secondary conversion control information to generate primary conversion coefficients. (19) The image processing apparatus as described in (18), wherein, It also features: an inverse quantization unit, which inverse quantizes the quantization coefficients to generate the previous secondary conversion coefficients; The preceding inverse coefficient conversion unit performs a preceding inverse secondary conversion on the preceding secondary conversion coefficients generated by the preceding inverse quantization unit to generate the preceding primary conversion coefficients. (20) An image processing method, Arithmetic decoding is performed with reference to a fixed context variable set for the first binary number of the binary sequence of the secondary transformation control information, which is the control information for the secondary transformation.

[0245] 100: Encoding device 121:2 Value-based Department 122: Selection Department 123: Context Model 124: Arithmetic Coding Department 125: Arithmetic Coding Department 126: Selection Department 200: Decoding device 221: Selection Department 222: Context Model 223: Arithmetic Decoding Department 224: Arithmetic Decoding Department 225: Selection Department 226: Multivalued part 300: Image Decoding Device 301: Control Department 311: Sort Buffer 312: Calculation Department 313: Orthogonal Transformation Part 314: Quantitative Department 315: Coding Department 316: Accumulation Buffer 317: Inverse Quantization Department 318: Inverse orthogonal transformation part 319: Calculation Department 320: In-loop filter section 321: Frame Memory 322: Forecasting Department 323: Rate Control Unit 400: Image encoding device 411: Accumulation Buffer 412: Decoding Department 413: Inverse Quantization Department 414: Inverse orthogonal transformation part 415: Calculation Department 416: In-loop filter section 417: Sort Buffer 418: Frame Memory 419: Forecasting Department 800: Computer 801: CPU 802:ROM 803: RAM 804: Busbar 810: Input / Output Interface 811: Input Section 812: Output Section 813: Memory Department 814: Communications Department 815: Drive unit 821: Removable Media

Claims

1. An image processing apparatus comprising: an encoding unit that sets context variables for the first and second binary numbers of a binary sequence of control information regarding secondary conversion, namely secondary conversion control information, and performs arithmetic encoding on each binary number of the prior secondary conversion control information by referring to the previously set context variables.

2. The image processing apparatus as described in claim 1, wherein, The aforementioned secondary conversion control information contains a secondary conversion identifier element that indicates the type of the aforementioned secondary conversion.

3. The image processing apparatus as described in claim 1, wherein, The preceding encoding unit converts the preceding secondary conversion control information into binary to generate a preceding binary sequence, and sets context variables for each binary number in the generated preceding binary sequence.

4. The image processing apparatus as described in claim 1, wherein, It also includes: a coefficient conversion unit, which performs a prior secondary conversion on the primary conversion coefficients of the corresponding block corresponding to the prior secondary conversion control information to generate secondary conversion coefficients.

5. The image processing apparatus as described in claim 4, wherein, It also has a quantization unit, which quantizes the previous secondary conversion coefficients generated by the previous coefficient conversion unit to generate quantized coefficients.

6. An image processing method, wherein context variables are set for the first and second binary numbers of the binary sequence of secondary conversion control information, that is, secondary conversion control information, and each binary number of the aforementioned secondary conversion control information is arithmetically encoded by referring to the set context variables.

7. An image processing apparatus comprising: a decoding unit that performs arithmetic decoding on each binary number of the aforementioned secondary conversion control information by referring to a context variable set for the first binary number and the second binary number of the binary sequence of control information regarding secondary conversion, i.e., secondary conversion control information.

8. The image processing apparatus as described in claim 7, wherein, The aforementioned secondary conversion control information contains a secondary conversion identifier element that indicates the type of the aforementioned secondary conversion.

9. The image processing apparatus as described in claim 7, wherein, The preceding decoding unit performs multi-valued transformation on the preceding binary sequence obtained by arithmetic decoding to generate preceding secondary conversion control information.

10. The image processing apparatus as described in claim 7, wherein, It also has: an inverse coefficient conversion unit, which performs inverse secondary conversion on the secondary conversion coefficients of the corresponding blocks corresponding to the previous secondary conversion control information to generate primary conversion coefficients.

11. The image processing apparatus as described in claim 10, wherein, It also includes: an inverse quantization unit, which inverse quantizes the quantization coefficients to generate the previous secondary conversion coefficients; and a previous inverse coefficient conversion unit, which performs a previous inverse secondary conversion on the previous secondary conversion coefficients generated by the previous inverse quantization unit to generate the previous primary conversion coefficients.

12. An image processing method, which performs arithmetic decoding on each binary number of the aforementioned secondary transformation control information by referring to a context variable set for the first binary number and the second binary number of the binary sequence of secondary transformation control information, that is, control information about secondary transformation.

Images