Image processing apparatus and method

Abstract

The present disclosure pertains to an information processing device and method capable of suppressing an increase in the load of encoding processing / decoding processing. A fixed context variable is set for a first bit of a bit sequence of secondary transformation control information, i.e. Control information related to secondary transformation, and the set fixed context variable is referenced to perform arithmetic encoding of each bit of the secondary transformation control information. The present disclosure can be applied to, for example, an image processing device, an image encoding device, an image decoding device, a transmission device, a reception device, a transmission / reception device, an information processing device, an image capturing device, a reproduction device, an electronic device, an image processing method, an information processing method, etc.

Description

[0001] This application is a divisional application of the application with national application number 202080060661.7, international application date September 4, 2020, national entry date February 25, 2022, and invention name “Image processing device and method”. Technical Field

[0002] The present disclosure relates to an image processing device and method, and more particularly to an image processing device and method capable of suppressing an increase in the load of encoding processing and decoding processing. Background Art

[0003] Conventionally, a coding method for deriving a prediction residual of a moving image and performing coefficient transformation, quantization, and coding has been proposed (e.g., NPL 1). In VVC WD6 described in NPL 1, there is a coding tool for performing a low-frequency secondary transform (LFST) on the transform coefficients after the primary transform to further improve energy compression. The ST identifier st_idx (also referred to as lfnst_idx) can be used as mode information related to the low-frequency secondary transform.

[0004] When encoding the ST identifier lfnst_idx, the ST identifier lfnst_idx is first binarized, and arithmetic coding is performed with reference to the context variable ctx corresponding to each binIdx of the bit sequence (bins) obtained by binarization. For example, according to the tree type treType and the MTS identifier mts_idx, the context variable ctx is assigned to the bit of binIdx=0 in the bit sequence.

[0005] [Citation List]

[0006] [Non-patent literature]

[0007] [NPL1]

[0008] Benjamin Bross, Jianle Chen, Shan Liu, “Versatile Video Coding (Draft 6)”, JVET-O2001-vE, m49908, 15th Meeting of the Joint Video Experts Team (JVET) of ITU-T SG 16WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11: 3-12 July 2019, Gothenburg, Sweden Summary of the invention

[0009] [Technical issues]

[0010] However, since LFSNT is applied only when tu_mts_idx==0 (DCT2xDCT2), the condition of "tu_mts_idx==0" is always true. Therefore, the branch of "tu_mts_idx==0" is redundant, which may increase the load of the encoding process / decoding process.

[0011] The present disclosure has been made in view of such circumstances, and is capable of suppressing an increase in the load of encoding processing / decoding processing.

[0012] [Solution to the problem]

[0013] An image processing device according to one aspect of the present technology includes: a setting unit for setting context variables for a first bit and a second bit of a bit string of secondary transform control information, wherein the secondary transform control information is control information about a secondary transform; and an encoding unit for arithmetically encoding each bit of the secondary transform control information with reference to the context variables set by the setting unit.

[0014] An image processing method according to one aspect of the present technology includes: setting context variables for the first bit and the second bit of a bit string of secondary transform control information, wherein the secondary transform control information is control information about secondary transform, and arithmetically encoding each bit of the secondary transform control information with reference to the set context variables.

[0015] An image encoding device according to one aspect of the present technology includes: a prediction unit configured to generate a predicted image of an input image input to the image encoding device; an orthogonal transform unit configured to perform an orthogonal transform on a prediction residual based on transform information to derive a transform coefficient, wherein the prediction residual is obtained by subtracting the predicted image from the input image; a quantization unit configured to quantize the transform coefficient based on the transform information; and an encoding unit configured to perform variable length encoding on the quantized transform coefficient to generate a bit string.

[0016] An image processing device according to one aspect of the present technology is an image processing device including an encoding unit, which is configured to set a fixed context variable for the first bit of a bit string of secondary transform control information, and to arithmetically encode each bit of the secondary transform control information with reference to the set fixed context variable, wherein the secondary transform control information is control information about the secondary transform.

[0017] An image processing method according to one aspect of the present technology is the following image processing method, which includes: setting a fixed context variable for the first bit of a bit string of secondary transformation control information, and arithmetically encoding each bit of the secondary transformation control information with reference to the set fixed context variable, wherein the secondary transformation control information is control information about the secondary transformation.

[0018] An image processing device on the other hand of the present technology is an image processing device including a decoding unit, which is configured to perform arithmetic decoding with reference to a fixed context variable set for the first bit of a bit string of secondary transform control information, wherein the secondary transform control information is control information about secondary transform.

[0019] An image processing method of another aspect of the present technology is an image processing method including performing arithmetic decoding with reference to a fixed context variable set for a first bit of a bit string of secondary transform control information, the secondary transform control information being control information on secondary transform.

[0020] In an image processing device and method of one aspect of the present technology, a fixed context variable is set for the first bit of a bit string of secondary transform control information, and each bit of the secondary transform control information is arithmetically encoded with reference to the set fixed context variable, and the secondary transform control information is control information about the secondary transform.

[0021] In an image processing device and method in another aspect of the present technology, arithmetic decoding is performed with reference to a fixed context variable set for a first bit of a bit string of secondary transform control information, which is control information on secondary transform. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] [ Figure 1 ] Figure 1 is a diagram showing ST identifiers.

[0023] [ Figure 2 ] Figure 2 is a diagram showing an example of a method of allocating a context variable ctx corresponding to each binIdx of a bit string of an ST identifier.

[0024] [ Figure 3 ] Figure 3 is a block diagram showing an example of main components of an encoding device.

[0025] [ Figure 4 ] Figure 4 is a flowchart showing an example of the flow of encoding processing.

[0026] [ Figure 5 ] Figure 5 is a block diagram showing an example of main components of a decoding device.

[0027] [ Figure 6 ] Figure 6 is a flowchart showing an example of the flow of a decoding process.

[0028] [ Figure 7 ] Figure 7 is a block diagram showing an example of main components of an image encoding device.

[0029] [ Figure 8 ] Figure 8 is a flowchart showing an example of the flow of image encoding processing.

[0030] [ Fig. 9 ] Fig. 9 is a block diagram showing an example of main components of an image decoding device.

[0031] [ Fig.10 ] Fig.10 is a flowchart showing an example of the flow of image decoding processing.

[0032] [ Fig.11 ] Fig.11 is a block diagram illustrating an example of main components of a computer. DETAILED DESCRIPTION

[0033] Hereinafter, a mode for carrying out the present disclosure (hereinafter referred to as an embodiment) will be described. The description will be made in the following order.

[0034] 1. Encoding of ST identifier

[0035] 2. First Embodiment (Encoding Device)

[0036] 3. Second Embodiment (Decoding Device)

[0037] 4. Third Embodiment (Image Decoding Device)

[0038] 5. Fourth Embodiment (Image Coding Device)

[0039] 6. Appendix

[0040] <1. Encoding of ST identifier>

[0041] <Documents supporting technical details and technical terms, etc.>

[0042] The scope disclosed in the present technology includes not only the details described in the embodiments but also the details described in the following non-patent documents and the like known at the time of filing the application, details of other documents mentioned in the following non-patent documents, and the like.

[0043] [NPL1] (described above)

[0044] [NPL2]

[0045] Jianle Chen, Yan Ye, Seung Hwan Kim, “Algorithm description for Versatile Video Coding and Test Model 6 (VTM 6)”, JVET-O2002-v2, m49914, Joint Video Experts Team (JVET) of ITU-T SG 16WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11, 15th Meeting: 3-12 July 2019, Gothenburg, Sweden

[0046] [NPL3]

[0047] ITU-T Recommendation H.264(04 / 2017) “Advanced video coding for generic audiovisual services”, April 2017

[0048] [NPL4]

[0049] ITU-T H.265 (02 / 18) Recommendation “High efficiency video coding”, February 2018

[0050] That is, the details described in the above-mentioned non-patent literature are also the basis for determining the support requirements. For example, even if the quadtree block structure and the quadtree plus binary tree (QTBT) block structure described in the above-mentioned non-patent literature are not directly described in the examples, it is assumed that they are included in the scope of the disclosure of the present technology and meet the support requirements of the claims. In addition, even if technical terms such as parsing, grammar, and semantics are not directly described in the examples, it is assumed that they are included in the scope of the disclosure of the present technology and meet the support requirements of the claims.

[0051] In addition, in this specification, unless otherwise specified, a "block" (not a block indicating a processing unit) used as a partial area or a processing unit of an image (picture) in the description indicates an arbitrary partial area in the picture, and its size, shape, characteristics, etc. are not limited. For example, it can be assumed that a "block" includes any partial area (processing unit), such as a transform block (TB), a transform unit (TU), a prediction block (PB), a prediction unit (PU), a minimum coding unit (SCU), a coding unit (CU), a maximum coding unit (LCU), a coding tree block (CTB), a coding tree unit (CTU), a sub-block, a macroblock, a tile, or a slice described in the above-mentioned non-patent literature.

[0052] In addition, when specifying the size of such a block, not only can the block size be directly specified, but the block size can also be indirectly specified. For example, the block size can be specified using identification information indicating the size. In addition, for example, the block size can be specified by a ratio or difference relative to the size of a reference block (e.g., LCU, SCU, etc.). For example, when transmitting information for specifying the block size as a syntax element or the like, the information for indirectly specifying the size as described above can be used as this information. In this way, the amount of information can be reduced, and thus the coding efficiency can be improved. In addition, specifying the block size also includes specifying the range of the block size (e.g., specifying an allowable block size range, etc.).

[0053] In addition, in this specification, encoding includes not only all the processes of transforming an image into a bitstream, but also a part of the processes. For example, encoding can include not only processes including prediction processing, orthogonal transformation, quantization, arithmetic coding, etc., but also processes collectively referred to as quantization and arithmetic coding, processes including prediction processing, quantization, and arithmetic coding, etc. Similarly, decoding includes not only all the processes of transforming a bitstream into an image, but also a part of the processes. For example, decoding can include not only processes including inverse arithmetic decoding, inverse quantization, inverse orthogonal transformation, prediction processing, etc., but also processes including inverse arithmetic decoding and inverse quantization, processes including inverse arithmetic decoding, inverse quantization, and prediction processing, etc.

[0054] <ST identifier>

[0055] In VVC WD6 described in NPL1, there is a coding tool for performing low-frequency second transformation (LFST) on the transformed coefficients after the first transformation to further improve energy compression. As the mode information (control information about the second transformation) regarding this low-frequency second transformation, there is an ST identifier st_idx (also referred to as lfnst_idx) indicating the type of the second transformation. The coding of this ST identifier is performed through the following process.

[0056] First, the ST identifier lfnst_idx is binarized to obtain a bitstring bins. The ST identifier lfnst_idx takes Figure 1 the values shown in Table A, and each value indicates the transformation type shown in the table. In addition, a bitstring (binarization) as shown in the table is obtained according to the binarization of the ST identifier lfnst_idx.

[0057] The ST identifier lfnst_idx is binarized by a truncated unary (TU) code to obtain a bitstring bins. As shown in Table B in Figure 1 this, this TU code is equivalent to a truncated Rice (TR) code with a Rice parameter cRiceParam = 0.

[0058] Next, arithmetic coding is performed with reference to the context variable ctx corresponding to each binIdx of the obtained bit string bins. The index identifying the context variable ctx is also referred to as ctxInc (or ctxIdx).

[0059] Specifically, according to the tree type treType and the MTS identifier mts_idx, the context variable ctx is assigned to the bit binIdx=0 in the bit string, such as Figure 1 As shown in Table C of . In the example of this table, ctxInc corresponding to binIdx=0 is derived as the value ctxInc=(mts_idx==0&&treeType!=SINGLE_TREE). Bypass coding is applied to binIdx=1 of the bit string bins.

[0060] Each bit in the bit string bins of the ST identifier lfnst_idx can also be understood as a flag corresponding to a transformation type. In this case, the value of the bit binIdx=0 corresponds to a flag indicating whether a secondary transformation can be performed (0: yes, 1: no), and the value of the bit binIdx=1 corresponds to a flag indicating whether a first secondary transformation can be performed (0: yes, 1: no).

[0061] At the time of decoding, arithmetic decoding is performed on each bit element with reference to the context variable ctx set as described above. Then, multi-value processing is performed on the bit string to obtain the ST identifier lfnst_idx.

[0062] However, since LFSNT is only applied when tu_mts_idx==0 (DCT2xDCT2), Figure 1 The condition of "tu_mts_idx==0" in the table shown in C is always true. Therefore, the branch of "tu_mts_idx==0" is redundant, which may increase the load of the encoding process / decoding process. In order to reduce the implementation cost, it is necessary to reduce the number of context variables and simplify the derivation of context variables while maintaining the encoding efficiency.

[0063] <Fixed context dispatch>

[0064] Therefore, a fixed context variable is set for the first bit of the bit string of the secondary transform control information, which is the control information about the secondary transform.

[0065] For example, in image processing (specifically image coding), a fixed context variable is set for the first bit of a bit string of secondary transform control information, which is control information about secondary transform, and arithmetic coding is performed on each bit of the secondary transform control information with reference to the set fixed context variable.

[0066] For example, an image processing device (specifically, an image encoding device that encodes an image) may include an encoding unit that sets a fixed context variable for the first bit of a bit string of secondary transform control information, where the secondary transform control information is control information about the secondary transform, and performs arithmetic encoding on each bit of the secondary transform control information with reference to the set fixed context variable.

[0067] In this way, the derivation of context variables of ST identifiers can be simplified while maintaining coding efficiency. In addition, the memory size for maintaining context variables of ST identifiers can be reduced while maintaining coding efficiency. In other words, the increase in the load of the coding process can be suppressed.

[0068] Furthermore, for example, in image processing (decoding of encoded data of an image), arithmetic decoding is performed with reference to a fixed context variable set for the first bit of a bit string of secondary transform control information, which is control information about secondary transform.

[0069] For example, an image processing device (specifically, an image decoding device that decodes encoded data of an image) may include a decoding unit that performs arithmetic decoding with reference to a fixed context variable set for the first bit of a bit string of secondary transform control information, which is control information about secondary transform.

[0070] In this way, the derivation of the context variable of the ST identifier can be simplified while maintaining the coding efficiency. In addition, the memory size for maintaining the context variable of the ST identifier can be reduced while maintaining the coding efficiency. In other words, the increase in the load of the decoding process can be suppressed.

[0071] For example, from Figure 2 As shown in the second row from the top of the table shown in A (Method 1), "0" can be assigned as the context variable ctx to the bit (first bit) of binIdx=0 in the bit string. That is, "0" can be set to the context variable ctx of the first bit of the bit string of the ST identifier, and arithmetic coding can be performed with reference to this value. In other words, arithmetic decoding can be performed with reference to "0" of the context variable ctx set to the first bit of the bit string of the ST identifier. In this way, in the derivation of the context variable in the first bit of the bit string of the ST identifier, the branch processing regarding the selection of the context variable according to treeType (and tu_mts_idx==0) can be omitted. Therefore, the increase in the load of the encoding process / decoding process can be suppressed. In addition, as Figure 2 As shown in the table shown in B, method 1 can reduce the number of contexts compared to the case of VTM. Therefore, the memory size of the context variable for holding the ST identifier can be reduced.

[0072] In addition, if Figure 2 As shown in the table (method 1) shown in A of , a bypass flag may be assigned to the bit (second bit) of binIdx=1 in the bit string. That is, a bypass flag (bypass) may be set for the first bit of the bit string of the ST identifier and arithmetic encoding may be performed based on the bypass flag. In other words, arithmetic decoding may be performed based on the bypass flag set for the first bit of the bit string of the ST identifier.

[0073] In addition, if Figure 2 As shown in the third row from the top of the table shown in A (Method 2), for example, the common context variable ctx can be assigned to the bit of binIdx=0 (the first bit) and the bit of binIdx=1 (the second bit). Figure 2 In the case of the table shown in A of , "0" is assigned to both the bit where binIdx=0 (the first bit) and the bit where binIdx=1 (the second bit) in the bit string. That is, a common context variable ctx (e.g., "0") can be set for the first bit and the second bit of the bit string of the ST identifier and arithmetic encoding of each bit can be performed with reference to the value. In other words, arithmetic decoding of each bit can be performed with reference to the common context variable ctx (e.g., "0") set for the first bit and the second bit of the bit string of the ST identifier. In this way, Figure 2 As shown in the table shown in B, method 2 can reduce the number of bypass-coded bits while maintaining the number of contexts, compared with the case of method 1. Therefore, the code amount of the ST identifier can be reduced.

[0074] <2. First Embodiment>

[0075] <Encoding device>

[0076] The present technology described in <1. Encoding of ST identifier> can be applied to any device. Application examples of the present technology will be described below. Figure 3 : is a block diagram showing an example of the configuration of an encoding device as one aspect of an information processing device to which the present technology is applied. Figure 3 The encoding device 100 shown in FIG. 1 is a device that encodes the ST identifier lfnst_idx. The encoding device 100 performs encoding using, for example, context-based adaptive binary arithmetic coding (CABAC).

[0077] at the same time, Figure 3 The main parts such as processing units and data flows are shown, but the processing units and data flows are not limited to Figure 3 That is, in Figure 3 The processing units not shown as blocks and Figure 3Processing and data flows not shown as arrows in the figure may exist in the encoding device 100.

[0078] like Figure 3 As shown in , the encoding device 100 includes a binarization unit 121, a selection unit 122, a context model 123, an arithmetic encoding unit 124, an arithmetic encoding unit 125, and a selection unit 126.

[0079] The binarization unit 121 obtains the syntax element value supplied to the encoding device 100 and binarizes it by a method defined for each syntax element to generate a binarized bit stream. The binarization unit 121 supplies the binarized bit stream to the selection unit 122.

[0080] The selection unit 122 acquires the binarized bit stream and the flag information isBypass provided from the binarization unit 121. The selection unit 122 selects the supply destination of the binarized bit stream based on the value of isBypass. For example, when isBypass=0, the selection unit 122 determines that the normal mode has been set and provides the binarized bit stream to the context model 123. In addition, when isBypass=1, the selection unit 122 determines that the bypass mode has been set and provides the binarized bit stream to the arithmetic coding unit 125.

[0081] The context model 123 dynamically switches the context model to be applied according to the encoding target and the surrounding situation. For example, the context model 123 holds the context variable ctx, and when the binarized bit stream is acquired from the selection unit 122, the context variable ctx corresponding to each bit position (binIdx) of the bit string defined for each syntax element is read. The context model 123 provides the binarized bit stream and the read context variable ctx to the arithmetic coding unit 124.

[0082] When the arithmetic coding unit 124 acquires the binarized bit stream and the context variable ctx supplied from the context model 123, the arithmetic coding unit 124 performs arithmetic coding (performs context coding) on ​​the value of the bit in binIdx of the binarized bit stream with reference to the probability state of the context variable ctx in the normal mode of CABAC. The arithmetic coding unit 124 supplies the coded data generated by the context coding to the selection unit 126. In addition, the arithmetic coding unit 124 supplies the context variable ctx after the context coding process to the context model 123 so that the context variable ctx is held therein.

[0083] The arithmetic coding unit 125 arithmetically codes (performs bypass coding) the binarized bit stream supplied from the selection unit 122 in the bypass mode of CABAC. The arithmetic coding unit 125 supplies the coded data generated by the bypass coding to the selection unit 126.

[0084] The selection unit 126 acquires the flag information isBypass and selects the coded data to be output based on the value of isBypass. For example, when isBypass=0, the selection unit 126 determines that the normal mode has been set, acquires the coded data provided from the arithmetic coding unit 124, and outputs the coded data to the outside of the CABAC 102 (coding device 100). In addition, when isBypass=1, the selection unit 126 determines that the bypass mode has been set, acquires the coded data provided from the arithmetic coding unit 125, and outputs the coded data to the outside of the coding device 100.

[0085] Meanwhile, the processing units (binarization unit 121 to selection unit 126) have any configuration. For example, each processing unit may be configured as a logic circuit for implementing the above-mentioned processing. In addition, each processing unit may include, for example, a central processing unit (CPU), a read-only memory (ROM), a random access memory (RAM), etc., and execute a program using these components to implement the above-mentioned processing. It goes without saying that each processing unit may have the two configurations mentioned above, implementing part of the above-mentioned processing according to the logic circuit, and implementing the other part of the processing by executing the program. The processing units may have independent configurations, for example, some processing units may implement part of the above-mentioned processing according to the logic circuit, some other processing units may implement the above-mentioned processing by executing the program, and some other processing units may implement the above-mentioned processing according to both the logic circuit and the execution of the program.

[0086] In the encoding device 100, the present technology described in <1. Encoding of ST identifier> is applied. That is, the binarization unit 121 binarizes the secondary transform control information (ST identifier lfnst_idx) to generate a bit string. The selection unit 122 sets a fixed context variable ctx (e.g., "0") for the first bit of the bit string. The arithmetic encoding unit 124 performs arithmetic encoding in the normal mode of CABAC with reference to the fixed context variable ctx (e.g., "0").

[0087] In this way, the encoding device 100 can simplify the derivation of the context variable of the ST identifier while maintaining the encoding efficiency. In addition, the encoding device 100 can reduce the memory size for maintaining the context variable of the ST identifier while maintaining the encoding efficiency. That is, the encoding device 100 can suppress the increase in the load of the encoding process.

[0088] At this time, the selection unit 122 may set a bypass flag for the second bit of the bit string (method 1). In that case, the arithmetic coding unit 125 performs arithmetic coding on the second bit in the bypass mode of CABAC.

[0089] In addition, the selection unit 122 may set the context variable ctx (e.g., "0") shared by the first bit for the second bit of the bit string (method 2). In that case, the arithmetic coding unit 124 performs arithmetic coding in the normal mode of CABAC with reference to the fixed context variable ctx (e.g., "0").

[0090] <Encoding Process Flow>

[0091] Next, we will refer to Figure 4 The flowchart describes an example of the flow of the encoding process performed by the encoding device 100.

[0092] When the encoding process starts, the binarization unit 121 of the encoding device 100 receives a syntax element value (syncVal) to be processed in step S101 .

[0093] In step S102 , the binarization unit 121 performs the binarization process defined for each syntax element to derive a bit string (synBins) of a syntax element value (syncVal).

[0094] In step S103 , the selection unit 122 reads the context variable ctx corresponding to each bit position (binIdx) of the bit string defined for each syntax element and the flag isBypass indicating whether the bypass mode has been set.

[0095] In step S104, the selection unit 122 determines whether the bypass mode is set. If isBypass=0 and it is determined that the normal mode has been set, the process proceeds to step S105.

[0096] In step S105, the arithmetic coding unit 124 performs context coding. That is, the arithmetic coding unit 124 encodes the value of the bit at the bit position (binIdx) of the bit string (synBins) in the normal mode of CABAC with reference to the probability state of the context variable ctx. When the processing of S105 ends, the processing proceeds to step S107.

[0097] If isBypass=1 in step S104 and it is determined that the bypass mode has been set, the process proceeds to step S106 .

[0098] In step S106, the arithmetic coding unit 125 performs bypass coding. That is, the arithmetic coding unit 125 encodes the value of the bit at the bit position (binIdx) of the bit string (synBins) in the bypass mode of CABAC. When the process of S106 ends, the process proceeds to step S107.

[0099] In step S107 , the selection unit 126 determines whether a predetermined interruption condition A is satisfied. The interruption condition A is defined based on the value of the bit string from binIdx=0 to binIdx=k (current position k of binIdx) and the binarization method of each syntax element.

[0100] If it is determined that the interruption condition A is not satisfied, the process returns to step S103, and the subsequent process is performed on the next bit position (binIdx). That is, the processes of steps S103 to S107 are performed for each bit position (binIdx).

[0101] Then, if it is determined in step S107 that the interruption condition A is satisfied, the CABAC process ends.

[0102] In such an encoding process, the present technology described in <1. Encoding of ST identifier> is applied. That is, the binarization unit 121 binarizes the secondary transform control information (ST identifier lfnst_idx) in step S102 to generate a bit string. In step S103, the selection unit 122 sets a fixed context variable ctx (e.g., "0") for the first bit of the bit string. In step S105, the arithmetic coding unit 124 performs arithmetic coding in the normal mode of CABAC with reference to the fixed context variable ctx (e.g., "0").

[0103] In this way, the encoding device 100 can simplify the derivation of the context variable of the ST identifier while maintaining the encoding efficiency. In addition, the encoding device 100 can reduce the memory size for maintaining the context variable of the ST identifier while maintaining the encoding efficiency. That is, the encoding device 100 can suppress the increase in the load of the encoding process.

[0104] Furthermore, in step S103, the selection unit 122 may set a bypass flag for the second bit of the bit string (method 1). In that case, the arithmetic coding unit 125 arithmetically codes the second bit in the bypass mode of CABAC in step S106.

[0105] In addition, in step S103, the selection unit 122 may set the context variable ctx (e.g., "0") shared by the first bit for the second bit of the bit string (method 2). In that case, the arithmetic coding unit 124 performs arithmetic coding in the normal mode of CABAC with reference to the fixed context variable ctx (e.g., "0") in step S105.

[0106] <3. Second Embodiment>

[0107] <Decoding device>

[0108] Figure 5 : is a block diagram showing an example of the configuration of a decoding device as an aspect to which the present technology is applied. Figure 5 The decoding device 200 shown in is a device that decodes the encoded data of the ST identifier lfnst_idx. The decoding device 200 performs decoding using a decoding method (for example, CABAC) corresponding to the encoding method of the encoding device 100. For example, the decoding device 200 decodes the encoded data of the ST identifier lfnst_idx generated by the encoding device 100.

[0109] at the same time, Figure 5 The main parts such as processing units and data flows are shown, but the processing units and data flows are not limited to Figure 5 That is, Figure 5 Processing units not shown as blocks and Figure 5 Processing and data flows not shown as arrows in the figure may exist in the decoding device 200.

[0110] like Figure 5 As shown in , the decoding device 200 includes a selection unit 221, a context model 222, an arithmetic decoding unit 223, an arithmetic decoding unit 224, a selection unit 225, and a multi-value conversion unit 226.

[0111] The selection unit 221 acquires the coded data and the flag information isBypass input to the decoding device 200. The selection unit 221 selects the supply destination of the coded data based on the value of isBypass. For example, when isBypass=0, the selection unit 221 determines that the normal mode has been set and supplies the coded data to the context model 222. In addition, when isBypass=1, the selection unit 221 determines that the bypass mode has been set and supplies the binarized bit stream to the arithmetic decoding unit 224.

[0112] The context model 222 dynamically switches the context model to be applied according to the decoding target and the surrounding situation. For example, the context model 222 holds the context variable ctx, and when the encoded data is acquired from the selection unit 221, the context variable ctx corresponding to each bit position (binIdx) of the bit string defined for each syntax element is read. The context model 222 provides the encoded data and the read context variable ctx to the arithmetic decoding unit 223.

[0113] When the arithmetic decoding unit 223 acquires the encoded data and the context variable ctx provided from the context model 222, the arithmetic decoding unit 223 arithmetically decodes the value of the bit at binIdx of the binarized bit stream in the normal mode of CABAC with reference to the probability state of the context variable ctx (performs context decoding). The arithmetic decoding unit 223 provides the binarized bit stream generated by the context decoding to the selection unit 225. In addition, the arithmetic decoding unit 223 provides the context variable ctx after the context decoding process to the context model 222 so that the context variable ctx is maintained therein.

[0114] The arithmetic decoding unit 224 arithmetically decodes (performs bypass decoding) the encoded data supplied from the selection unit 221 in the bypass mode of CABAC. The arithmetic decoding unit 224 supplies the binarized bit stream generated by the bypass decoding to the selection unit 225.

[0115] The selection unit 225 acquires the flag information isBypass and selects a binarized bit stream to be supplied to the multi-value conversion unit 226 based on the value of isBypass. For example, when isBypass=0, the selection unit 225 determines that the normal mode has been set, acquires the binarized bit stream supplied from the arithmetic decoding unit 223, and supplies the binarized bit stream to the multi-value conversion unit 226. In addition, when isBypass=1, the selection unit 225 determines that the bypass mode has been set, acquires the binarized bit stream supplied from the arithmetic decoding unit 224, and supplies the binarized bit stream to the multi-value conversion unit 226.

[0116] The multi-value conversion unit 226 obtains the binarized bit stream supplied from the selection unit 225 and converts the binarized bit stream into a multi-value bit stream by a method defined for each syntax element to generate a syntax element value. The multi-value conversion unit 226 outputs the syntax element value to the outside of the decoding device 200.

[0117] Meanwhile, the processing units (decoding unit 221 to multi-value conversion unit 226) have any configuration. For example, each processing unit may be configured as a logic circuit for implementing the above-mentioned processing. In addition, each processing unit may include, for example, a CPU, a ROM, a RAM, etc. and use these components to execute a program to implement the above-mentioned processing. It goes without saying that each processing unit may have the two configurations mentioned above, implementing part of the above-mentioned processing according to the logic circuit, and implementing the other part of the processing by executing the program. The processing units may have independent configurations: for example, some processing units may implement part of the above-mentioned processing according to the logic circuit, some other processing units may implement the above-mentioned processing by executing the program, and some other processing units may implement the above-mentioned processing according to both the execution of the logic circuit and the program.

[0118] In the decoding device 200, the present technology described in <1. Encoding of ST identifier> is applied. That is, the arithmetic decoding unit 223 performs arithmetic decoding in the normal mode of CABAC with reference to a fixed context variable ctx (for example, "0") set for the first bit of the bit string of the secondary transform control information (ST identifier lfnst_idx).

[0119] In this way, the decoding device 200 can simplify the derivation of the context variable of the ST identifier while maintaining the coding efficiency. In addition, the decoding device 200 can reduce the memory size for maintaining the context variable of the ST identifier while maintaining the coding efficiency. That is, the decoding device 200 can suppress the increase in the load of the decoding process.

[0120] At this time, the arithmetic decoding unit 224 may perform arithmetic decoding on the second bit of the bit string in the bypass mode of CABAC based on the bypass flag set for the second bit.

[0121] Furthermore, the arithmetic decoding unit 223 may perform arithmetic decoding in the normal mode of CABAC with reference to a context variable ctx (eg, “0”) shared by the first bit set for the second bit of the bit string.

[0122] <Decoding Process Flow>

[0123] Next, we will refer to Figure 6 The flowchart of describes the decoding process performed by the decoding device 200.

[0124] When the decoding process starts, the selection unit 221 of the decoding device 200 reads in step S201 a context variable ctx corresponding to each bit position (binIdx) of a bit string defined for each syntax element and a flag isBypass indicating whether the bypass mode has been set.

[0125] In step S202, the selection unit 221 determines whether the bypass mode is set. If isBypass=0 and it is determined that the normal mode has been set, the process proceeds to step S203.

[0126] In step S203, the arithmetic decoding unit 223 performs context decoding. That is, the arithmetic decoding unit 223 decodes the encoded data in the normal mode of CABAC with reference to the probability state of the context variable ctx to generate the value of the bit at the bit position (binIdx) of the bit string (synBins). When the processing of S203 ends, the processing proceeds to step S205.

[0127] If isBypass=1 in step S202 and it is determined that the bypass mode has been set, the process proceeds to step S204.

[0128] In step S204, the arithmetic decoding unit 224 performs bypass decoding. That is, the arithmetic decoding unit 224 decodes the encoded data in the bypass mode of CABAC to generate the value of the bit at the bit position (binIdx) of the bit string (synBins). When the process of S204 ends, the process proceeds to step S205.

[0129] In step S205 , the selection unit 225 determines whether a predetermined interruption condition A is satisfied. The interruption condition A is defined based on the value of the bit string from binIdx=0 to binIdx=k (current position k of binIdx) and the binarization method of each syntax element.

[0130] If it is determined that the interrupt condition A is not satisfied, the process returns to step S201 and subsequent processing is performed to generate the value of the bit at the next bit position (binIdx). That is, the processing of steps S201 to S205 is performed for each bit position (binIdx).

[0131] If it is determined in step S205 that the interruption condition A is satisfied, the process proceeds to step S206 .

[0132] In step S206 , the multi-value conversion unit 226 derives a syntax element value (syncVal) from the bit string (synBins) by the multi-value conversion process defined for each syntax element.

[0133] In step S207 , the multi-value converting unit 226 outputs the derived syntax element value (syncVal) to the outside of the decoding device 200 .

[0134] In this decoding process, the present technology described in <1. Encoding of ST identifier> is applied. That is, the arithmetic decoding unit 223 performs arithmetic decoding in the normal mode of CABAC with reference to the fixed context variable ctx (e.g., "0") set for the first bit of the bit string of the secondary transform control information (ST identifier lfnst_idx) in step S203.

[0135] In this way, the decoding device 200 can simplify the derivation of the context variable of the ST identifier while maintaining the coding efficiency. In addition, the decoding device 200 can reduce the memory size for maintaining the context variable of the ST identifier while maintaining the coding efficiency. That is, the decoding device 200 can suppress the increase in the load of the decoding process.

[0136] Furthermore, the arithmetic decoding unit 224 may perform arithmetic decoding on the second bit of the bit string in the bypass mode of CABAC based on the bypass flag set for the second bit in step S204 (method 1).

[0137] Furthermore, the arithmetic decoding unit 223 may perform arithmetic decoding in the normal mode of CABAC with reference to the context variable ctx (eg, “0”) shared by the first bit set for the second bit of the bit string in step S203 (method two).

[0138] <4. Third Embodiment>

[0139] <Image Coding Device>

[0140] Figure 7 : is a block diagram showing an example of the configuration of an image encoding device as one aspect of an information processing device to which the present technology is applied. Figure 7 The image encoding device 300 shown in is a device that encodes image data of a moving image. For example, the image encoding device 300 can encode the image data of a moving image by an encoding method described in any of the non-patent documents mentioned above.

[0141] although Figure 7 The main components such as processing units (blocks) and data flows are shown, but the processing units and data flows are not limited to Figure 7 That is, Figure 7 Processing units not shown as blocks and Figure 7 Processing and data flows not shown as arrows in the figure may exist in the image encoding device 300.

[0142] like Figure 7As shown in the figure, the image encoding device 300 includes a control unit 301, a reordering buffer 311, a computing unit 312, an orthogonal transform unit 313, a quantization unit 314, an encoding unit 315, an accumulation buffer 316, an inverse quantization unit 317, an inverse orthogonal transform unit 318, a computing unit 319, a loop filter unit 320, a frame memory 321, a prediction unit 322 and a rate control unit 323.

[0143] <Control Unit>

[0144] Based on the block size of the external or predetermined processing unit, the control unit 301 divides the moving image data held by the reorder buffer 311 into blocks (CU, PU, ​​transform block, etc.) in the processing unit. In addition, the control unit 301 determines the encoding parameters (header information Hinfo, prediction mode information Pinfo, transform information Tinfo, filter information Finfo, etc.) to be provided to each block based on rate-distortion optimization (RDO), for example.

[0145] The details of such encoding parameters will be described later. When the control unit 301 determines the encoding parameters as described above, the control unit 301 provides them to each block. Specifically, it is described as follows.

[0146] Header information Hinfo is provided for each block.

[0147] The prediction mode information Pinfo is provided to the encoding unit 315 and the prediction unit 322 .

[0148] The transformation information Tinfo is provided to the encoding unit 315 , the orthogonal transformation unit 313 , the quantization unit 314 , the inverse quantization unit 317 , and the inverse orthogonal transformation unit 318 .

[0149] The filter information Finfo is provided to the loop filter unit 320 .

[0150] <Reorder Buffer>

[0151] Each field (input image) of the moving image data is input to the image encoding device 300 in the order in which it is reproduced (displayed). The reordering buffer 311 acquires and holds (stores) the input image in the order in which it is reproduced (displayed). The reordering buffer 311 reorders the input images in the encoding order (decoding order) or divides the input images into blocks in the processing unit based on the control of the control unit 301. The reordering buffer 311 supplies each processed input image to the calculation unit 312. In addition, the reordering buffer 311 supplies each input image (original image) to the prediction unit 322 and the loop filter unit 320.

[0152] <Computational Unit>

[0153] The calculation unit 312 receives as input an image I corresponding to a block in the processing unit and a predicted image P provided from the prediction unit 322, subtracts the predicted image P from the image I according to the following formula to derive a predicted residual D, and provides the predicted residual D to the orthogonal transformation unit 313.

[0154] D=IP

[0155] <Orthogonal Transformation Unit>

[0156] The orthogonal transform unit 313 receives the prediction residual D provided from the calculation unit 312 and the transform information Tinfo provided from the control unit 301 as input, and performs an orthogonal transform on the prediction residual D based on the transform information Tinfo to derive a transform coefficient Coeff. For example, the orthogonal transform unit 313 performs a primary transform on the prediction residual D to generate a primary transform coefficient, and performs a secondary transform on the primary transform coefficient based on the ST identifier to generate a secondary transform coefficient. The orthogonal transform unit 313 provides the obtained secondary transform coefficient as the transform coefficient Coeff to the quantization unit 314.

[0157] <Quantization Unit>

[0158] The quantization unit 314 receives the transform coefficient Coeff provided from the orthogonal transform unit 313 and the transform information Tinfo provided from the control unit 301 as input, and scales (quantizes) the transform coefficient Coeff based on the transform information Tinfo. The quantization rate is controlled by the rate control unit 323. The quantization unit 314 provides the quantized transform coefficient obtained by this quantization, that is, the quantized transform coefficient level level, to the encoding unit 315 and the inverse quantization unit 317.

[0159] <Coding unit>

[0160] The encoding unit 315 receives as input the quantized transform coefficient level scale supplied from the quantization unit 314, various encoding parameters (header information Hinfo, prediction mode information Pinfo, transform information Tinfo, filter information Finfo, etc.) supplied from the control unit 301, information related to the filter such as the filter coefficient supplied from the loop filter unit 320, and information related to the optimal prediction mode supplied from the prediction unit 322. The encoding unit 315 performs variable length encoding (e.g., arithmetic encoding) on ​​the quantized transform coefficient level scale to generate a bit string (encoded data).

[0161] In addition, the encoding unit 315 derives residual information Rinfo from the quantized transform coefficient level, and encodes the residual information Rinfo to generate a bitstream.

[0162] In addition, the encoding unit 315 includes information about the filter provided from the loop filter unit 320 in the filter information Finfo and includes information about the optimal prediction mode provided from the prediction unit 322 in the prediction mode information Pinfo. Then, the encoding unit 315 encodes the various encoding parameters mentioned above (header information Hinfo, prediction mode information Pinfo, conversion information Tinfo, filter information Finfo, etc.) to generate a bitstream.

[0163] Furthermore, the encoding unit 315 multiplexes the bit streams of the various types of information generated as described above to generate encoded data. The encoding unit 315 supplies the encoded data to the accumulation buffer 316.

[0164] <Accumulation Buffer>

[0165] The accumulation buffer 316 temporarily holds the encoded data obtained in the encoding unit 315. The accumulation buffer 316 outputs the held encoded data as, for example, a bit stream to the outside of the image encoding device 300 at a predetermined timing. For example, the encoded data is transmitted to the decoding side via an arbitrary recording medium, an arbitrary transmission medium, an arbitrary information processing device, etc. That is, the accumulation buffer 316 is also a transmission unit that transmits the encoded data (bit stream).

[0166] <Inverse Quantization Unit>

[0167] The inverse quantization unit 317 performs processing related to inverse quantization. For example, the inverse quantization unit 317 receives the quantized transform coefficient level level provided from the quantization unit 314 and the transform information Tinfo provided from the control unit 301 as input, and scales (inverse quantizes) the value of the quantized transform coefficient level level based on the transform information Tinfo. This inverse quantization is an inverse process of the quantization performed in the quantization unit 314. The inverse quantization unit 317 provides the transform coefficient Coeff_IQ obtained by this inverse quantization to the inverse orthogonal transform unit 318.

[0168] <Inverse Orthogonal Transformation Unit>

[0169] The inverse orthogonal transform unit 318 performs processing related to inverse orthogonal transform. For example, the inverse orthogonal transform unit 318 receives the transform coefficient Coeff_IQ provided from the inverse quantization unit 317 and the transform information Tinfo provided from the control unit 301 as input, and performs inverse orthogonal transform on the transform coefficient Coeff_IQ based on the transform information Tinfo to derive the prediction residual D'. The inverse orthogonal transform is an inverse process of the orthogonal transform performed by the orthogonal transform unit 313. That is, the inverse orthogonal transform unit 318 can perform adaptive inverse orthogonal transform (AMT) to adaptively select the type (transform coefficient) of inverse orthogonal transform.

[0170] The inverse orthogonal transform unit 318 supplies the prediction residual D' obtained by this inverse orthogonal transform to the calculation unit 319. Since the inverse orthogonal transform unit 318 is the same as the inverse orthogonal transform unit (to be described later) on the decoding side, the description given on the decoding side (to be described later) can be applied to the inverse orthogonal transform unit 318.

[0171] <Computational Unit>

[0172] The calculation unit 319 receives the prediction residual D' provided from the inverse orthogonal transform unit 318 and the prediction image P provided from the prediction unit 322. The calculation unit 319 adds the prediction residual D' to the prediction image P corresponding to the prediction residual D' to derive a local decoded image Rlocal. The calculation unit 319 provides the derived local decoded image Rlocal to the loop filter unit 320 and the frame memory 321.

[0173] <Loop filter unit>

[0174] The loop filter unit 320 performs processing related to the loop filtering process. For example, the loop filter unit 320 receives the local decoded image Rlocal provided from the calculation unit 319, the filter information Finfo provided from the control unit 301, and the input image (original image) provided from the reordering buffer 311 as input. At the same time, the information input to the loop filter unit 320 is arbitrary, and information other than this information can be input. For example, information such as prediction mode, motion information, code amount target value, quantization parameter QP, picture type, and block (CU, CTU, etc.) can be input to the loop filter unit 320 as needed.

[0175] The loop filter unit 320 appropriately filters the local decoded image Rlocal based on the filter information Finfo. The loop filter unit 320 also performs filtering processing using an input image (original image) and other input information as necessary.

[0176] For example, as described in NPL 11, the loop filter unit 320 applies four loop filters in this order: a bidirectional filter, a deblocking filter (DBF), an adaptive offset filter (sample adaptive offset (SAO)), and an adaptive loop filter (ALF). Which filter is applied and in what order is arbitrary and can be appropriately selected.

[0177] Of course, the filtering process performed by the loop filter unit 320 is arbitrary and is not limited to the above example. For example, the loop filter unit 320 may apply a Wiener filter or the like.

[0178] The loop filter unit 320 supplies the filtered local decoded image Rlocal to the frame memory 321. When information on the filter such as a filter coefficient is transmitted to the decoding side, the loop filter unit 320 supplies the information on the filter to the encoding unit 315.

[0179] <Frame Memory>

[0180] The frame memory 321 performs processing related to the storage of data related to the image. For example, the frame memory 321 receives the local decoded image Rlocal provided from the calculation unit 319 and the filtered local decoded image Rlocal provided from the loop filter unit 320 as input and holds (stores) them. In addition, the frame memory 321 reconstructs the decoded image R for each picture unit using the local decoded image Rlocal, and holds the decoded image R (stores it in a buffer in the frame memory 321). In response to a request from the prediction unit 322, the frame memory 321 provides the decoded image R (or a part thereof) to the prediction unit 322.

[0181] <Prediction Unit>

[0182] The prediction unit 322 performs processing on the generation of a predicted image. For example, the prediction unit 322 receives as input the prediction mode information Pinfo provided from the control unit 301, the input image (original image) provided from the reordering buffer 311, and the decoded image R (or a portion thereof) read from the frame memory 321. The prediction unit 322 uses the prediction mode information Pinfo and the input image (original image) to perform prediction processing such as inter-frame prediction and intra-frame prediction, performs prediction using the decoded image R as a reference image, and performs motion compensation processing based on the prediction result to generate a predicted image P. The prediction unit 322 provides the generated predicted image P to the calculation unit 312 and the calculation unit 319. In addition, the prediction unit 322 provides information on the prediction mode selected by the above-mentioned processing, that is, the optimal prediction mode, to the encoding unit 315 as needed.

[0183] <Rate Control Unit>

[0184] The rate control unit 323 performs processing related to rate control. For example, the rate control unit 323 controls the rate of the quantization operation of the quantization unit 314 based on the code amount of the encoded data accumulated in the accumulation buffer 316 so that overflow or underflow does not occur.

[0185] Meanwhile, the processing units (control unit 301 and reordering buffer 311 to rate control unit 323) have an arbitrary configuration. For example, each processing unit may be configured as a logic circuit for implementing the above-mentioned processing. In addition, each processing unit may include, for example, a CPU, a ROM, a RAM, etc. and execute a program using these components to implement the above-mentioned processing. Of course, each processing unit may have the two configurations mentioned above, implementing part of the above-mentioned processing according to the logic circuit, and implementing the other part of the processing by executing the program. The processing units may have independent configurations, for example, some processing units may implement part of the above-mentioned processing according to the logic circuit, some other processing units may implement the above-mentioned processing by executing the program, and some other processing units may implement the above-mentioned processing according to both the logic circuit and the execution of the program.

[0186] In the image encoding device 300 having the above configuration, the present technology described in <1. Encoding of ST identifier> can be applied to the encoding unit 315. That is, the encoding unit 315 has Figure 3 The encoding device 100 shown in FIG. 1 has the same configuration as that of FIG. 1 and performs the same processing.

[0187] In this way, the image encoding device 300 can simplify the derivation of the context variable of the ST identifier while maintaining the encoding efficiency. In addition, the image encoding device 300 can reduce the memory size for maintaining the context variable of the ST identifier while maintaining the encoding efficiency. That is, the image encoding device 300 can suppress the increase in the load of the encoding process.

[0188] <Flow of Image Coding Process>

[0189] Next, we will refer to Figure 8 The flowchart describes an example of the flow of an image encoding process performed by the image encoding device 300 having the above-mentioned configuration.

[0190] When the image encoding process starts, in step S301 , the reordering buffer 311 is controlled by the control unit 301 to reorder the order of frames of input moving image data from the display order to the encoding order.

[0191] In step S302 , the control unit 301 sets a processing unit (performs block division) on the input image held in the rearrangement buffer 311 .

[0192] In step S303 , the control unit 301 determines (sets) encoding parameters of the input image held in the rearrangement buffer 311 .

[0193] In step S304, the prediction unit 322 performs prediction processing to generate a predicted image in the optimal prediction mode, etc. For example, in the prediction processing, the prediction unit 322 performs intra prediction to generate a predicted image in the optimal intra prediction mode, etc., performs inter prediction to generate a predicted image in the optimal inter prediction mode, etc., and selects the optimal prediction mode based on the cost function value, etc.

[0194] In step S305, the calculation unit 312 calculates the difference between the input image and the predicted image in the optimal mode selected by the prediction process in step S304. That is, the calculation unit 312 generates a prediction residual D between the input image and the predicted image. The data amount of the prediction residual D obtained in this way is reduced compared to the original image data. Therefore, the data amount can be compressed compared to the case where the image is encoded as it is.

[0195] In step S306 , the orthogonal transform unit 313 performs an orthogonal transform process on the prediction residual D generated by the process of step S305 to derive a transform coefficient Coeff.

[0196] In step S307 , the quantization unit 314 quantizes the transform coefficient Coeff obtained by the process of step S306 by using the quantization parameter calculated by the control unit 301 or the like, and derives the transform coefficient level after quantization.

[0197] In step S308 , the inverse quantization unit 317 inversely quantizes the quantized transform coefficient level generated by the process of step S307 with a feature corresponding to the quantization feature of step S307 , and derives the transform coefficient Coeff_IQ.

[0198] In step S309, the inverse orthogonal transform unit 318 performs inverse orthogonal transform on the transform coefficient Coeff_IQ obtained by the process of step S308 using a method corresponding to the orthogonal transform process of step S306 to derive the prediction residual D'. Since this inverse orthogonal transform process is the same as the inverse orthogonal transform process (to be described later) performed on the decoding side, the description performed on the decoding side (to be performed later) can be applied to the inverse orthogonal transform process in step S309.

[0199] In step S310 , the calculation unit 319 adds the predicted image obtained by the prediction process of step S304 and the prediction residual D′ derived by the process of step S309 to generate a local decoded image.

[0200] In step S311 , the loop filter unit 320 performs a loop filtering process on the local decoded image derived through the process of step S310 .

[0201] In step S312 , the frame memory 321 stores the local decoded image derived through the process of step S310 and the local decoded image filtered in step S311 .

[0202] In step S313, the encoding unit 315 encodes the quantized transform coefficient level obtained by the process of step S307. For example, the encoding unit 315 encodes the quantized transform coefficient level as information about the image by arithmetic coding or the like to generate encoded data. At this time, the encoding unit 315 encodes various encoding parameters (header information Hinfo, prediction mode information Pinfo, and transform information Tinfo). In addition, the encoding unit 315 derives residual information RInfo from the quantized transform coefficient level and encodes the residual information RInfo.

[0203] In step S314, the accumulation buffer 316 accumulates the encoded data obtained in this way and outputs the encoded data as a bit stream to the outside of the image encoding device 300, for example. The bit stream is transmitted to the decoding side via, for example, a transmission line or a recording medium. In addition, the rate control unit 323 performs rate control as needed.

[0204] When the process of step S314 is completed, the image encoding process is completed.

[0205] In the image encoding process in the above-mentioned flow, the present technology is applied to the encoding process of step S313. That is, the same as Figure 4 That is, the encoding unit 315 performs the encoding process of the present technology described in <1. Encoding of ST identifier>.

[0206] In this way, the image encoding device 300 can simplify the derivation of the context variable of the ST identifier while maintaining the encoding efficiency. In addition, the image encoding device 300 can reduce the memory size for maintaining the context variable of the ST identifier while maintaining the encoding efficiency. That is, the image encoding device 300 can suppress the increase in the load of the encoding process.

[0207] <5. Fourth embodiment>

[0208] <Image decoding device>

[0209] Fig. 9 : is a block diagram showing an example of the configuration of an image decoding device as one aspect of the present technology is applied. Fig. 9The image decoding device 400 shown in is a device for decoding the coded data of the moving image. For example, the image decoding device 400 can decode the coded data by the decoding method described in any of the non-patent documents mentioned above. For example, the image decoding device 400 decodes the coded data (bit stream) generated by the above-mentioned image coding device 300.

[0210] although Fig. 9 The main components such as processing units (blocks) and data flows are shown, but the processing units and data flows are not limited to Fig. 9 That is, Fig. 9 Processing units not shown as blocks and Fig. 9 Processing and data flows not shown as arrows in the figure may exist in the image decoding device 400.

[0211] exist Fig. 9 , the image decoding device 400 includes an accumulation buffer 411, a decoding unit 412, an inverse quantization unit 413, an inverse orthogonal transformation unit 414, a calculation unit 415, a loop filter unit 416, a reordering buffer 417, a frame memory 418, and a prediction unit 419. The prediction unit 419 includes an intra-frame prediction unit and an inter-frame prediction unit, which are not shown. The image decoding device 400 is a device for generating moving image data by decoding encoded data (bit stream).

[0212] <Accumulation Buffer>

[0213] The accumulation buffer 411 acquires a bit stream input to the image decoding device 400 and holds (stores) the bit stream. The accumulation buffer 411 supplies the stored bit stream to the decoding unit 412 at a predetermined timing or when a predetermined condition is satisfied.

[0214] <Decoding Unit>

[0215] The decoding unit 412 performs processing related to image decoding. For example, the decoding unit 412 receives the bit stream provided from the accumulation buffer 411 as input, and performs variable length decoding on the syntax value of each syntax element according to the definition of the syntax table to derive parameters from the bit stream.

[0216] The parameters derived from the syntax elements and the syntax values ​​of the syntax elements may include, for example, information such as header information Hinfo, prediction mode information Pinfo, transform information Tinfo, residual information Rinfo, and filter information Finfo. That is, the decoding unit 412 parses (analyzes and obtains) the information from the bitstream. The information will be described below.

[0217] <Header Information Hinfo>

[0218] The header information Hinfo may include, for example, header information such as video parameter set (VPS) / sequence parameter set (SPS) / picture parameter set (PPS) / slice header (SH). The header information Hinfo may include, for example, the following information, which defines the image size (width PicWidth, height PicHeight), bit depth (luminance bitDepthY, chrominance bitDepthC), chrominance array type ChromaArrayType, maximum value MaxCUSize / minimum value MinCUSize of CU size, maximum depth MaxQTDepth / minimum depth MinQTDepth of quadtree partitioning, maximum depth MaxBTDepth / minimum depth MinBTDepth of binary tree partitioning, maximum value MaxTSSize of transform skip block (also called maximum transform skip block size), on / off flag of each encoding tool (also called valid flag), etc.

[0219] For example, the on / off flag of the coding tool included in the header information Hinfo includes an on / off flag related to the transformation and quantization processing described below. The on / off flag of the coding tool can also be interpreted as a flag indicating whether there is a syntax related to the coding tool in the coded data. In addition, the on / off flag indicates that the coding tool can be used when its value is 1 (true), and indicates that the coding tool cannot be used when its value is 0 (false). The construction of the flag value can be reversed.

[0220] Inter-component prediction enabling flag (ccp_enabled_flag): This is flag information indicating whether inter-component prediction (cross-component prediction (CCP), also referred to as CC prediction) is available. For example, if its value is "1" (true), this flag information indicates that CCP is available, and if its value is "0" (false), it indicates that CCP is not available.

[0221] The CCP is also called inter-component linear prediction (CCLM or CCLMP).

[0222] <Prediction mode information Pinfo>

[0223] The prediction mode information Pinfo may include, for example, information such as size information PBSize (prediction block size) of a processing target PB (prediction block), intra prediction mode information IPinfo, and motion prediction information MVinfo.

[0224] The intra prediction mode information IPinfo may include, for example, prev_intra_luma_pred_flag, mpm_idx, and rem_intra_pred_mode in JCTVC-W1005, 7.3.8.5 coding unit syntax, a luma intra prediction mode IntraPredModeY derived from its syntax, and the like.

[0225] In addition, the intra-frame prediction mode information IPinfo may include, for example, an inter-component prediction flag (ccp_flag (cclmp_flag)), a multi-class linear prediction mode flag (mclm_flag), a chroma sample location type identifier (chroma_sample_loc_type_idx), a chroma MPM identifier (chroma_mpm_idx), a luma intra-frame prediction mode (IntraPredModeC) derived from these syntaxes, and the like.

[0226] The inter-component prediction flag (ccp_flag (cclmp_flag)) is flag information indicating whether inter-component linear prediction is applied. For example, when ccp_flag==1, it indicates that inter-component prediction is applied, and when ccp_flag==0, it indicates that inter-component prediction is not applied.

[0227] The multi-class linear prediction mode flag (mclm_flag) is information about the linear prediction mode (linear prediction mode information). More specifically, the multi-class linear prediction mode flag (mclm_flag) is flag information indicating whether the multi-class linear prediction mode is set. For example, "0" indicates a 1-class mode (single-class mode) (e.g., CCLMP), and "1" indicates a 2-class mode (multi-class mode) (e.g., MCLMP).

[0228] The chroma sample position type identifier (chroma_sample_loc_type_idx) is an identifier for identifying the type of the pixel position of the chroma component (also referred to as the chroma sample position type). For example, when the chroma array type (ChromaArrayType) as information on the color format indicates the 420 format, the chroma sample position type identifier is assigned as shown in the following formula.

[0229] chroma_sample_loc_type_idx==0:Type2

[0230] chroma_sample_loc_type_idx==1:Type3

[0231] chroma_sample_loc_type_idx==2:Type0

[0232] chroma_sample_loc_type_idx==3:Type1

[0233] This color difference sample location type identifier (chroma_sample_loc_type_idx) is transmitted (stored) as information (chroma_sample_loc_info()) on the pixel location of the color difference component.

[0234] The chroma MPM identifier (chroma_mpm_idx) is an identifier indicating which prediction mode candidate in the chroma intra prediction mode candidate list (intraPredModeCandListC) is designated as the chroma intra prediction mode.

[0235] The motion prediction information MVinfo may include, for example, information such as merge_idx, merge_flag, inter_pred_idc, ref_idx_LX, mvp_lX_flag, X={0,1}, and mvd (for example, see JCTVC-W1005, 7.3.8.6 Prediction Unit Syntax).

[0236] Of course, the information included in the prediction mode information Pinfo is arbitrary, and information other than this information may be included therein.

[0237] <Transformation information Tinfo>

[0238] The transformation information Tinfo may include, for example, the following information: Of course, the information included in the transformation information Tinfo is arbitrary, and may include information other than this information.

[0239] Process the width size TBWSize and height TBHSize of the target transform block (or the logarithmic values ​​of TBWSize and TBHSize with base 2, log2TBWSize and log2TBHSize, which are also available)

[0240] Transform skip flag (ts_flag): A flag indicating whether to skip (inverse) primary transform and (inverse) secondary transform

[0241] Scan Identifier (scanIdx)

[0242] Quantization parameter (qp)

[0243] Quantization matrix (scaling_matrix (e.g., JCTVC-W1005, 7.3.4 Scaling List Data Syntax))

[0244] <Residual information Rinfo>

[0245] The residual information Rinfo (for example, see 7.3.8.11 Residual Coding Syntax of JCTVC-W1005) may include, for example, the following syntax.

[0246] cbf (coded_block_flag): A flag indicating the presence or absence of residual data

[0247] last_sig_coeff_x_pos: the last non-zero coefficient X coordinate

[0248] last_sig_coeff_y_pos: the last non-zero coefficient Y coordinate

[0249] coded_sub_block_flag: A flag indicating the presence or absence of non-zero coefficients in a sub-block

[0250] sig_coeff_flag: A flag indicating the presence or absence of non-zero coefficients

[0251] gr1_flag: A flag indicating whether the non-zero coefficient level is greater than 1 (also called the GR1 flag)

[0252] gr2_flag: A flag indicating whether the non-zero coefficient level is greater than 2 (also called the GR2 flag) sign_flag: A flag indicating whether the non-zero coefficient is positive or negative (also called the sine code)

[0253] coeff_abs_level_remaining: non-zero coefficient residual level (also called non-zero coefficient residual level),

[0254] and the like.

[0255] Of course, the information included in the residual information Rinfo is arbitrary, and may include information other than this information.

[0256] <Filter information Finfo>

[0257] The filter information Finfo may include, for example, control information regarding the following filtering process.

[0258] Control information about the deblocking filter (DBF)

[0259] Control information about pixel adaptive offset (SAO)

[0260] Control information about the adaptive loop filter (ALF)

[0261] Control information about other linear / non-linear filters

[0262] More specifically, the filter information Finfo may include, for example, a picture to which each filter is applied, information for specifying a region in a picture, filter on / off control information in a CU unit, filter on / off control information about boundaries of slices and tiles, etc. Of course, the information included in the filter information Finfo is arbitrary, and may include information other than this information.

[0263] Returning to the description of the decoding unit 412 , the decoding unit 412 derives the quantized transform coefficient level of each coefficient position in each transform block with reference to the residual information Rinfo, and supplies the quantized transform coefficient level to the inverse quantization unit 413 .

[0264] In addition, the decoding unit 412 provides the parsed header information Hinfo, prediction mode information Pinfo, quantized transform coefficient level, transform information Tinfo and filter information Finfo to each block. Specifically, it is as follows.

[0265] The header information Hinfo is supplied to the inverse quantization unit 413 , the inverse orthogonal transform unit 414 , the prediction unit 419 , and the loop filter unit 416 .

[0266] The prediction mode information Pinfo is provided to the inverse quantization unit 413 and the prediction unit 419 .

[0267] The transform information Tinfo is supplied to the inverse quantization unit 413 and the inverse orthogonal transform unit 414 .

[0268] The filter information Finfo is provided to the loop filter unit 416 .

[0269] Of course, the above example is an example and is not limited to this example. For example, each encoding parameter can be provided to any processing unit. In addition, other information can be provided to any processing unit.

[0270] <Inverse Quantization Unit>

[0271] The inverse quantization unit 413 has at least a configuration required to perform processing related to inverse quantization. For example, the inverse quantization unit 413 receives the quantized transform coefficient level and the transform information Tinfo provided from the decoding unit 412 as input, and scales (inverse quantizes) the value of the quantized transform coefficient level based on the transform information Tinfo to derive the transform coefficient Coeff_IQ after inverse quantization.

[0272] This inverse quantization is performed as an inverse process of the quantization performed by the quantization unit 314. In addition, this inverse quantization is the same process as the inverse quantization performed by the inverse quantization unit 317. That is, the inverse quantization unit 317 performs the same process (inverse quantization) as that of the inverse quantization unit 413.

[0273] The inverse quantization unit 413 provides the derived transform coefficient Coeff_IQ to the inverse orthogonal transform unit 414 .

[0274] <Inverse Orthogonal Transformation Unit>

[0275] The inverse orthogonal transform unit 414 performs processing related to inverse orthogonal transform. For example, the inverse orthogonal transform unit 414 receives the transform coefficient Coeff_IQ provided from the inverse quantization unit 413 and the transform information Tinfo provided from the decoding unit 412 as input, and performs inverse orthogonal transform processing on the transform coefficient Coeff_IQ based on the transform information Tinfo to derive the prediction residual D'. For example, the inverse orthogonal transform unit 414 performs an inverse secondary transform on the transform coefficient Coeff_IQ based on the ST identifier to generate a primary transform coefficient, and performs a primary transform on the primary transform coefficient to generate a prediction residual D'.

[0276] This inverse orthogonal transform is performed as an inverse process of the orthogonal transform performed by the orthogonal transform unit 313. In addition, this inverse orthogonal transform is the same process as the inverse orthogonal transform performed by the inverse orthogonal transform unit 318. That is, the inverse orthogonal transform unit 318 performs the same process (inverse orthogonal transform) as that of the inverse orthogonal transform unit 414.

[0277] The inverse orthogonal transform unit 414 provides the derived prediction residual D′ to the calculation unit 415 .

[0278] <Computational Unit>

[0279] The calculation unit 415 performs processing on adding information about the image. For example, the calculation unit 415 receives as input the prediction residual D' provided from the inverse orthogonal transform unit 414 and the prediction image P provided from the prediction unit 419. As shown in the following formula, the calculation unit 415 adds the prediction residual D' and the prediction image P (prediction signal) corresponding to the prediction residual D' to derive the local decoded image Rlocal.

[0280] Rlocal=D'+P

[0281] The calculation unit 415 provides the derived local decoded image Rlocal to the loop filter unit 416 and the frame memory 418 .

[0282] <Loop filter unit>

[0283] The loop filter unit 416 performs processing related to the loop filtering process. For example, the loop filter unit 416 receives as input the local decoded image Rlocal provided from the calculation unit 415 and the filter information Finfo provided from the decoding unit 412. Meanwhile, the information input to the loop filter unit 416 is arbitrary, and information other than this information may be input.

[0284] The loop filter unit 416 appropriately filters the local decoded image Rlocal based on the filter information Finfo.

[0285] For example, as described in NPL 11, the loop filter unit 416 applies four loop filters in this order: a bidirectional filter, a deblocking filter (DBF), an adaptive offset filter (sample adaptive offset (SAO)), and an adaptive loop filter (ALF). Which filter is applied and in what order is arbitrary and can be appropriately selected.

[0286] The loop filter unit 416 performs a filtering process corresponding to the filtering process performed by the encoding side (e.g., the loop filter unit 320 of the image encoding device 300). Of course, the filtering process performed by the loop filter unit 416 is arbitrary and not limited to the above example. For example, the loop filter unit 416 can apply a Wiener filter or the like.

[0287] The loop filter unit 416 provides the filtered local decoded image Rlocal to the reordering buffer 417 and the frame memory 418 .

[0288] <Reorder Buffer>

[0289] The reordering buffer 417 receives the local decoded image Rlocal supplied from the loop filter unit 416 as input and holds (stores) it. The reordering buffer 417 reconstructs the decoded image R for each picture unit using the local decoded image Rlocal and holds it (stores it in the buffer). The reordering buffer 417 reorders the obtained decoded images R from the decoding order to the reproduction order. The reordering buffer 417 outputs the reordered decoded image R group as mobile image data to the outside of the image decoding device 400.

[0290] <Frame Memory>

[0291] The frame memory 418 performs processing related to storage of image-related data. For example, the frame memory 418 receives the local decoded image Rlocal provided from the calculation unit 415 as input, reconstructs the decoded image R for each picture unit, and stores it in a buffer in the frame memory 418.

[0292] In addition, the frame memory 418 receives the loop-filtered local decoded image Rlocal provided from the loop filter unit 416 as input, reconstructs the decoded image R for each picture, and stores it in a buffer in the frame memory 418. The frame memory 418 appropriately provides the stored decoded image R (or a portion thereof) to the prediction unit 419 as a reference image.

[0293] The frame memory 418 may store header information Hinfo, prediction mode information Pinfo, transformation information Tinfo, filter information Finfo, and the like related to generation of a decoded image.

[0294] <Prediction Unit>

[0295] The prediction unit 419 performs processing related to the generation of the predicted image. For example, the prediction unit 419 receives the prediction mode information Pinfo provided from the decoding unit 412 as an input and performs prediction by the prediction method specified by the prediction mode information Pinfo to derive the predicted image P. When deriving, the prediction unit 419 uses the decoded image R (or a part thereof) specified by the prediction mode information Pinfo stored in the frame memory 418 before or after filtering as a reference image. The prediction unit 419 provides the derived predicted image P to the calculation unit 415.

[0296] These processing units (accumulation buffer 411 to prediction unit 419) have an arbitrary configuration. For example, each processing unit can be configured as a logic circuit for implementing the above-mentioned processing. In addition, each processing unit may include, for example, a CPU, ROM, RAM, etc. and execute a program using these components to implement the above-mentioned processing. It goes without saying that each processing unit can have the two configurations mentioned above, implementing part of the above-mentioned processing according to the logic circuit, and implementing the other part of the processing by executing the program. The processing units can have independent configurations, for example, some processing units can implement part of the above-mentioned processing according to the logic circuit, some other processing units can implement the above-mentioned processing by executing the program, and some other processing units can implement the above-mentioned processing according to both the logic circuit and the execution of the program.

[0297] In the image decoding device 400 having the above configuration, the present technology described in <1. Encoding of ST identifier> can be applied to the decoding unit 412. That is, the decoding unit 412 has Figure 5 The decoding device 200 shown has the same configuration and performs the same processing.

[0298] In this way, the image decoding device 400 can simplify the derivation of the context variable of the ST identifier while maintaining the coding efficiency. In addition, the image decoding device 400 can reduce the memory size for maintaining the context variable of the ST identifier while maintaining the coding efficiency. That is, the image decoding device 400 can suppress the increase in the load of the decoding process.

[0299] <Flow of Image Coding Process>

[0300] Next, we will refer to Fig.10 The flowchart describes an example of the flow of the image decoding process performed by the image decoding device 400 having the above-described configuration.

[0301] When the image decoding process starts, the accumulation buffer 411 acquires encoded data (bit stream) supplied from the outside of the image decoding device 400 in step S401 and holds (accumulates) it.

[0302] In step S402, the decoding unit 412 decodes the coded data (bit stream) and obtains the quantized transform coefficient level. In addition, the decoding unit 412 parses (analyzes and obtains) various coding parameters from the coded data (bit stream) through the decoding.

[0303] In step S403 , the inverse quantization unit 413 performs inverse quantization, which is an inverse process of quantization performed on the encoding side, on the quantized transform coefficient level obtained by the process of step S402 to obtain a transform coefficient Coeff_IQ.

[0304] In step S404 , the inverse orthogonal transform unit 414 performs an inverse orthogonal transform process, which is an inverse process of the orthogonal transform process performed on the encoding side, on the transform coefficient Coeff_IQ obtained in step S403 to obtain a prediction residual D′.

[0305] In step S405 , the prediction unit 419 performs prediction processing using the prediction method specified on the encoding side based on the information parsed in step S402 , and generates a predicted image P by referring to the reference image stored in the frame memory 418 .

[0306] In step S406 , the calculation unit 415 adds the prediction residual D′ obtained in step S404 to the prediction image P obtained in step S405 to derive a local decoded image Rlocal.

[0307] In step S407 , the loop filter unit 416 performs a loop filter process on the local decoded image Rlocal obtained by the process of step S406 .

[0308] In step S408, the rearrangement buffer 417 derives a decoded image R using the filtered local decoded image Rlocal obtained by the process of step S407, and reorders the order of the decoded image R group from the decoding order to the reproduction order. The decoded image R group reordered in the reproduction order is output as a moving image to the outside of the image decoding device 400.

[0309] Furthermore, in step S409 , the frame memory 418 stores at least one of the local decoded image Rlocal obtained by the process of step S406 and the filtered local decoded image Rlocal obtained by the process of step S407 .

[0310] When the process of step S409 is completed, the image decoding process is completed.

[0311] In the image decoding process in the above flow, the present technology is applied to the decoding process of step S402. That is, the decoding process of step S402 is performed in accordance with Figure 6 Decoding process in the same flow. That is, the decoding unit 412 performs the decoding process to which the present technique described in <1. Encoding of ST identifier> is applied.

[0312] In this way, the image decoding device 400 can simplify the derivation of the context variable of the ST identifier while maintaining the coding efficiency. In addition, the image decoding device 400 can reduce the memory size for maintaining the context variable of the ST identifier while maintaining the coding efficiency. That is, the image decoding device 400 can suppress the increase in the load of the decoding process.

[0313] <6. Appendix>

[0314] <Computer>

[0315] The above series of processing can be performed by hardware or software. In the case where a series of processing is performed by software, a program configuring the software is installed on a computer. Here, the computer includes, for example, a computer built into dedicated hardware and a general-purpose personal computer on which various programs are installed to be able to perform various functions.

[0316] Fig.11 : is a block diagram showing an example of the hardware configuration of a computer that executes the above-described series of processes according to a program.

[0317] exist Fig.11 In a computer 800 shown in , a central processing unit (CPU) 801 , a read only memory (ROM) 802 , and a random access memory (RAM) 803 are connected to one another via a bus 804 .

[0318] An input / output interface 810 is also connected to the bus 804. An input unit 811, an output unit 812, a storage unit 813, a communication unit 814, and a drive 815 are connected to the input / output interface 810.

[0319] The input unit 811 includes, for example, a keyboard, a mouse, a microphone, a touch panel, an input terminal, etc. The output unit 812 includes, for example, a display, a speaker, an output terminal, etc. The storage unit 813 includes, for example, a hard disk, a RAM disk, a nonvolatile memory, etc. The communication unit 814 includes, for example, a network interface. The drive 815 drives a removable medium 821 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

[0320] In the computer configured as above, for example, the CPU 801 performs the above series of processing by loading a program stored in the storage unit 813 to the RAM 803 via the input / output interface 810 and the bus 804 and executing the program. Data and the like necessary for the CPU 801 to perform various processing are also appropriately stored in the RAM 803.

[0321] The program executed by the computer may be recorded on the removable medium 821 as a package medium or the like to be applied. In this case, the program can be installed in the storage unit 813 via the input / output interface 810 by inserting the removable medium 821 into the drive 815 .

[0322] In addition, the program can also be provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital satellite broadcasting. In this case, the program can be received by the communication unit 814 and installed in the storage unit 813.

[0323] In addition, the program may be installed in the ROM 802 or the storage unit 813 in advance.

[0324] <Application target of this technology>

[0325] The present technology can be applied to any image encoding / decoding method. That is, as long as it does not contradict the above-mentioned present technology, the specifications of various processes for image encoding / decoding, such as transformation (inverse transformation), quantization (inverse quantization), encoding (decoding) and prediction are arbitrary and not limited to the above-mentioned examples. In addition, as long as it does not contradict the above-mentioned present technology, some such processes can be omitted.

[0326] In addition, the present technology can be applied to a multi-view image encoding / decoding system that encodes / decodes a multi-view image including an image of a plurality of viewpoints (views). In that case, the present technology can be applied to encoding / decoding of each viewpoint (view).

[0327] In addition, the present technology can be applied to a hierarchical image encoding (scalable encoding) / decoding system that encodes / decodes a layered hierarchical image to have a scalable function for a predetermined parameter. In that case, the present technology can be applied to encoding / decoding of each layer.

[0328] Although the encoding device 100 , the decoding device 200 , the image encoding device 300 , and the image decoding device 400 have been described above as application examples of the present technology, the present technology can be applied to any configuration.

[0329] For example, the present technology can be applied to various electronic devices, such as transmitters and receivers in satellite broadcasting (e.g., television receivers and cellular phones), wired broadcasting such as cable television, transmission on the Internet, transmission to terminals based on cellular communications, etc., or devices that record images in media such as optical disks, magnetic disks, and flash memories or reproduce images from these storage media (e.g., hard disk recorders and video cameras).

[0330] In addition, for example, the present technology can also be implemented as part of a component of a device, such as a processor used as a system large-scale integration (LSI) circuit, etc. (e.g., a video processor), a module using multiple processors, etc. (e.g., a video module), a unit using multiple modules, etc. (e.g., a video unit), or a collection of units with additional other functions (e.g., a video collection).

[0331] In addition, the present technology can also be applied to, for example, a network system including a plurality of devices. For example, the present technology can be implemented as cloud computing in which functions are shared and processed jointly by a plurality of devices via a network. For example, the present technology can also be implemented in a cloud service that provides services regarding images (moving images) to any terminal such as a computer, an audio-visual (AV) device, a portable information processing terminal, an Internet of Things (IoT) device, etc.

[0332] Meanwhile, in this specification, a system refers to a collection of multiple components (devices, modules (components), etc.), and it does not matter whether all components are arranged in a single housing. Therefore, multiple devices housed in different housings and connected via a network and a device in which multiple modules are housed in one housing are both systems.

[0333] <Fields / Applications to which this technology is applicable>

[0334] Systems, devices, processing units, etc. to which the present technology is applied can be used in any field such as transportation, medical care, crime prevention, agriculture, animal husbandry, mining, beauty, factories, household appliances, weather and natural monitoring, etc. In addition, its application is arbitrary.

[0335] For example, the present technology can be applied to systems and devices for providing content for appreciation, etc. In addition, for example, the present technology can also be applied to systems and devices for traffic such as traffic condition monitoring and automatic driving control. In addition, for example, the present technology can also be applied to systems and devices for safety. In addition, for example, the present technology can be applied to systems and devices for automatic control of machines, etc. In addition, for example, the present technology can also be applied to systems and devices for agriculture and animal husbandry. In addition, for example, the present technology can also be applied to systems and devices for monitoring natural conditions such as volcanoes, forests, and oceans and wild animals. In addition, for example, the present technology can also be applied to systems and devices for sports.

[0336] <Others>

[0337] Meanwhile, in the present specification, a "flag" is information for identifying a plurality of states, and includes not only information for identifying two states of true (1) and false (0), but also information capable of identifying three or more states. Therefore, the "flag" may have not only two values ​​of 1 / 0 but also three or more values. That is, the number of bits constituting the "flag" is arbitrary, and it may be 1 bit or more bits. In addition, the identification information (also including the flag) is assumed to have not only a form in which the identification information is included in the bit stream, but also a form in which the difference information between the identification information and specific reference information is included in the bit stream, and therefore, in the present specification, the "flag" and the "identification information" include not only the information itself, but also the difference information relative to the reference information.

[0338] In addition, various types of information (metadata, etc.) about the coded data (bitstream) can be sent or recorded in any form, as long as the various types of information are associated with the coded data. Here, the term "association" means, for example, making another data available (linkable) when processing one data. That is, the associated data can be collected as one data or can be separate data. For example, information associated with the coded data (image) can be sent on a transmission path different from the transmission path of the coded data (image). In addition, for example, the information associated with the coded data (image) can be recorded on a recording medium different from the recording medium of the coded data (image) (or recorded in another recording area of ​​the same recording medium). At the same time, the "association" can be a part of the data, not all of the data. For example, an image and information corresponding to the image can be associated with each other in arbitrary units such as multiple frames, one frame, or a part of a frame.

[0339] At the same time, in this specification, the terms "synthesis", "multiplexing", "addition", "integration", "inclusion", "storage", "submission", "connection", "insertion" and the like refer to combining multiple things into one, such as combining encoded data and metadata into one data, and refer to a method of the above-mentioned "association".

[0340] In addition, the embodiment of the present technology is not limited to the above-described embodiment, but various changes can be made within the scope not departing from the gist of the present technology.

[0341] For example, a configuration described as one device (or one processing unit) may be divided to be configured as a plurality of devices (or processing units). Conversely, a configuration described as a plurality of devices (or processing units) in the above description may be configured together as one device (or one processing unit). Furthermore, it goes without saying that components other than the above components may be added to the configuration of each device (or each processing unit). Furthermore, a portion of the configuration of one device (or one processing unit) may be included in the configuration of another device (or another processing unit), as long as the configuration or operation of the entire system is substantially the same.

[0342] In addition, for example, the above-mentioned program can be executed in an arbitrary device. In that case, the device can have necessary functions (functional blocks, etc.) so that necessary information can be obtained.

[0343] In addition, for example, each step in the flowchart may be performed by one device, or may be performed by multiple devices in a shared manner. In addition, in the case where a single step includes multiple processes, the multiple processes may be performed by one device or may be performed by multiple devices in a shared manner. In other words, the multiple processes included in a single step may be performed as processes of multiple steps. Conversely, the processes described as multiple steps may be performed together as a single step.

[0344] In addition, for example, the program executed by the computer may be executed so that the processing steps describing the program are executed in chronological order according to the order described in this specification, or are executed separately in parallel or at necessary timings such as in response to a call. That is, each processing step may be executed in an order different from the above order as long as no contradiction occurs. In addition, various processing steps describing the program may be executed in parallel with the processing of another program, or may be executed in combination with the processing of another program.

[0345] In addition, for example, multiple technologies related to the present technology can be implemented separately and independently as long as no contradiction occurs. It goes without saying that any number of the present technologies can also be implemented in parallel. For example, part or all of the present technology described in any embodiment can be implemented in combination with part or all of the present technology described in other embodiments. In addition, part or all of the above-mentioned present technology can be implemented in combination with other technologies not described above.

[0346] At the same time, the present technology can also adopt the following configurations.

[0347] (1) An image processing device includes a coding unit, wherein the coding unit is configured to: set a fixed context variable for the first bit of a bit string of secondary transformation control information, and arithmetically encode each bit of the secondary transformation control information with reference to the set fixed context variable, wherein the secondary transformation control information is control information about secondary transformation.

[0348] (2) The image processing device according to (1), wherein the fixed context variable is “0”.

[0349] (3) The image processing device according to (1), wherein the encoding unit further sets a bypass flag for the second bit of the bit string.

[0350] (4) The image processing device according to (1), wherein the encoding unit further sets a context variable common to the first bit element for the second bit element of the bit string.

[0351] (5) The image processing device according to (4), wherein the common context variable is "0".

[0352] (6) The image processing device according to (1), wherein the secondary transform control information includes a secondary transform identifier indicating a type of the secondary transform.

[0353] (7) The image processing device according to (1), wherein the encoding unit binarizes the secondary transform control information to generate the bit string, and sets the fixed context variable for a first bit of the generated bit string.

[0354] (8) The image processing device according to (1), further including a coefficient transform unit configured to perform the secondary transform on the primary transform coefficients of the transform block corresponding to the secondary transform control information to generate secondary transform coefficients.

[0355] (9) The image processing device according to (8), further including a quantization unit configured to quantize the secondary transform coefficient generated by the coefficient transform unit to generate a quantized coefficient.

[0356] (10) An image processing method, comprising: setting a fixed context variable for the first bit of a bit string of secondary transformation control information, and arithmetically encoding each bit of the secondary transformation control information with reference to the set fixed context variable, wherein the secondary transformation control information is control information about secondary transformation.

[0357] (11) An image processing device includes a decoding unit configured to perform arithmetic decoding with reference to a fixed context variable set for a first bit of a bit string of secondary transform control information, the secondary transform control information being control information regarding secondary transform.

[0358] (12) The image processing device according to (11), wherein the fixed context variable is “0”.

[0359] (13) The image processing device according to (11), wherein the decoding unit further performs arithmetic decoding based on a bypass flag set for a second bit element of the bit string.

[0360] (14) The image processing device according to (11), wherein the decoding unit further performs arithmetic decoding with reference to a context variable set for a second bit element of the bit string and common to the first bit element.

[0361] (15) The image processing device according to (14), wherein the common context variable is "0".

[0362] (16) The image processing device according to (11), wherein the secondary transform control information includes a secondary transform identifier indicating a type of the secondary transform.

[0363] (17) The image processing device according to (11), wherein the decoding unit performs multi-value conversion on a bit string obtained by arithmetic decoding to generate the secondary transform control information.

[0364] (18) The image processing device according to (11) further includes an inverse coefficient transform unit configured to perform an inverse secondary transform on the secondary transform coefficients of the transform block corresponding to the secondary transform control information to generate primary transform coefficients.

[0365] (19) The image processing device according to (18), further comprising an inverse quantization unit configured to inverse quantize the quantized coefficient to generate the secondary transform coefficient,

[0366] The inverse coefficient transform unit performs an inverse secondary transform on the secondary transform coefficients generated by the inverse quantization unit to generate the primary transform coefficients.

[0367] (20) An image processing method, comprising: performing arithmetic decoding with reference to a fixed context variable set for a first bit of a bit string of secondary transform control information, wherein the secondary transform control information is control information regarding secondary transform.

[0368] [reference numerals list]

[0369] 100 Encoding device

[0370] 121 Binarization unit

[0371] 122 Select Unit

[0372] 123 context model

[0373] 124 Arithmetic Coding Unit

[0374] 125 Arithmetic Coding Unit

[0375] 126 Select Unit

[0376] 200 decoding device

[0377] 221 Selection Unit

[0378] 222Context Model

[0379] 223 Arithmetic decoding unit

[0380] 224 Arithmetic Decoding Unit

[0381] 225 Selected Units

[0382] 226 Multi-value conversion unit

[0383] 300 Image decoding device

[0384] 315 coding units

[0385] 400 Image Coding Unit

[0386] 412 decoding unit

Claims

1. An image processing device, comprising: A setting unit, which sets a context variable for a first bit and a second bit of a bit string of secondary transformation control information, wherein the secondary transformation control information is control information about secondary transformation; as well as The encoding unit performs arithmetic encoding on each bit of the secondary transform control information with reference to the context variable set by the setting unit.

2. The image processing device according to claim 1, wherein: The secondary transform control information includes a secondary transform identifier indicating a type of the secondary transform.

3. The image processing device according to claim 1, wherein: The setting unit binarizes the secondary transform control information to generate the bit string, and sets a context variable for each bit element of the generated bit string.

4. The image processing device according to claim 1, wherein: A coefficient transform unit is also included, which performs the secondary transform on the primary transform coefficients of the block corresponding to the secondary transform control information to generate secondary transform coefficients.

5. The image processing device according to claim 4, wherein: A quantization unit is further included, which quantizes the secondary transformation coefficient generated by the coefficient transformation unit to generate a quantized coefficient.

6. An image processing method, comprising: setting a context variable for a first bit and a second bit of a bit string of secondary transform control information, wherein the secondary transform control information is control information about secondary transform, Each bit of the secondary transform control information is arithmetically encoded with reference to the set context variable.

7. The image processing method according to claim 6, wherein: The secondary transform control information includes a secondary transform identifier indicating a type of the secondary transform.

8. The image processing method according to claim 6, wherein: The secondary transform control information is binarized to generate the bit string, and a context variable is set for each bit element of the generated bit string.

9. The image processing method according to claim 6, further comprising: The secondary transform is performed on the primary transform coefficients of the block corresponding to the secondary transform control information to generate secondary transform coefficients.

10. The image processing method according to claim 9, further comprising: The generated secondary transform coefficients are quantized to generate quantized coefficients.

11. An image encoding device, comprising: a prediction unit configured to generate a prediction image of an input image input to the image encoding device; an orthogonal transform unit configured to perform an orthogonal transform on a prediction residual based on transform information to derive a transform coefficient, wherein the prediction residual is obtained by subtracting the prediction image from the input image; a quantization unit configured to quantize the transform coefficients based on the transform information; and The encoding unit is configured to perform variable length encoding on the quantized transform coefficient to generate a bit string.

Images