Coding device, decoding device and program product

Abstract

A coding device performs coding processing on block units obtained by dividing an image composed of a luminance signal and a color difference signal, the coding device comprising: a prediction unit that predicts a target color difference block as a coding target block of the color difference signal and generates a predicted color difference block; a residual generation unit that generates a color difference prediction residual, the color difference prediction residual representing the difference between the target color difference block and the predicted color difference block; a transformation unit that applies a transformation process to the color difference prediction residual and generates a transformation coefficient, the transformation unit controlling the transformation type applied to the transformation process according to the signal format of the image.

Description

Technical Field

[0001] The present invention relates to an encoding device, a decoding device, and a program. Background Art

[0002] In video coding schemes such as HEVC (High Efficiency Video Coding) and VVC (Versatile Video Coding), the encoding device switches between intra-frame prediction and inter-frame prediction for each target block, which is a block-by-block representation of the original image. The encoding device generates a prediction residual, which is the difference (error) between the prediction block of the block-by-block prediction image and the target block. The encoding device applies a transform to the generated prediction residual, concentrating the energy distribution of the spatially spread prediction residual to achieve efficient encoding.

[0003] In VVC, multiple transform types can be selectively applied to the prediction residual to improve the concentration of the energy distribution of the prediction residual. Specifically, when the default transform type is selected for the prediction residual, the encoder applies DCT (Discrete Cosine Transform) type 2 (hereinafter referred to as DCT2) in the horizontal and vertical directions and signals flag information indicating that the transform type is the default mode.

[0004] On the other hand, in the case where a mode other than the default mode is selected, the encoding device applies either DCT type 8 (hereinafter referred to as DCT8) or DST (Discrete Sine Transform) type 7 (hereinafter referred to as DST7) to the prediction residual in the horizontal and vertical directions, respectively, and sends a signal together with flag information indicating that any one of the combinations of DCT8 and DST7 (a total of 4 modes) is selected, together with flag information indicating that the transform category is not the default mode.

[0005] However, research has shown that in VVC, only modes other than the default mode can be applied to the luma signal, and only the default mode can be applied to the chroma signal. This is because in the 4:2:0 video signal format, the resolution of the chroma signal is half that of the luma signal, and the chroma signal contains only a small amount of high-frequency components.

[0006] Prior art literature

[0007] Non-patent literature

[0008] JVET-P2001 “Versatile Video Coding (Draft 7)” Summary of the Invention

[0009] According to the first method, an encoding device is an encoding device that performs encoding processing on block units obtained by dividing an image composed of a luminance signal and a color difference signal, and the encoding device includes: a prediction unit that predicts a target color difference block as an encoding target block of the color difference signal and generates a predicted color difference block; a residual generation unit that generates a color difference prediction residual, wherein the color difference prediction residual represents the difference between the target color difference block and the predicted color difference block; a transformation unit that applies transformation processing to the color difference prediction residual and generates a transformation coefficient, and the transformation unit controls the transformation type applied to the transformation processing according to the signal format of the image.

[0010] According to the second method, a decoding device is a decoding device that performs decoding processing on block units obtained by dividing an image composed of a luminance signal and a color difference signal, and the decoding device includes: a prediction unit that predicts a target color difference block as a decoding target block of the color difference signal and generates a predicted color difference block; an inverse transformation unit that applies inverse transformation processing to the transformation coefficient from the encoding side to generate a color difference prediction residual representing the difference between the target color difference block and the predicted color difference block, and the inverse transformation unit controls the transformation type applied to the inverse transformation processing according to the signal format of the image.

[0011] The program according to the third aspect causes a computer to function as the encoding device according to the first aspect.

[0012] The program according to the fourth aspect causes a computer to function as the decoding device according to the second aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 is a diagram showing the structure of an encoding device according to an embodiment.

[0014] Figure 2 This is a diagram showing a total of three transform types (Basis functions), namely DCT2, DCT8, and DCT7 according to the embodiment.

[0015] Figure 3 It is a diagram showing a conversion process of a color difference signal according to an embodiment.

[0016] Figure 4 3 is a diagram showing an operation flow related to a transform process in an encoding device according to an embodiment.

[0017] Figure 5 It is a diagram showing the structure of a decoding device according to an embodiment.

[0018] Figure 6 It is a diagram showing an operation flow related to inverse transform processing in a decoding device according to an embodiment. DETAILED DESCRIPTION

[0019] In the 4:4:4 signal format, the resolution of the color difference signal is the same as that of the luminance signal, and the amount of information contained in the color difference signal is significantly greater than in the 4:2:0 format. Furthermore, even in the 4:2:2 format, the vertical resolution of the color difference signal and the luminance signal is equal, so, like in the 4:4:4 format, the amount of information contained in the color difference signal is greater.

[0020] However, in the current VVC standard scheme, only a default mode transform can be applied to a color difference signal regardless of the signal format, and thus there is room for improving the encoding efficiency of the color difference signal.

[0021] Therefore, in this embodiment, an encoding device, a decoding device, and a program are provided that improve the encoding efficiency of a color difference signal.

[0022] The encoding device and decoding device according to the embodiment will be described with reference to the accompanying drawings. The image encoding device and image decoding device according to the embodiment respectively encode and decode moving images represented by MPEG. In the following description of the drawings, the same or similar parts are marked with the same or similar symbols.

[0023] (Encoding device)

[0024] First, the structure of the encoding device according to the present embodiment will be described. Figure 1 2 is a diagram showing the structure of the encoding device 1 according to the present embodiment.

[0025] like Figure 1 As shown, the encoding device 1 includes a block division unit 100 , a residual generation unit 110 , a transform and quantization unit 120 , an entropy encoding unit 130 , an inverse quantization and inverse transform unit 140 , a synthesis unit 150 , a memory 160 , and a prediction unit 170 .

[0026] The block division unit 100 divides the original input image, which is a unit of a frame (or picture) constituting a moving image, into a plurality of image blocks and outputs the resulting image blocks to the residual generation unit 110. The image block size may be, for example, 32×32 pixels, 16×16 pixels, 8×8 pixels, or 4×4 pixels. The shape of an image block is not limited to a square and may also be a rectangle (non-square). An image block is the unit for encoding by the encoding device 1 (i.e., the encoding target block) and the unit for decoding by the decoding device (i.e., the decoding target block). Such an image block is sometimes referred to as a CU (Coding Unit).

[0027] The block segmentation unit 100 performs block segmentation on the luminance signal and color difference signal that constitute the image, outputting luminance blocks and color difference blocks. The segmentation of the luminance signal and color difference signal can also be controlled independently. Hereinafter, when luminance blocks and color difference blocks are not distinguished, they are collectively referred to as encoding target blocks.

[0028] The residual generation unit 110 calculates a prediction residual representing a difference (error) between the encoding target block output by the block division unit 100 and a prediction block obtained by predicting the encoding target block by the prediction unit 170. Specifically, the residual generation unit 110 calculates the prediction residual by subtracting each pixel value of the prediction block from each pixel value of the encoding target block, and outputs the calculated prediction residual to the transform and quantization unit 120.

[0029] The transform and quantization unit 120 performs transform processing and quantization processing in units of blocks. The transform and quantization unit 120 includes a transform unit 121 and a quantization unit 122.

[0030] The transform unit 121 transforms the prediction residual output by the residual generator 110, calculates transform coefficients, and outputs the calculated transform coefficients to the quantizer 122. Transformation refers to the process of converting the prediction residual from the spatial domain to the frequency domain, such as discrete cosine transform (DCT), discrete sine transform (DST), Karhunen-Loewenstein transform (KLT), etc. However, the transform process may not include transforming the pixel domain signal into the frequency domain, but may include transform skipping, such as scaling, to adjust the signal. The transform unit 121 outputs information regarding the transform process applied to the current block to the entropy encoder 130.

[0031] To improve the concentration of the energy distribution of the prediction residual, the transform unit 121 can selectively apply multiple transform types to the prediction residual. In this embodiment, the selectable transform types are DCT2, DCT8, and DCT7. However, the selectable transform types may further include transform types such as transform skip.

[0032] Specifically, the transform unit 121 includes an MTS (Multiple Transform Selection) function for switching between multiple transform types for each target block. The transform unit 121 determines a transform type from DCT2, DST7, and DCT8 for the horizontal and vertical directions, respectively, and outputs transform type information indicating the determined transform type to the entropy coding unit 130 and the inverse transform unit 142.

[0033] Figure 2 This figure shows a total of three transform types, namely DCT2, DCT8, and DST7, in this embodiment. Figure 2In the example, N represents the size of the target block. DCT8 includes a basis whose impulse response decreases monotonically. Specifically, DCT8 is a transform whose impulse response T0(j) of the lowest-frequency filter decreases monotonically (where j = 0, ..., N-1). One end of the DCT8 transform waveform has a large value and is open. DST7 includes a basis whose impulse response increases monotonically. Specifically, DST7 is a transform whose impulse response T0(j) of the lowest-frequency filter increases monotonically (where j = 0, ..., N-1). One end of the DST7 transform waveform is closed.

[0034] In this embodiment, DCT2, DCT8, and DST7 are used as examples of various types of transform processing applied to the prediction residual. However, any transform processing that selectively switches and applies a transform is acceptable and is not limited to these three types of transforms. For example, other DCTs or DSTs such as DCT1 and DCT5 may be used, or a transform such as a discrete wavelet transform may be used.

[0035] The quantization unit 122 quantizes the transform coefficients output from the transform unit 121 using a quantization parameter (Qp) and a quantization matrix, and outputs the quantized transform coefficients, ie, quantized transform coefficients, to the entropy coding unit 130 and the inverse quantization and inverse transform unit 140 .

[0036] The entropy coding unit 130 performs entropy coding on the quantized transform coefficients output by the quantization unit 122, performs data compression, generates coded data (bitstream), and outputs the coded data to the outside of the encoding device 1. For entropy coding, Huffman coding, CABAC (Context-based Adaptive Binary Arithmetic Coding), or the like can be used.

[0037] The entropy coding unit 130 also encodes the information about the transformation process input from the transformation unit 121, outputs a bitstream, and transmits (signals) the bitstream to the decoding side. Furthermore, the entropy coding unit 130 encodes the information about the prediction process input from the prediction unit 170, outputs a bitstream, and transmits the bitstream to the decoding side.

[0038] The inverse quantization and inverse transformation unit 140 performs inverse quantization and inverse transformation on a block basis. The inverse quantization and inverse transformation unit 140 includes an inverse quantization unit 141 and an inverse transformation unit 142 .

[0039] The inverse quantization unit 141 performs an inverse quantization process corresponding to the quantization process performed by the quantization unit 122. Specifically, the inverse quantization unit 141 restores the transform coefficient by inverse quantizing the quantized transform coefficient output by the quantization unit 122 using a quantization parameter (Qp) and a quantization matrix, and outputs the restored transform coefficient to the inverse transform unit 142.

[0040] The inverse transform unit 142 performs an inverse transform corresponding to the transform performed by the transform unit 121 based on the transform type information output by the transform unit 121. For example, if the transform unit 121 performs a discrete cosine transform, the inverse transform unit 142 performs an inverse discrete cosine transform. The inverse transform unit 142 performs an inverse transform on the transform coefficients output by the inverse quantization unit 141, restores the prediction residual, and outputs the restored prediction residual, i.e., the restored prediction residual, to the synthesis unit 150.

[0041] The synthesis unit 150 synthesizes the restored prediction residual output by the inverse transform unit 142 with the prediction block output by the prediction unit 170 on a pixel-by-pixel basis. The synthesis unit 150 adds each pixel value of the restored prediction residual to each pixel value of the prediction block to decode (reconstruct) the encoding target block and outputs the decoded block to the memory 160. Note that the decoded block may also be referred to as a reconstructed block.

[0042] Memory 160 stores the decoded blocks output by synthesis unit 150 and stores the decoded blocks as decoded images in units of frames. Memory 160 outputs the stored decoded blocks or decoded images to prediction unit 170. A loop filter may be inserted between synthesis unit 150 and memory 160.

[0043] The prediction unit 170 performs prediction processing in units of blocks and includes an inter prediction unit 171 , an intra prediction unit 172 , and a switching unit 173 .

[0044] The inter-frame prediction unit 171 performs inter-frame prediction using inter-frame correlation. Specifically, using the decoded image stored in the memory 160 as a reference image, the inter-frame prediction unit 171 calculates a motion vector using a method such as block matching, predicts the encoding target block, and generates an inter-frame prediction block. The generated inter-frame prediction block is then output to the switching unit 173. Here, the inter-frame prediction unit 171 selects the optimal inter-frame prediction method from among inter-frame prediction using multiple reference images (typically bidirectional prediction) and inter-frame prediction using a single reference image (unidirectional prediction), and performs inter-frame prediction using the selected inter-frame prediction method. The inter-frame prediction unit 171 outputs information related to the inter-frame prediction (such as the motion vector) to the entropy coding unit 130.

[0045] The intra-frame prediction unit 172 performs intra-frame prediction using spatial correlation within a frame. Specifically, the intra-frame prediction unit 172 generates an intra-frame prediction block by referring to decoded pixels located around the encoding target block in the decoded image stored in the memory 160, and outputs the generated intra-frame prediction block to the switching unit 173. The intra-frame prediction unit 172 selects an intra-frame prediction mode to be applied to the encoding target block from a plurality of intra-frame prediction modes, and predicts the encoding target block using the selected intra-frame prediction mode.

[0046] The switching section 173 switches the inter prediction block output from the inter prediction section 171 and the intra prediction block output from the intra prediction section 172 , and outputs one of the prediction blocks to the residual generation section 110 and the synthesis section 150 .

[0047] The encoding device 1 thus configured encodes a luminance block (hereinafter referred to as the target luminance block) and a chrominance block (hereinafter referred to as the target chrominance block). The following description of the encoding device 1 focuses on the encoding process of the target chrominance block.

[0048] The encoding device 1 according to this embodiment performs encoding processing in block units obtained by dividing an image composed of a luminance signal and a color difference signal. The encoding device 1 includes a prediction unit 170, a residual generation unit 110, and a transformation unit 121. The prediction unit 170 predicts a target color difference block, which is a block to be coded as a color difference signal, and generates a predicted color difference block. The residual generation unit 110 generates a color difference prediction residual representing the difference between the target color difference block and the predicted color difference block. The transformation unit 121 applies a transformation process to the color difference prediction residual and generates a color difference transform coefficient.

[0049] Next, the transform unit 121 according to this embodiment will be described. In this embodiment, the transform unit 121 can apply MTS, which switches the transform type for each encoding target block, not only to the luminance block but also to the chrominance block.

[0050] First, the transform processing for the luma signal will be described. When encoding a luma block, the transform unit 121 applies DCT2 in the horizontal and vertical directions when the default transform type is selected for the luma prediction residual, and signals flag information indicating that the transform type is the default mode via the entropy coding unit 130. On the other hand, when a non-default mode is selected, the transform unit 121 applies either DCT8 or DST7 in the horizontal and vertical directions to the luma prediction residual, and signals flag information indicating that one of the four combinations of DCT8 and DST7 has been selected, along with flag information indicating that the transform type is not the default mode, via the entropy coding unit 130.

[0051] Second, the conversion process for the color difference signal will be described. According to this embodiment, the conversion unit 121 controls the conversion process for the color difference signal based on the image signal format. A signal format refers to the method of extracting the color difference subsampling of the color difference signal through sampling. A signal format is sometimes referred to as a color difference format. In this embodiment, three signal formats are assumed: 4:4:4, 4:2:2, and 4:2:0.

[0052] Here, in the 4:2:0 signal format, the resolution of the color difference signal is half the vertical and horizontal resolution of the luminance signal, resulting in a low amount of high-frequency components in the color difference signal. In the 4:4:4 signal format, the resolution of the color difference signal is the same as that of the luminance signal, and the amount of information contained in the color difference signal is significantly greater than in the 4:2:0 format. Even in the 4:2:2 format, since the vertical resolution of the color difference signal and the luminance signal is equal, the amount of information contained in the color difference signal is greater, similar to the 4:4:4 format.

[0053] In this embodiment, when the signal format of the image to which the target chrominance block belongs is 4:2:0, the transform unit 121 applies DCT2 to the chrominance prediction residual in both the horizontal and vertical directions (ie, only the transform process in the default mode may be applied).

[0054] On the other hand, when the signal format of the image to which the target color difference block belongs is 4:2:2, the transform unit 121 applies DCT2 in the horizontal direction and any one of DCT2, DST7, and DCT8 in the vertical direction. When the signal format of the image to which the target color difference block belongs is 4:4:4, the transform unit 121 applies any one of DCT2, DST7, and DCT8 in the horizontal direction and vertical direction, respectively.

[0055] In this embodiment, the conversion unit 121 includes a conversion candidate determination unit 121 a and a conversion type determination unit 121 b .

[0056] The transform candidate determination unit 121a determines, for each target chrominance block, transform type candidates applicable to the transform process based on the signal format. Specifically, the transform candidate determination unit 121a determines horizontal transform type candidates and vertical transform type candidates based on the signal format. The horizontal transform type candidates are transform type candidates applicable to the transform process of the chrominance prediction residual in the horizontal direction, and the vertical transform type candidates are transform type candidates applicable to the transform process of the chrominance prediction residual in the vertical direction.

[0057] Figure 3 This is a diagram showing the conversion process for the color difference signal according to this embodiment.

[0058] like Figure 3 As shown, when the signal format is 4:2:0, the transform candidate determination unit 121a determines DCT2 as a horizontal transform type candidate and a vertical transform type candidate.

[0059] When the signal format is 4:2:2, the transform candidate determination unit 121a determines DCT2 as the horizontal transform type candidate, and determines DCT2, DST7, and DCT8 as the vertical transform type candidates.

[0060] When the signal format is 4:4:4, the transform candidate determination unit 121a determines DCT2, DST7, and DCT8 as horizontal transform type candidates and vertical transform type candidates. However, when the signal format is 4:4:4, the transform candidate determination unit 121a may determine not only all applicable transform type combinations (3 horizontal types × 3 vertical types = 9 types) as transform type candidates, but also a predetermined portion of combinations as transform type candidates. For example, Figure 3 As shown, the optional transform combinations (horizontal×vertical) can be set to (DCT2×DCT2), (DST7×DST7), (SDT7×DCT8), (DCT8×DST7), and (DCT8×DCT8).

[0061] The transform category determination unit 121b determines, for each object color difference block, a transform category to be applied to the transform processing in the horizontal direction from the horizontal transform category candidates (hereinafter referred to as the "horizontal transform category"), and determines, for each object color difference block, a transform category to be applied to the transform processing in the vertical direction from the vertical transform category candidates (hereinafter referred to as the "vertical transform category") .

[0062] For example, the transform type determination unit 121b performs a trial transform process based on the horizontal transform type candidates and the vertical transform type candidates to determine the combination of the horizontal transform type and the vertical transform type that achieves the highest coding efficiency. The transform unit 121 then applies the determined combination of the horizontal transform type and the vertical transform type to the chroma prediction residual to generate chroma transform coefficients, and outputs the chroma transform coefficients to the quantization unit 122.

[0063] The entropy coding unit 130 entropy codes the flag information indicating the horizontal transform type and the vertical transform type determined by the transform type determination unit 121b and transmits it to the decoding side. Here, the entropy coding unit 130 transmits the flag information indicating the horizontal transform type and the vertical transform type for each target chroma block.

[0064] like Figure 3 As shown, when the signal format is 4:2:0, there is only one type of selectable horizontal transform process and vertical transform process, so the entropy coding unit 130 does not need to transmit flag information indicating the transform process to be applied. On the other hand, when the signal format is 4:2:2 or 4:4:4, entropy coding is performed on flag information indicating the combination of transforms to be applied.

[0065] Furthermore, the entropy coding unit 130 performs entropy coding on flag information indicating the signal format of the original image and transmits the result to the decoding side.

[0066] Next, the operation flow related to the transform process in the encoding device 1 according to the present embodiment will be described. Figure 4 This is a diagram showing the operation flow related to the transformation processing of one target color difference block.

[0067] like Figure 4 As shown, in step S11, the transform candidate determination unit 121a determines transform type candidates applicable to the transform process (specifically, horizontal transform type candidates and vertical transform type candidates) based on the signal format. The transform candidate determination unit 121a may obtain information indicating the signal format from the block division unit 100 or from a pre-processing unit (not shown).

[0068] In step S12, the transform type determination unit 121b determines a horizontal transform type from the horizontal transform type candidates determined by the transform candidate determination unit 121a, and determines a vertical transform type from the vertical transform type candidates determined by the transform candidate determination unit 121a.

[0069] In step S13 , the transform unit 121 generates chroma transform coefficients by applying the combination of the horizontal transform type and the vertical transform type determined by the transform type determination unit 121 b to the chroma prediction residual output by the residual generation unit 110 , and outputs the chroma transform coefficients to the quantization unit 122 .

[0070] In step S14, the entropy coding unit 130 performs entropy coding on flag information indicating the combination of the horizontal transform type and the vertical transform type determined by the transform type determination unit 121b, and outputs the result as a stream (sent to the decoding side). Furthermore, the entropy coding unit 130 performs entropy coding on the quantized chroma transform coefficients output by the quantization unit 122, and outputs the result as a stream.

[0071] As described above, the encoding device 1 according to this embodiment controls the transform type applied to the transform process of the chrominance prediction residual according to the signal format of the image, thereby improving the encoding efficiency of the chrominance signal.

[0072] (Decoding device)

[0073] Next, the structure of the decoding device according to the present embodiment will be mainly described in terms of differences from the encoding device 1 . Figure 5 2 is a diagram showing the structure of the decoding device 2 according to this embodiment.

[0074] like Figure 5 As shown, the decoding device 2 includes an entropy decoding unit 200 , an inverse quantization and inverse transformation unit 210 , a synthesis unit 220 , a memory 230 , and a prediction unit 240 .

[0075] The entropy decoding unit 200 decodes the coded data and outputs the quantized transform coefficients corresponding to the decoding target block to the inverse quantization and inverse transformation unit 210. Furthermore, the entropy decoding unit 200 obtains information regarding the transform process and outputs this information to the inverse quantization and inverse transformation unit 210 (inverse transformation unit 212). This information regarding the transform process includes flag information indicating the combination of the horizontal transform type and the vertical transform type, as described above. Furthermore, the entropy decoding unit 200 obtains information regarding the prediction process and outputs this information to the prediction unit 240.

[0076] The inverse quantization and inverse transformation unit 210 performs inverse quantization and inverse transformation in units of blocks. The inverse quantization and inverse transformation unit 210 includes an inverse quantization unit 211 and an inverse transformation unit 212.

[0077] The inverse quantization unit 211 performs an inverse quantization process corresponding to the quantization process performed by the quantization unit 122 of the encoding device 1. The inverse quantization unit 211 inversely quantizes the quantized transform coefficients output by the entropy decoding unit 200 using a quantization parameter (Qp) and a quantization matrix to restore the transform coefficients of the decoding target block, and outputs the restored transform coefficients to the inverse transform unit 212.

[0078] The inverse transform unit 212 performs an inverse transform process corresponding to the transform process performed by the transform unit 121 of the encoding device 1. The inverse transform unit 212 performs an inverse transform process on the transform coefficients output by the inverse quantization unit 211, restores the prediction residual, and outputs the restored prediction residual (restored prediction residual) to the synthesis unit 220.

[0079] The synthesis unit 220 synthesizes the prediction residual output by the inverse transform unit 212 and the prediction block output by the prediction unit 240 on a pixel-by-pixel basis to decode (reconstruct) the original block and outputs the decoded block to the memory 230 .

[0080] Memory 230 stores the decoded blocks output by synthesis unit 220, storing the decoded blocks as decoded images in frame units. Memory 230 outputs the decoded blocks or decoded images to prediction unit 240. Memory 230 also outputs the decoded images in frame units to the outside of decoding device 2. A loop filter may be inserted between synthesis unit 220 and memory 230.

[0081] The prediction unit 240 performs prediction in block units and includes an inter prediction unit 241 , an intra prediction unit 242 , and a switching unit 243 .

[0082] The inter-frame prediction unit 241 performs inter-frame prediction using inter-frame correlation. Specifically, based on the information on inter-frame prediction (e.g., motion vector information) output from the entropy decoding unit 200, the inter-frame prediction unit 241 uses the decoded image stored in the memory 230 as a reference image to predict the encoding target block and generate an inter-frame prediction block. The generated inter-frame prediction block is then output to the switching unit 243.

[0083] The intra prediction unit 242 performs intra prediction using spatial correlation within a frame. Specifically, the intra prediction unit 242 uses the intra prediction mode corresponding to the information on intra prediction (e.g., intra prediction mode information) output by the entropy decoding unit 200, references decoded pixels located around the encoding target block in the decoded image stored in the memory 230, generates an intra prediction block, and outputs the generated intra prediction block to the switching unit 243.

[0084] The switching section 243 switches the inter prediction block output from the inter prediction section 241 and the intra prediction block output from the intra prediction section 242 , and outputs one of the prediction blocks to the synthesis section 220 .

[0085] The decoding device 2 thus configured decodes the luminance block (target luminance block) and the chrominance block (target chrominance block) as the decoding target blocks. The following description of the decoding device 2 focuses on the decoding process of the target chrominance block.

[0086] The decoding device 2 according to this embodiment performs decoding processing in block units obtained by dividing an image composed of a luminance signal and a color difference signal. The decoding device 2 includes a prediction unit 240, which predicts a target color difference block and generates a predicted color difference block, and an inverse transformation unit 212, which applies an inverse transformation to the color difference transform coefficients from the encoding side and generates a color difference prediction residual representing the difference between the target color difference block and the predicted color difference block.

[0087] Next, the inverse transform unit 212 according to this embodiment will be described. In this embodiment, the inverse transform unit 212 can apply MTS, which switches the transform type for each decoding target block, not only to the luminance block but also to the chrominance block.

[0088] First, the inverse transform processing of the luma signal will be described. During luma block decoding, if the entropy decoding unit 200 receives flag information indicating that the transform type is the default mode, the inverse transform unit 212 applies DCT2 to the luma transform coefficients in both the horizontal and vertical directions. On the other hand, if the entropy decoding unit 200 receives flag information indicating that the transform type is not the default mode and flag information indicating which combination of DCT8 and DST7 (a total of four modes) is selected, the inverse transform unit 212 applies either DCT8 or DST7 to the luma transform coefficients in both the horizontal and vertical directions.

[0089] Next, the inverse transform process for the color difference signal will be described. In this embodiment, the inverse transform unit 212 controls the inverse transform process for the color difference signal based on the image signal format. Here, the entropy decoder 200 obtains information indicating the signal format, and the entropy decoder 200 notifies the inverse transform unit 212 of the signal format.

[0090] In this embodiment, when the signal format of the image to which the target chrominance block belongs is 4:2:0, the inverse transform unit 212 applies DCT2 to the chrominance transform coefficients in both the horizontal and vertical directions (ie, only the inverse transform process in the default mode may be applied).

[0091] On the other hand, when the signal format of the image to which the target color difference block belongs is 4:2:2, the inverse transform unit 212 applies DCT2 in the horizontal direction and any one of DCT2, DST7, or DCT8 in the vertical direction based on the flag information indicating the combination of the horizontal transform type and the vertical transform type. When the signal format of the image to which the target color difference block belongs is 4:4:4, the inverse transform unit 212 applies any one of DCT2, DST7, or DCT8 in the horizontal direction and in the vertical direction based on the flag information indicating the combination of the horizontal transform type and the vertical transform type.

[0092] In this embodiment, the inverse transformation unit 212 includes a transformation candidate determination unit 212a and a transformation type determination unit 212b.

[0093] The transform candidate determination unit 212a determines, for each target color difference block, transform category candidates applicable to the inverse transform process based on the signal format. Specifically, the transform candidate determination unit 212a determines, based on the signal format, horizontal transform category candidates and vertical transform category candidates. The horizontal transform category candidates are transform category candidates applicable to the inverse transform process of the color difference transform coefficients in the horizontal direction, and the vertical transform category candidates are transform category candidates applicable to the inverse transform process of the color difference transform coefficients in the vertical direction. The method for determining the horizontal transform category candidates and the vertical transform category candidates is the same as the method for determining the horizontal transform category candidates in the encoding device 1 (refer to Figure 3 )same.

[0094] The transform category determination unit 212b determines, for each object color difference block, a horizontal transform category to be applied to the inverse transform processing in the horizontal direction from the horizontal transform category candidates based on the flag information indicating the combination of the horizontal transform category and the vertical transform category, and determines, for each object color difference block, a vertical transform category to be applied to the inverse transform processing in the vertical direction from the vertical transform category candidates.

[0095] Next, the operation flow related to the inverse transform process in the decoding device 2 according to the present embodiment will be described. Figure 6 This is a diagram showing the operation flow related to the inverse transform processing for one target color difference block.

[0096] like Figure 6 As shown, in step S21 , the transform candidate determination unit 212 a determines transform type candidates (specifically, horizontal transform type candidates and vertical transform type candidates) applicable to the inverse transform process based on the signal format.

[0097] In step S22, the transform type determination unit 212b determines a horizontal transform type from the horizontal transform type candidates determined by the transform candidate determination unit 212a, and determines a vertical transform type from the vertical transform type candidates determined by the transform candidate determination unit 212a, based on the flag information indicating the combination of the horizontal transform type and the vertical transform type. However, if the signal format is 4:2:0, the transform type determination unit 212b may apply DCT2 to the color difference transform coefficients in both the horizontal and vertical directions (default mode) without using the flag information.

[0098] In step S23 , the inverse transform unit 212 generates a chroma prediction residual by applying the combination of the horizontal transform type and the vertical transform type determined by the transform type determination unit 212 b to the chroma transform coefficient output by the inverse quantization unit 211 , and outputs the chroma prediction residual to the synthesis unit 220 .

[0099] As described above, the decoding device 2 according to this embodiment can improve the coding efficiency of the color difference signal by controlling the transform type applied to the inverse transform process of the color difference transform coefficients according to the signal format of the image.

[0100] (Variant 1)

[0101] In the above embodiment, the horizontal transform type candidates and the vertical transform type candidates may include transform skip. For example, the transform candidate determination units 121a and 212a may use (DCT2×DCT2), (transform skip×transform skip), (DST7×DST7), (DST7×DCT8), (DCT8×DST7), and (DCT8×DCT8) as combinations of horizontal transform type candidates and vertical transform type candidates (horizontal×vertical).

[0102] (Variant 2)

[0103] In the embodiment, the horizontal transform category and the vertical transform category may be determined by utilizing the correlation between the luminance block and the color difference block.

[0104] In this case, the transform category determination unit 121b of the encoding device 1 determines the transform category to be applied to the transform processing of the chrominance prediction residual from the transform category candidates determined by the transform candidate determination unit 121a, based on the transform category applied in the transform of the prediction residual of the luminance block (hereinafter referred to as the reference luminance block) corresponding to the position of the object chrominance block.

[0105] On the other hand, the transform category determination unit 212b of the decoding device 2 also determines the transform category to be applied to the inverse transform processing of the transform coefficients of the reference luminance block from the transform category candidates determined by the transform candidate determination unit 212a based on the transform category applied in the inverse transform processing of the transform coefficients of the reference luminance block.

[0106] For example, when the signal format is 4:4:4, the correlation between the luminance block and the chrominance block is high. Therefore, when the signal format is 4:4:4, the transform type determination unit 121b (and 212b) may determine the horizontal transform type and vertical transform type applied in the transform (or inverse transform) of the reference luminance block as the horizontal transform type and vertical transform type to be applied in the transform (or inverse transform) of the target chrominance block. However, if the combination of the candidate horizontal transform type and the candidate vertical transform type does not match the combination of the horizontal transform type and the vertical transform type applied in the transform (or inverse transform) of the reference luminance block, the transform type determination unit 121b (and 212b) may determine to use a predetermined combination of the horizontal transform type and the vertical transform type. When the signal format is 4:4:4, it is not necessary to transmit flag information indicating the combination of the horizontal transform type and the vertical transform type. Alternatively, only a copy flag indicating whether the combination of the horizontal transform type and the vertical transform type applied in the transform process of the reference luminance block is applied to the transform process of the target chrominance block may be transmitted, without transmitting flag information indicating the combination. In this case, if the copy flag is 1, the horizontal transform type and the vertical transform type applied in the transform process of the reference luminance block are determined as the horizontal transform type and the vertical transform type applied in the transform process of the target chrominance block, as described above. However, if the combination of candidate horizontal transform types and candidate vertical transform types for the target chrominance block does not match the combination of horizontal transform types and vertical transform types applied in the transform process (or inverse transform process) of the reference luminance block, the transform type determination unit 121b (and 212b) may determine a predetermined combination of horizontal transform types and vertical transform types. On the other hand, if the copy flag is 0, DCT2 may be applied to the transform coefficients of the target chrominance block in both the horizontal and vertical directions (default mode).

[0107] When the signal format is 4:2:2, the vertical correlation between luminance blocks and chrominance blocks is high. Therefore, when the signal format is 4:2:2, the transform type determination unit 121b (and 212b) may determine the vertical transform type applied in the transform (or inverse transform) of the reference luminance block as the vertical transform type to be applied in the transform (or inverse transform) of the target chrominance block. However, if the vertical transform type candidate for the target chrominance block does not match the vertical transform type applied in the transform (or inverse transform) of the reference luminance block, the transform type determination unit 121b (and 212b) may determine a predetermined vertical transform type. When the signal format is 4:2:2, it is not necessary to transmit flag information indicating the vertical transform type. Alternatively, it is possible to transmit only a copy flag indicating whether the vertical transform type applied in the transform of the reference luminance block is applied to the transform of the target chrominance block, without transmitting flag information indicating the vertical transform type. In this case, if the copy flag is 1, as described above, the vertical transform type applied in the transform process of the reference luminance block is determined as the vertical transform type to be applied in the transform process of the target chrominance block. However, if the vertical transform type candidate for the target chrominance block does not match the vertical transform type applied in the transform process (or inverse transform process) of the reference luminance block, the transform type determination unit 121b (and 212b) may determine a predetermined vertical transform type. On the other hand, if the copy flag is 0, DCT2 (default mode) may be applied to the vertical transform coefficients of the target chrominance block.

[0108] (Other Implementation Methods)

[0109] A program may be provided that causes a computer to execute the various processes performed by the encoding device 1. A program may also be provided that causes a computer to execute the various processes performed by the decoding device 2. The program may be recorded on a computer-readable medium. The program may be installed on a computer using the computer-readable medium. The computer-readable medium recording the program may be a non-transitory recording medium. The non-transitory recording medium is not particularly limited and may be, for example, a recording medium such as a CD-ROM or DVD-ROM.

[0110] Alternatively, the circuits that execute the various processes performed by encoding device 1 may be integrated, forming encoding device 1 from a semiconductor integrated circuit (chip set, SoC). Similarly, the circuits that execute the various processes performed by decoding device 2 may be integrated, forming decoding device 2 from a semiconductor integrated circuit (chip set, SoC).

[0111] Although the embodiments have been described in detail with reference to the drawings, the specific configuration is not limited to the configuration described above, and various design changes can be made without departing from the spirit of the invention.

[0112] This application claims the benefit of Japanese Patent Application No. 2019-235621 (filed on December 26, 2019), the entire contents of which are incorporated herein by reference.

Claims

1. An encoding device for performing encoding processing in block units obtained by dividing an image composed of a luminance signal and a color difference signal, characterized in that The encoding device has: a prediction unit that predicts a target color difference block as a coding target block of the color difference signal and generates a predicted color difference block, a residual generating unit for generating a chroma prediction residual, wherein the chroma prediction residual represents a difference between the target chroma block and the predicted chroma block; and a transform unit that applies a transform process to the chrominance prediction residual and generates a transform coefficient, and The conversion unit includes: a transform candidate determination unit that determines transform type candidates applicable to the transform process based on a signal format, and a transformation type determination unit, When the candidates for the transform type determined by the transform candidate determination unit include a transform type for a transform process applied to a reference luminance block, the transform type for the transform process applied to the reference luminance block is determined as the transform type applied to the transform process, the reference luminance block being a luminance block corresponding to a position of the target chrominance block; and When the candidates for the transform type determined by the transform candidate determination unit do not include a transform type applied to the transform process of the reference luminance block, a predetermined transform type is determined as the transform type applied to the transform process.

2. The encoding device according to claim 1, wherein When the combination of horizontal transform category candidates and vertical transform category candidates does not include the combination of horizontal transform category and vertical transform category applied to the transform processing of the reference luminance block, the transform unit determines a predetermined combination of horizontal transform category and vertical transform category as the horizontal transform category and vertical transform category applied to the transform processing.

3. The encoding device according to claim 2, characterized in that The encoding device further includes an entropy encoding unit, The transform candidate determination unit determines horizontal transform type candidates and vertical transform type candidates based on the signal format, wherein the horizontal transform type candidates are transform type candidates applicable to the transform process in the horizontal direction, and the vertical transform type candidates are transform type candidates applicable to the transform process in the vertical direction. The transform type determination unit determines a horizontal transform type to be applied to the transform process in the horizontal direction from among the horizontal transform type candidates, and determines a vertical transform type to be applied to the transform process in the vertical direction from among the vertical transform type candidates, and The entropy coding unit transmits flag information indicating the horizontal transform type and the vertical transform type determined by the transform type determination unit to a decoding side.

4. The encoding device according to claim 1, wherein The method further includes an entropy coding unit configured to transmit a copy flag to a decoding side, the copy flag indicating whether the transform type applied to the transform process of the reference luminance block is applied to the transform process of the target chrominance block.

5. A decoding device that performs decoding processing in block units obtained by dividing an image composed of a luminance signal and a color difference signal, characterized in that The decoding device comprises: a prediction unit that predicts a target color difference block, which is a decoding target block of the color difference signal, and generates a predicted color difference block; and an inverse transform unit that applies an inverse transform process to the transform coefficients from the encoding side to generate a chroma prediction residual representing a difference between the target chroma block and the predicted chroma block; The inverse transformation unit includes: a transform candidate determination unit that determines a candidate transform type applicable to the inverse transform process based on a signal format; as well as Transformation type determination unit: When the transform type candidates determined by the transform candidate determination unit include a transform type applied to an inverse transform process of a reference luminance block, the transform type applied to the inverse transform process of the reference luminance block is determined as the transform type applied to the inverse transform process, the reference luminance block being a luminance block corresponding to the position of the target chrominance block; and When the transform type applied to the inverse transform process of the reference luminance block does not exist among the transform type candidates determined by the transform candidate determination unit, a predetermined transform type is determined as the transform type applied to the inverse transform process.

6. The decoding device according to claim 5, characterized in that When the combination of horizontal transform category candidates and vertical transform category candidates does not include the combination of horizontal transform category and vertical transform category of the inverse transform processing applied to the reference luminance block, the inverse transform unit determines a predetermined combination of horizontal transform category and vertical transform category as the horizontal transform category and vertical transform category applied to the inverse transform processing.

7. The decoding device according to claim 5, wherein: The decoding device further includes an entropy decoding unit configured to obtain flag information indicating a horizontal transform type and a vertical transform type determined on the encoding side. The transform candidate determination unit determines horizontal transform type candidates applicable to the inverse transform process in the horizontal direction and vertical transform type candidates applicable to the inverse transform process in the vertical direction according to a signal format of the image. The transform type determination unit determines the transform type from the horizontal transform type candidates and determines the transform type from the vertical transform type candidates based on the flag information.

8. The decoding device according to claim 5, wherein: The entropy decoding unit is further provided, wherein the entropy decoding unit obtains a copy flag indicating whether the transform type applied to the reference luminance block is applied to the transform process of the target color difference block. When the copy flag indicates "no", the transform type determination unit determines a default mode as the transform type to be applied to the inverse transform process.

9. A program product, characterized in that The present invention includes a program that, when executed by a computer, causes the computer to function as the encoding device according to any one of claims 1 to 4.

10. A program product, characterized in that The present invention includes a program that, when executed by a computer, causes the computer to function as the decoding device according to any one of claims 5 to 8.

Images