Image decoding device, image decoding method, and program

Abstract

To execute processing on sub-blocks of a block in parallel even if ISP is applied to the block.SOLUTION: An image decoding device 200 comprises a prediction signal generation unit 242C that generates a prediction signal through intra prediction. In a block to which intra sub-partition is applied, when the prediction signal generation unit 242C generates prediction signals for all sub-blocks in the block, if the width of the sub-block is less than a first threshold, the prediction signal generation unit refers to pixels of a decoded block adjacent to the block, and if the width of the sub-block is equal to or more than the first threshold, refers to decoded pixels of an immediately previously processed sub-block. When the block size of the above-mentioned block is 32 pixels, the block is partitioned into two. When the block size of the above-mentioned block is larger than 32 pixels, the block is partitioned into four.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to an image decoding device, an image decoding method, and a program. [Background technology]

[0002] Conventionally, a technique called "Intra Sub-Partitions (ISP)" is known, in which a block is divided into sub-blocks and intra prediction is sequentially performed for each sub-block (see, for example, Non-Patent Documents 1 and 2). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2018-037936 [Non-patent literature]

[0004] [Non-Patent Document 1] Versatile Video Coding (Draft 4), JVET-M1001 [Non-patent document 2] CE3:Intra Sub-Partitions Coding Mode(Tests 1.1.1 and 1.1.2), JVET-M0102 [Non-patent document 3] Jiang Jie, Guo Balong, Mo Wei, Fan Kefeng, Block-Based Parallel Intra Prediction Scheme for HEVC, Journal of Multimedia, Vol.7, No.4, August 2012 Summary of the Invention [Problem to be solved by the invention]

[0005] However, in conventional ISP, when performing intra prediction of a sub-block, the decoded image of the sub-block processed immediately before in the same block is referenced. Therefore, the processing of each sub-block must be performed serially. In particular, when implemented in hardware, this processing can become a bottleneck in processing speed.

[0006] Therefore, the present invention has been made in consideration of the above-mentioned problems, and aims to provide an image decoding device, an image decoding method, and a program that can execute processing of each sub-block in parallel, even in a block to which an ISP is applied. [Means for solving the problem]

[0007] A first feature of the present invention is an image decoding device including a prediction signal generation unit that generates a prediction signal by intra prediction, wherein when generating prediction signals for all sub-blocks in a block to which intra sub-division is applied, the prediction signal generation unit refers to pixels of a decoded block adjacent to the block if the width of the sub-block is less than a first threshold, and refers to decoded pixels of the sub-block processed immediately before if the width of the sub-block is equal to or greater than the first threshold, and divides the block into two if the block size of the block is 32 pixels, and divides the block into four if the block size of the block is greater than 32 pixels.

[0008] A second feature of the present invention is an image coding device including a prediction signal generation unit that generates a prediction signal by intra prediction, wherein when generating prediction signals for all sub-blocks in a block to which intra sub-division is applied, the prediction signal generation unit refers to pixels of a decoded block adjacent to the block if the width of the sub-block is less than a first threshold, and refers to decoded pixels of the sub-block processed immediately before if the width of the sub-block is equal to or greater than the first threshold, and divides the block into two if the block size of the block is 32 pixels, and divides the block into four if the block size of the block is greater than 32 pixels.

[0009] A third feature of the present invention is an image decoding method including a step of generating a prediction signal by intra prediction, wherein in the step, when generating prediction signals for all sub-blocks in a block to which intra sub-division is applied, if the width of the sub-block is less than a first threshold, pixels of a decoded block adjacent to the block are referenced, and if the width of the sub-block is equal to or greater than the first threshold, decoded pixels of the sub-block processed immediately before are referenced, and if the block size of the block is 32 pixels, the block is divided into two, and if the block size of the block is greater than 32 pixels, the block is divided into four.

[0010] A fourth feature of the present invention is summarized as a program for use in an image decoding device, causing a computer to execute a step of generating a prediction signal by intra prediction, in which, in a block to which intra sub-division is applied, when generating prediction signals for all sub-blocks in the block, if the width of the sub-block is less than a first threshold, pixels of a decoded block adjacent to the block are referenced, and if the width of the sub-block is equal to or greater than the first threshold, decoded pixels of the sub-block processed immediately before are referenced, and if the block size of the block is 32 pixels, the block is divided into two, and if the block size of the block is greater than 32 pixels, the block is divided into four. [Effects of the Invention]

[0011] According to the present invention, it is possible to provide an image decoding device, an image decoding method, and a program that are capable of executing the processing of each sub-block in parallel even in a block to which ISP is applied. [Brief explanation of the drawings]

[0012] [Figure 1] 1 is a diagram illustrating an example of a configuration of an image processing system 10 according to an embodiment. [Figure 2] FIG. 2 is a diagram illustrating an example of functional blocks of an image decoding device 200 according to an embodiment. [Figure 3] FIG. 10 is a diagram showing an example of functional blocks of an intra-prediction unit 242 of an image decoding device 200 according to an embodiment. [Figure 4] 10 is a flowchart showing an example of a processing procedure of an intra prediction mode decoding unit 242B of the intra prediction unit 242 of the image decoding device 200 according to an embodiment. [Figure 5] FIG. 10 is a diagram illustrating an example of a method of dividing into sub-blocks used in an embodiment. [Figure 6] 10 is a flowchart showing an example of a procedure for determining the division direction and the number of divisions in an ISP used in an embodiment. [Figure 7]10 is a flowchart showing an example of a procedure for determining the division direction and the number of divisions in an ISP used in an embodiment. [Figure 8] 1 is a flowchart illustrating an example of a method for building an MPM list used in one embodiment. [Figure 9] 1 is a flowchart illustrating an example of a method for building an MPM list used in one embodiment. [Figure 10] 10 shows an arrangement of reference pixels during intra prediction when vertical 4-division is applied in ISP in one embodiment. [Figure 11] 10 shows an example of an arrangement of reference pixels during intra prediction when vertical 4-division is applied in ISP in one embodiment. [Figure 12] 1 is a diagram illustrating an example of functional blocks of an image encoding device 100 according to an embodiment. [Figure 13] FIG. 1 is a diagram showing an example of functional blocks of an intra-prediction unit 112 of an image encoding device 100 according to an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that the components in the following embodiments can be appropriately replaced with existing components, etc., and various variations, including combinations with other existing components, are possible. Therefore, the description of the following embodiments does not limit the content of the invention described in the claims.

[0014] (First embodiment) An image processing system 10 according to a first embodiment of the present invention will be described below with reference to Figures 1 to 13. Figure 1 is a diagram showing an image processing system 10 according to the present embodiment.

[0015] As shown in FIG. 1, an image processing system 10 includes an image encoding device 100 and an image decoding device 200.

[0016] The image encoding device 100 is configured to generate encoded data by encoding an input image signal. The image decoding device 200 is configured to generate an output image signal by decoding the encoded data.

[0017] Here, the coded data may be transmitted from the image coding device 100 to the image decoding device 200 via a transmission path. Alternatively, the coded data may be stored in a storage medium and then provided from the image coding device 100 to the image decoding device 200.

[0018] (Image decoding device 200) The image decoding device 200 according to this embodiment will be described below with reference to Fig. 2. Fig. 2 is a diagram showing an example of functional blocks of the image decoding device 200 according to this embodiment.

[0019] As shown in FIG. 2, the image decoding device 200 includes a decoding unit 210, an inverse transform / inverse quantization unit 220, an adder 230, an inter prediction unit 241, an intra prediction unit 242, an in-loop filtering unit 250, and a frame buffer 260.

[0020] The decoding unit 210 is configured to decode the coded data generated by the image coding device 100, and to decode the coefficient level values.

[0021] Here, for example, the decoding is the decoding of entropy coded data that assigns codes of different lengths based on the occurrence probability of coefficient level values.

[0022] The decoding unit 210 may be configured to obtain the control data by decoding the encoded data.

[0023] Here, the control data may include size data such as a coding block (CU: Coding Unit) size, a prediction block (PU: Prediction Unit) size, and a transform block (TU: Transform Unit) size.

[0024] The inverse transform / inverse quantization unit 220 is configured to perform inverse transform processing on the coefficient level values ​​output from the decoding unit 210. Here, the inverse transform / inverse quantization unit 220 may be configured to perform inverse quantization on the coefficient level values ​​prior to the inverse transform processing.

[0025] The adder 230 is configured to add the prediction signal to the prediction residual signal output from the inverse transform / inverse quantization unit 220 to generate a pre-filtered decoded signal, and output the pre-filtered decoded signal to the intra prediction unit 242 and the in-loop filter processing unit 250.

[0026] Here, the unfiltered decoded signal forms a reference block used by the intra prediction unit 242.

[0027] The inter prediction unit 241 is configured to generate a prediction signal by inter prediction (inter-frame prediction).

[0028] Specifically, the inter prediction unit 241 is configured to generate a prediction signal for each prediction block based on a motion vector decoded from encoded data and a reference signal included in a reference frame. The inter prediction unit 241 is configured to output the prediction signal to the adder 230.

[0029] The intra prediction unit 242 is configured to generate a prediction signal by intra prediction (prediction within a frame).

[0030] Specifically, the intra prediction unit 242 is configured to identify a reference block included in the target frame, and generate a prediction signal for each prediction block based on the identified reference block. The intra prediction unit 242 is configured to output the prediction signal to the adder 230.

[0031] The in-loop filtering unit 250 is configured to perform filtering on the unfiltered decoded signal output from the adder 230 , and to output the filtered decoded signal to the frame buffer 260 .

[0032] Here, for example, the filtering process is deblocking filtering process that reduces distortion occurring at the boundary portions of blocks (encoded blocks, predicted blocks, transformed blocks, or sub-blocks obtained by dividing these).

[0033] The frame buffer 260 is configured to store reference frames used by the inter prediction unit 241.

[0034] Here, the filtered decoded signal forms a reference frame used by the inter prediction unit 241.

[0035] (Intra prediction unit 242) The intra prediction unit 242 according to this embodiment will be described below with reference to Fig. 3. Fig. 3 is a diagram showing an example of functional blocks of the intra prediction unit 242 of the image decoding device 200 according to this embodiment.

[0036] As shown in FIG. 3, the intra prediction unit 242 includes an intra prediction mode decoding unit 242B and a prediction signal generation unit 242C.

[0037] The intra prediction unit 242 is an example of a prediction unit configured to generate a prediction signal by intra prediction (prediction within a frame).

[0038] The intra prediction mode decoding unit 242B decodes information necessary for performing intra prediction for each block.

[0039] 4 is a flowchart showing an example of the processing procedure of the intra prediction mode decoding unit 242 B. Hereinafter, an example of the processing procedure of the intra prediction mode decoding unit 242 B will be described using this flowchart.

[0040] 4, in step S41, the intra prediction mode decoding unit 242B decodes information (ref_idx) relating to pixel positions to be referenced when performing intra prediction. The value that "ref_idx" can take is either "0", "1", or "2".

[0041] When "ref_idx=0", the intra prediction mode decoding unit 242B performs intra prediction by referring to pixel values ​​located at the block boundary with the block currently being processed (hereinafter referred to as the block to be processed) among pixels of decoded blocks adjacent to the block currently being processed (hereinafter referred to as the block to be processed).

[0042] When "ref_idx=1", the intra prediction mode decoding unit 242B performs intra prediction by referring to a pixel that is one row or one column away from the pixel located at the block boundary described above.

[0043] When "ref_idx=2", the intra prediction mode decoding unit 242B performs intra prediction by referring to pixels three rows or three columns away from the pixel located at the block boundary described above.

[0044] In step S42, the intra prediction mode decoding unit 242B determines whether or not a predetermined condition 1 is satisfied.

[0045] Here, predetermined condition 1 may include a condition that the value of "ref_idx" is "0." Furthermore, predetermined condition 1 may include a condition that the size of the block to be processed (the number of pixels contained in the block) is equal to or greater than a predetermined number of pixels. This number of pixels may be set to, for example, 32 pixels. Furthermore, predetermined condition 1 may include a condition that the vertical size (number of pixels) of the block to be processed is equal to or greater than 8 pixels. Furthermore, predetermined condition 1 may include a condition that the horizontal size of the block to be processed is equal to or greater than 8 pixels.

[0046] If the predetermined condition 1 is satisfied, the process proceeds to step S43, and if the predetermined condition 1 is not satisfied, the process proceeds to step S46.

[0047] In step S43, the intra prediction mode decoding unit 242B decodes the "ISP mode flag."

[0048] When "ISP mode flag=0", the intra prediction mode decoding unit 242B does not apply ISP to the block to be processed, but when "ISP mode flag=1", it applies ISP to the block to be processed.

[0049] In step S44, the intra prediction mode decoding unit 242B determines whether or not predetermined condition 2 is satisfied. If predetermined condition 2 is satisfied, the process proceeds to step S45, and if predetermined condition 2 is not satisfied, the process proceeds to step S46. Here, the predetermined condition 2 may include a condition that the value of the "ISP mode flag" is "1." Furthermore, the predetermined condition 2 may include a condition that the shape of the processing target block is neither horizontally nor vertically long.

[0050] Note that "horizontally long" can be defined, for example, as the horizontal size (number of pixels) of the target block being at least twice as large as the vertical size (number of pixels).Similarly, "vertically long" can be defined, for example, as the vertical size of the target block being at least twice as large as the horizontal size.

[0051] For example, "horizontal" can be defined as the horizontal size of the block to be processed being larger than the vertical size, and "vertical" can be defined as the vertical size of the block to be processed being larger than the horizontal size.

[0052] If the predetermined condition 2 includes the above-mentioned condition, the intra prediction mode decoding unit 242B omits decoding of the "ISP split flag" in step S45 for a block determined to be horizontally or vertically oriented. In this case, the intra prediction mode decoding unit 242B can determine the ISP split direction according to the block shape, such as horizontal split for a horizontally oriented block and vertical split for a vertically oriented block.

[0053] In step S45, the intra prediction mode decoding unit 242B decodes the "ISP split flag."

[0054] Here, if "ISP split flag=0", it indicates that the split direction in the ISP is horizontal (i.e., horizontal split), and if "ISP split flag=1", it indicates that the split direction in the ISP is vertical (i.e., vertical split).

[0055] ISP is a technology that divides a block to be processed into a plurality of sub-blocks, and performs intra-prediction and generates a decoded image for each sub-block.

[0056] An example of such a division method is shown in Figure 5. As shown in Figure 5, in the case of horizontal division, in this embodiment, the image is divided into sub-blocks such that the number of pixels in the horizontal direction is the same as before the division, and the number of pixels in the vertical direction is 1 / 2 or 1 / 4 of before the division (horizontal 2-division or horizontal 4-division). Similarly, in the case of vertical division, the image is divided into sub-blocks such that the number of pixels in the vertical direction is the same as before the division, and the number of pixels in the horizontal direction is 1 / 2 or 1 / 4 of before the division (vertical 2-division or vertical 4-division).

[0057] Here, a procedure for determining the division direction and the number of divisions in the ISP used in this embodiment will be described with reference to Fig. 6. Fig. 6 is a flowchart showing an example of a procedure for determining the division direction and the number of divisions in the ISP used in this embodiment.

[0058] As shown in FIG. 6, in step S61, based on the "ISP split flag" decoded in step S45, if vertical splitting occurs, the procedure proceeds to step S62, and if horizontal splitting occurs, the procedure proceeds to step S63.

[0059] In step S62, the block size of the block to be processed is determined. Here, the block size can be defined as the number of pixels in the block, for example.

[0060] If the specified condition 1 includes a condition that the size of the block to be processed is 32 pixels or more, in step S62, the number of divisions in the ISP can be determined, for example, as "vertical 2 division" if the block size is 32 pixels, and "vertical 4 division" if the block size is greater than 32 pixels.

[0061] Similarly, in step S63, the number of divisions in the ISP can be determined, for example, "horizontal 2 divisions" when the block size is 32 pixels, and "horizontal 4 divisions" when the block size is greater than 32 pixels.

[0062] The division direction and number of divisions in the ISP can also be determined by the procedure shown in Fig. 7. Fig. 7 shows an example of the determination procedure when predetermined condition 1 includes both a condition that the vertical size of the processing target block is 8 pixels or more and a condition that the horizontal size of the processing target block is 8 pixels or more.

[0063] As shown in FIG. 7, in step S71, similarly to step S61, if vertical division is performed, the procedure proceeds to step S72, and if horizontal division is performed, the procedure proceeds to step S73.

[0064] In step S72, the number of divisions in the ISP can be determined such that if the horizontal size of the block to be processed is 8 pixels, it is "vertical 2 divisions", and if the horizontal size of the block to be processed is greater than 8 pixels, it is "vertical 4 divisions".

[0065] Here, if the horizontal and vertical sizes of a block are defined as being only powers of two in number of pixels, then the above-mentioned size being greater than 8 pixels is equivalent to the size being 16 pixels or greater.

[0066] Similarly, in step S73, the number of divisions in the ISP can be determined such that if the vertical size of the block to be processed is 8 pixels, it is "horizontal 2 divisions", and if the vertical size of the block to be processed is greater than 8 pixels, it is "horizontal 4 divisions".

[0067] By adopting the procedure shown in Figure 7, it is possible to perform division so that the horizontal and vertical sizes of the divided sub-blocks are always at least four pixels. Furthermore, if the minimum horizontal and vertical sizes without using an ISP are four pixels, then by performing the above-described processing, it is guaranteed that the size of the sub-blocks will be at least the minimum size without using an ISP (the minimum size of a block without division). As a result, the processing complexity when using an ISP can be kept below the processing complexity when not using an ISP.

[0068] Note that the procedure shown in FIG. 7 is merely an example, and the same effect can be achieved as long as the division method is such that the size of the sub-blocks after division is always equal to or greater than the minimum size that would be obtained without division.

[0069] As described above, after the intra prediction mode decoding unit 242B decodes the "ISP split flag" indicating the split direction of the ISP, the process proceeds to step S46.

[0070] In step S46, the intra prediction mode decoding unit 242B determines whether predetermined condition 3 is satisfied.

[0071] Here, the predetermined condition 3 may include a condition that the value of the "ISP mode flag" is "0." When the intra prediction mode decoding unit 242B does not decode the "ISP mode flag," it can consider the value of the "ISP mode flag" to be "0."

[0072] If the predetermined condition 3 is satisfied, the process proceeds to step S47, and if the predetermined condition 3 is not satisfied, the process proceeds to step S48.

[0073] In step S47, the intra prediction mode decoding unit 242B decodes the "MPM flag."

[0074] Here, "MPM flag=1" means that the intra prediction mode of the block to be processed is one of the prediction modes present in the MPM list, which will be described later.

[0075] On the other hand, if "MPM flag=0", this means that a prediction mode that does not exist in the MPM list is to be applied to the current block.

[0076] It should be noted that if the "MPM flag" is not decoded, that is, if the predetermined condition 3 is not satisfied, for example, the intra prediction mode decoding unit 242B can consider the value of the "MPM flag" to be "1".

[0077] In step S48, the intra prediction mode decoding unit 242B decodes the intra prediction mode based on the information decoded up to that point. The intra prediction mode includes, for example, a DC mode, a planar mode, a directional prediction mode, etc., as described in Non-Patent Document 1.

[0078] To decode the intra prediction mode, the intra prediction mode decoding unit 242B first creates an MPM list. The MPM list can be created using a known method such as that disclosed in Non-Patent Document 1. The MPM list creation method may be switched depending on the value of the above-mentioned "ISP mode flag."

[0079] Fig. 8 is a flowchart showing an example of a method for creating an MPM list when the value of the "ISP mode flag" is "1." As shown in Fig. 8, a default mode corresponding to the division direction is defined in advance.

[0080] Next, the intra prediction mode decoding unit 242B constructs an MPM list using the intra prediction modes used in the two decoded blocks A and B adjacent to the current block.

[0081] Here, the decoded block A is the block adjacent to the target block above, and the decoded block B is the block adjacent to the target block on the left. The exact method for determining the decoded blocks A and B can be the method described in Non-Patent Document 1.

[0082] FIG. 8 shows an example of the procedure for constructing an MPM list when the intra prediction modes used in the decoded blocks A and B are both directional predictions but have different prediction directions.

[0083] If the prediction direction used in the decoded block A has a smaller angular difference from the default mode than the prediction direction used in the decoded block AB, the intra-prediction mode decoding unit 242B sets the prediction mode used in the decoded block A to position 0 in the MPM list, and sets the prediction mode used in the decoded block B to position 1 in the MPM list.

[0084] Thereafter, the intra prediction mode decoding unit 242B sets prediction modes that are slightly different in angle from the prediction mode of the decoded block A to the second and third positions in the MPM list, and sets prediction modes that are slightly different in angle from the prediction mode of the decoded block B to the fourth and fifth positions in the MPM list.

[0085] Here, A and B in FIG. 8 are defined by numerical values ​​corresponding to the angle of the prediction direction, with larger numerical values ​​indicating larger angles and smaller numerical values ​​indicating smaller angles.

[0086] Similarly, if the prediction direction used in decoded block B has a smaller angular difference from the default mode than the prediction direction used in decoded block A, the intra-prediction mode decoding unit 242B sets the prediction mode used in decoded block B to number 0 in the MPM list, and sets the prediction mode used in decoded block A to number 1 in the MPM list.

[0087] Thereafter, the intra prediction mode decoding unit 242B sets prediction modes that are slightly different in angle from the prediction mode of the decoded block B to the second and third positions in the MPM list, and sets prediction modes that are slightly different in angle from the prediction mode of the decoded block A to the fourth and fifth positions in the MPM list.

[0088] If code words are assigned so that prediction modes with smaller index (Idx) values ​​in the MPM list can be encoded with less code, setting the index of modes with small angle differences from the default mode to be smaller in this way improves encoding efficiency, as the prediction mode for the prediction direction appropriate for the split direction can be expressed with less code.

[0089] Note that the method for constructing an MPM list described here is merely an example. Similar effects can be achieved by using a method for constructing an MPM list that takes into account the division direction and reduces the index of the mode that is considered more appropriate.

[0090] In the example of FIG. 8, the default mode setting is defined based on the division method of the ISP, but as shown in the example of FIG. 9, it can also be defined based on the shape of the sub-blocks after division.

[0091] As shown in FIG. 9, a default mode is defined depending on whether the shape of the divided sub-blocks is horizontal, vertical, or other, and the method of constructing the MPM list can be switched depending on the default mode.

[0092] Here, horizontally long can be defined as, for example, the horizontal size (number of pixels) of the target block being at least twice as large as the vertical size (number of pixels).Similarly, vertically long can be defined as, for example, the vertical size of the target block being at least twice as large as the horizontal size. Furthermore, for example, horizontal length can be defined as the horizontal size of the target block being larger than the vertical size, and vertical length can be defined as the vertical size of the target block being larger than the horizontal size.

[0093] In this case, by including the case where the horizontal and vertical sizes are equal in either portrait or landscape, the default mode can be defined in two patterns as in the example of FIG.

[0094] In addition, in FIG. 9, the default mode is defined based on the shape of the block after division, but the same processing is possible even if the default mode is defined using the shape of the block before division.

[0095] As described above, after the intra prediction mode decoding unit 242B decodes the "MPM flag," the processing proceeds to step S48.

[0096] In step S48, if "MPM flag = 1", the intra prediction mode decoding unit 242B decodes the index of the prediction mode to be used in the block to be processed, and as described above, determines the prediction mode corresponding to the decoded index in the constructed MPM list as the prediction mode of the block to be processed.

[0097] On the other hand, in step S48, if "MPM flag = 0", the intra prediction mode decoding unit 242B decodes information regarding which prediction mode to use from among the prediction modes not included in the above-mentioned MPM list, and determines the decoded prediction mode as the prediction mode for the block to be processed.

[0098] Since the prediction mode used in the target block can be decoded through the above procedure, the process proceeds to step S49 and ends.

[0099] The prediction signal generation unit 242C shown in Fig. 3 is configured to generate a prediction signal based on the prediction mode of the block to be processed decoded by the intra prediction mode decoding unit 242B. Here, the prediction signal generation method can use, for example, a known method described in Non-Patent Document 1.

[0100] FIG. 10 shows an example of an arrangement of reference pixels during intra prediction when vertical 4-division is applied in ISP.

[0101] In the ISP, when a block T is divided vertically (when vertical division is performed), prediction processing and decoded image generation processing are performed sequentially starting from sub-block #1 on the left side.

[0102] As shown in Figure 10, when intra prediction is performed on the second sub-block #2 from the left, pixel X1 of the decoded block A adjacent to the upper side of sub-block #2 and decoded pixel Y1 of the sub-block #1 adjacent to the left side of sub-block #2 are referenced.

[0103] The arrangement of reference pixels shown in Fig. 10 is merely an example. For example, when intra prediction is performed on sub-block #2 in vertical division, pixels other than pixel Y1 in the arrangement shown in Fig. 10 may be referenced as long as they are decoded pixels in sub-block #1.

[0104] Similarly, in the ISP, when a block is divided in the horizontal direction (when horizontal division is performed), prediction processing and decoded image generation processing are performed sequentially from the upper sub-block.

[0105] Here, the positions of the reference pixels when intra prediction is performed for each sub-block #1 to #4 can be, for example, pixel X2 of the decoded block A adjacent to the upper side of sub-block #1 to #4 (block T) and pixel Y2 of the decoded block B / C adjacent to the left side of sub-block #1 to #4 (block T), as shown in Figure 11.

[0106] That is, the decoded pixels of the sub-blocks #1 to #3 in the current block T may not be referenced for intra prediction of the second and subsequent sub-blocks #2 to #4.

[0107] The pixel arrangement shown in Fig. 11 is merely an example. Pixels in a decoded block adjacent to the current block T may be referenced other than pixels X2 / Y2 in the arrangement shown in Fig. 11.

[0108] 11 shows an example in which vertical division is performed, but the same reference pixel arrangement can also be used in the case of horizontal division. In this way, by not referring to the decoded pixels in the block, the prediction process and the decoded image generation process for each sub-block can be performed in parallel.

[0109] In this embodiment, the prediction modes of the sub-blocks in the same block are the same. Therefore, the generation process of the prediction signal when the reference pixel arrangement shown in Figure 11 is used can be the same as the generation process of the normal intra-prediction signal without dividing the target block T into sub-blocks.

[0110] Therefore, by adopting the reference pixel arrangement shown in Fig. 11, it is possible to realize a configuration in which the process of generating a prediction signal is performed on a block-by-block basis, regardless of whether the ISP is appropriate. In such a case, the process of generating a decoded image in an ISP-applied block (the process of generating a pre-filtering decoded signal) can be performed in parallel for each sub-block.

[0111] The arrangement of the reference pixels can also be changed depending on the block size. For example, the arrangement of the reference pixels can be changed depending on whether the width and height of the sub-block are less than a predetermined threshold.

[0112] For example, as shown in Figure 6, if the width of the sub-block is less than four pixels, the reference pixel arrangement shown in Figure 11 can be used, and if the width of the sub-block is four pixels or more, the reference pixel arrangement shown in Figure 10 can be used.

[0113] Similarly, for example, if the height of the sub-block is less than four pixels, the reference pixels can be arranged as shown in Figure 11, and if the height of the sub-block is four pixels or more, the reference pixels can be arranged as shown in Figure 10.

[0114] That is, for example, when the size of sub-blocks #1 to #4 is less than a first threshold, the prediction signal generation unit 242C may be configured to refer to pixels (e.g., reference pixels X2 / Y2) of decoded blocks A to C adjacent to the target block T, as shown in FIG. 11, when generating prediction signals for all sub-blocks #1 to #4 in the target block T.

[0115] On the other hand, for example, when the size of sub-block #2 is equal to or greater than the first threshold, the prediction signal generation unit 242C may be configured to refer to a decoded pixel (e.g., reference pixel Y1) of the sub-block #1 processed immediately before when generating a prediction signal for sub-block #2, as shown in FIG. 10.

[0116] In addition, when at least one of the width or height of the sub-block is less than the first threshold, the prediction signal generation unit 242C may be configured to refer to pixels (e.g., reference pixels X2 / Y2) of decoded blocks A to C adjacent to the target block T, as shown in Figure 11, when generating prediction signals for all sub-blocks #1 to #4 in the target block T.

[0117] On the other hand, for example, when both the width and height of sub-block #2 are equal to or greater than the first threshold, the prediction signal generation unit 242C may be configured to refer to a decoded pixel (e.g., reference pixel Y1) of the sub-block #1 processed immediately before when generating a prediction signal for sub-block #2, as shown in FIG. 10.

[0118] Furthermore, when the widths of the sub-blocks #1 to #4 are less than the first threshold, the prediction signal generation unit 242C may be configured to refer to pixels (e.g., reference pixels X2 / Y2) of decoded blocks A to C adjacent to the target block T, as shown in FIG. 11, when generating prediction signals for all sub-blocks #1 to #4 in the target block T.

[0119] On the other hand, for example, when the width of sub-block #2 is equal to or greater than the first threshold, the prediction signal generation unit 242C may be configured to refer to a decoded pixel (e.g., reference pixel Y1) of the sub-block #1 processed immediately before when generating a prediction signal for sub-block #2, as shown in FIG. 10.

[0120] In this way, by switching the arrangement of reference pixels depending on the block size, for example, in small sub-blocks, the reference pixels can be arranged in a way that allows parallel processing, thereby preventing an increase in implementation difficulty, and in large sub-blocks, coding efficiency can be improved by referring to the decoded pixels of the previous sub-block.

[0121] Parallel processing techniques for intra prediction are also disclosed in, for example, Patent Document 1 and Non-Patent Document 3. These Non-Patent Documents support the case where different prediction modes are used for processing units corresponding to the sub-blocks according to the present embodiment.

[0122] On the other hand, in this embodiment, the prediction mode of each sub-block in the same block is common.By adopting this configuration, the generation process of the predicted value is exactly the same when the block is divided into sub-blocks and when the block is not divided into sub-blocks.Therefore, it is not necessary to implement different processes depending on whether the block is divided or not, so that it is possible to prevent the difficulty of implementation, especially in hardware, from increasing.

[0123] (Image encoding device 100) The image encoding device 100 according to this embodiment will be described below with reference to Fig. 12. Fig. 12 is a diagram showing an example of functional blocks of the image encoding device 100 according to this embodiment.

[0124] As shown in FIG. 12 , the image encoding device 100 includes an inter prediction unit 111, an intra prediction unit 112, a subtractor 121, an adder 122, a transform / quantization unit 131, an inverse transform / inverse quantization unit 132, an encoding unit 140, an in-loop filter processing unit 150, and a frame buffer 160.

[0125] Like the inter prediction unit 241, the inter prediction unit 111 is configured to generate a prediction signal by inter prediction (inter-frame prediction).

[0126] Specifically, the inter prediction unit 111 is configured to identify a reference block included in the reference frame by comparing the frame to be coded (hereinafter referred to as the target frame) with a reference frame stored in the frame buffer 160, and to determine a motion vector for the identified reference block.

[0127] Furthermore, the inter prediction unit 111 is configured to generate, for each prediction block, a prediction signal included in the prediction block based on the reference block and the motion vector. The inter prediction unit 111 is configured to output the prediction signal to the subtractor 121 and the adder 122. Here, the reference frame is a frame different from the target frame.

[0128] Like the intra prediction unit 242, the intra prediction unit 112 is configured to generate a prediction signal by intra prediction (intra-frame prediction).

[0129] Specifically, the intra prediction unit 112 is configured to identify a reference block included in the target frame, and generate a prediction signal for each prediction block based on the identified reference block. The intra prediction unit 112 is also configured to output the prediction signal to the subtractor 121 and the adder 122.

[0130] Here, the reference block is a block that is referred to for a block to be predicted (hereinafter referred to as a target block). For example, the reference block is a block adjacent to the target block.

[0131] The subtractor 121 is configured to subtract the prediction signal from the input image signal and output the prediction residual signal to the transform / quantization unit 131. Here, the subtractor 121 is configured to generate a prediction residual signal that is the difference between the prediction signal generated by intra prediction or inter prediction and the input image signal.

[0132] The adder 122 is configured to add the prediction signal to the prediction residual signal output from the inverse transform / inverse quantization unit 132 to generate a pre-filter decoded signal, and to output the pre-filter decoded signal to the intra prediction unit 112 and the in-loop filter processing unit 150.

[0133] Here, the unfiltered decoded signal forms a reference block used by the intra prediction unit 112.

[0134] The transform / quantization unit 131 is configured to perform a transform process on the prediction residual signal and to obtain coefficient level values. Furthermore, the transform / quantization unit 131 may be configured to quantize the coefficient level values.

[0135] Here, the transform process is a process of transforming a prediction residual signal into a frequency component signal. In this transform process, a basis pattern (transform matrix) corresponding to a discrete cosine transform (DCT) or a basis pattern (transform matrix) corresponding to a discrete sine transform (DST) may be used.

[0136] The inverse transform and inverse quantization unit 132 is configured to perform inverse transform processing on the coefficient level values ​​output from the transform and quantization unit 131. Here, the inverse transform and inverse quantization unit 132 may be configured to perform inverse quantization on the coefficient level values ​​prior to the inverse transform processing.

[0137] Here, the inverse transform processing and inverse quantization are performed in the reverse order to the transform processing and quantization performed by the transform / quantization unit 131 .

[0138] The encoding unit 140 is configured to encode the coefficient level values ​​output from the transform / quantization unit 131 and output encoded data.

[0139] Here, for example, the coding is entropy coding that assigns codes of different lengths based on the probability of occurrence of coefficient level values.

[0140] The encoding unit 140 is also configured to encode control data used in the decoding process in addition to the coefficient level values.

[0141] As mentioned above, the control data may include size data such as the coding block size, the prediction block size, and the transform block size.

[0142] Similar to the in-loop filter processing unit 250, the in-loop filter processing unit 150 is configured to perform filtering on the unfiltered decoded signal output from the adder 122 and to output the filtered decoded signal to the frame buffer 160.

[0143] Here, for example, the filtering is deblocking filtering that reduces distortion occurring at the boundary portions of blocks (encoded blocks, predicted blocks, or transformed blocks).

[0144] The frame buffer 160 is configured to store reference frames used by the inter prediction unit 111.

[0145] Here, the filtered decoded signal forms a reference frame used in the inter prediction unit 111.

[0146] (Intra prediction unit 112) The intra prediction unit 112 according to this embodiment will be described below with reference to Fig. 13. Fig. 13 is a diagram showing an example of functional blocks of the intra prediction unit 112 of the image encoding device 112 according to this embodiment.

[0147] As shown in FIG. 13, the intra prediction unit 112 includes an intra prediction mode determination unit 112A, an intra prediction mode coding unit 112B, and a prediction signal generation unit 112C.

[0148] The intra prediction unit 112 is an example of a prediction unit configured to generate a prediction signal by intra prediction (prediction within a frame).

[0149] The intra prediction mode determination unit 112A is configured to determine information to be coded by the subsequent intra prediction mode coding unit 112B, such as the intra prediction mode of the block, whether or not ISP is applied, etc. As the determination method can be a known method, details thereof will be omitted.

[0150] The intra-prediction mode encoding unit 112B is configured to encode the prediction mode of the current block. The processing content is the same as that of the intra-prediction mode decoding unit 242B. Specifically, for example, the processing is the same as that of the intra-prediction mode decoding unit 242B shown in FIG. 4, except that the "decoding" part is changed to "encoding."

[0151] The prediction signal generation unit 112C is configured to generate a prediction signal of the block in accordance with the prediction mode coded in the intra prediction mode 112B. The prediction signal generation method is the same as that of the prediction signal generation unit 242C.

[0152] According to the image processing system 1 of this embodiment, even for a block to which ISP is applied, intra prediction can refer to pixels of adjacent blocks that have already been encoded (decoded), so that processing of each sub-block can be performed in parallel, thereby reducing the number of cycles required to process one block in hardware.

[0153] The image encoding device 100 and the image decoding device 200 described above may be realized as a program that causes a computer to execute each function (each step).

[0154] In the above embodiments, the present invention has been described as being applied to the image encoding device 100 and the image decoding device 200, but the present invention is not limited to this and can be similarly applied to an image encoding / decoding system having the functions of the image encoding device 100 and the image decoding device 200. [Explanation of symbols]

[0155] 10...Image processing system 100...Image encoding device 111, 241...Inter prediction section 112, 242...Intra prediction section 112A... Intra prediction mode determination unit 112B... Intra prediction mode encoding unit 112C, 242C...Prediction signal generation unit 121...Subtractor 122, 230...adder 131...Transformation and quantization unit 132, 220...Inverse transform and inverse quantization units 140...encoding section 150, 250...In-loop filter processing section 160, 260...frame buffer 200...Image decoding device 210...Decoding unit 242B... Intra prediction mode decoding unit

Claims

1. An image decoding device, a prediction signal generation unit that generates a prediction signal by intra prediction, the prediction signal generation unit, in generating prediction signals of all sub-blocks in a block to which intra sub-partition is applied, refers to pixels of a decoded block adjacent to the block if the width of the sub-block is less than a first threshold, and refers to decoded pixels of a sub-block processed immediately before if the width of the sub-block is equal to or greater than the first threshold; If the block size of the block is 32 pixels, the block is divided into two, 10. An image decoding device according to claim 9, wherein if the block size of the block is greater than 32 pixels, the block is divided into four.

2. An image encoding device, a prediction signal generation unit that generates a prediction signal by intra prediction, the prediction signal generation unit, in generating prediction signals of all sub-blocks in a block to which intra sub-partition is applied, refers to pixels of a decoded block adjacent to the block if the width of the sub-block is less than a first threshold, and refers to decoded pixels of a sub-block processed immediately before if the width of the sub-block is equal to or greater than the first threshold; If the block size of the block is 32 pixels, the block is divided into two, 10. An image coding device according to claim 9, wherein if the block size of the block is greater than 32 pixels, the block is divided into four.

3. generating a prediction signal by intra prediction, In the step, when generating prediction signals of all sub-blocks in a block to which intra sub-division is applied, if the width of the sub-block is less than a first threshold, pixels of a decoded block adjacent to the block are referenced, and if the width of the sub-block is equal to or greater than the first threshold, decoded pixels of the sub-block processed immediately before are referenced; If the block size of the block is 32 pixels, the block is divided into two, 2. An image decoding method, comprising: dividing the block into four blocks if the block size is greater than 32 pixels.

4. A program for use in an image decoding device, the program being installed on a computer: Execute a step of generating a prediction signal by intra prediction, In the step, when generating prediction signals of all sub-blocks in a block to which intra sub-division is applied, if the width of the sub-block is less than a first threshold, pixels of a decoded block adjacent to the block are referenced, and if the width of the sub-block is equal to or greater than the first threshold, decoded pixels of the sub-block processed immediately before are referenced; If the block size of the block is 32 pixels, the block is divided into two parts, If the block size of the block is greater than 32 pixels, the block is divided into four.

Images