Image decoding device, image decoding method, and program

Abstract

To execute processing on sub-blocks of a block in parallel even if ISP is applied to the block.SOLUTION: An image decoding device 200 comprises a prediction signal generation unit 242C that generates a prediction signal through intra prediction. In a block to which intra sub-partition is applied, when the prediction signal generation unit 242C generates prediction signals for all sub-blocks in the block, if the width of the sub-block is less than a first threshold, the prediction signal generation unit refers to pixels of a decoded block adjacent to the block, and if the width of the sub-block is equal to or more than the first threshold, refers to decoded pixels of an immediately previously processed sub-block. When the block size of the above-mentioned block is 32 pixels, the block is partitioned into two. When the block size of the above-mentioned block is larger than 32 pixels, the block is partitioned into four.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to an image decoding device, an image decoding method, and a program. [Background technology]

[0002] Conventionally, a technique called "Intra Sub-Partitions (ISP)" is known in which a block is divided into sub-blocks and intra prediction is sequentially performed for each sub-block (see, for example, Non-Patent Documents 1 and 2). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2018-037936 A [Non-patent literature]

[0004] [Non-Patent Document 1] Versatile Video Coding (Draft 4), JVET-M1001 [Non-Patent Document 2] CE3:Intra Sub-Partitions Coding Mode(Tests 1.1.1 and 1.1.2), JVET-M0102 [Non-Patent Document 3] Jiang Jie, Guo Balong, Mo Wei, Fan Kefeng, Block-Based Parallel Intra Prediction Scheme for HEVC, Journal of Multimedia, Vol.7, No.4, August 2012 Summary of the Invention [Problem to be solved by the invention]

[0005] However, in conventional ISP, when performing intra prediction of a certain sub-block, the decoded image of the sub-block processed immediately before in the same block is referenced. Therefore, the processing of each sub-block needs to be executed serially. In particular, when implemented in hardware, there is a problem that such processing may become a bottleneck in the processing speed.

[0006] Therefore, the present invention has been made in consideration of the above-mentioned problems, and aims to provide an image decoding device, an image decoding method, and a program that can execute processing of each sub-block in parallel, even in a block to which an ISP is applied. [Means for solving the problem]

[0007] A first feature of the present invention is an image decoding device including a prediction signal generation unit that generates a prediction signal by intra prediction, wherein when generating prediction signals for all sub-blocks in a block to which intra sub-division is applied, the prediction signal generation unit refers to pixels of a decoded block adjacent to the block if the width of the sub-block is less than a first threshold, and refers to decoded pixels of the sub-block processed immediately before if the width of the sub-block is equal to or greater than the first threshold, and divides the block into two if the block size of the block is 32 pixels, and divides the block into four if the block size of the block is greater than 32 pixels.

[0008] A second feature of the present invention is an image coding device including a prediction signal generation unit that generates a prediction signal by intra prediction, wherein when generating prediction signals for all sub-blocks in a block to which intra sub-division is applied, the prediction signal generation unit refers to pixels of a decoded block adjacent to the block if the width of the sub-block is less than a first threshold, and refers to decoded pixels of the sub-block processed immediately before if the width of the sub-block is equal to or greater than the first threshold, and divides the block into two if the block size of the block is 32 pixels, and divides the block into four if the block size of the block is greater than 32 pixels.

[0009] A third feature of the present invention is an image decoding method comprising the steps of: generating a prediction signal by intra prediction; in the step, in generating prediction signals for all sub-blocks in a block to which intra sub-division is applied, if the width of the sub-block is less than a first threshold, pixels of a decoded block adjacent to the block are referenced; if the width of the sub-block is equal to or greater than the first threshold, decoded pixels of the sub-block processed immediately before are referenced; if the block size of the block is 32 pixels, the block is divided into two; and if the block size of the block is greater than 32 pixels, the block is divided into four.

[0010] A fourth feature of the present invention is a program for use in an image decoding device, the program causing a computer to execute a step of generating a prediction signal by intra prediction, in which, in a block to which intra sub-division is applied, when generating prediction signals for all sub-blocks in the block, if the width of the sub-block is less than a first threshold, pixels of a decoded block adjacent to the block are referenced, and if the width of the sub-block is equal to or greater than the first threshold, decoded pixels of the sub-block processed immediately before are referenced, and if the block size of the block is 32 pixels, the block is divided into two, and if the block size of the block is greater than 32 pixels, the block is divided into four. Effect of the Invention

[0011] According to the present invention, it is possible to provide an image decoding device, an image decoding method, and a program that are capable of executing the processing of each sub-block in parallel even in a block to which an ISP is applied. [Brief description of the drawings]

[0012] [Figure 1] FIG. 1 illustrates an example of a configuration of an image processing system 10 according to an embodiment. [Diagram 2] FIG. 2 is a diagram illustrating an example of functional blocks of an image decoding device 200 according to an embodiment. [Diagram 3] FIG. 1 is a diagram showing an example of functional blocks of an intra-prediction unit 242 of an image decoding device 200 according to an embodiment. [Figure 4] 13 is a flowchart showing an example of a processing procedure of an intra-prediction mode decoding unit 242B of the intra-prediction unit 242 of the image decoding device 200 according to an embodiment. [Diagram 5] FIG. 13 is a diagram showing an example of a method of dividing into sub-blocks used in an embodiment. [Figure 6] 11 is a flowchart showing an example of a procedure for determining a division direction and a division number in an ISP used in an embodiment. [Figure 7]11 is a flowchart showing an example of a procedure for determining a division direction and a division number in an ISP used in an embodiment. [Figure 8] 1 is a flowchart illustrating an example of a method for building an MPM list used in one embodiment. [Figure 9] 1 is a flowchart illustrating an example of a method for building an MPM list used in one embodiment. [Figure 10] 13 shows an arrangement of reference pixels during intra prediction when vertical 4-division is applied in an ISP in one embodiment. [Figure 11] 13 illustrates an example of an arrangement of reference pixels during intra prediction when vertical 4-division is applied in an ISP in one embodiment. [Figure 12] FIG. 2 is a diagram illustrating an example of functional blocks of an image encoding device 100 according to an embodiment. [Figure 13] FIG. 1 is a diagram showing an example of functional blocks of an intra-prediction unit 112 of an image encoding device 100 according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Hereinafter, the embodiments of the present invention will be described with reference to the drawings. Note that the components in the following embodiments can be appropriately replaced with existing components, and various variations including combinations with other existing components are possible. Therefore, the description of the following embodiments does not limit the content of the invention described in the claims.

[0014] (First embodiment) An image processing system 10 according to a first embodiment of the present invention will be described below with reference to Fig. 1 to Fig. 13. Fig. 1 is a diagram showing an image processing system 10 according to the present embodiment.

[0015] As shown in FIG. 1, an image processing system 10 includes an image encoding device 100 and an image decoding device 200.

[0016] The image encoding device 100 is configured to generate encoded data by encoding an input image signal. The image decoding device 200 is configured to generate an output image signal by decoding the encoded data.

[0017] Here, the encoded data may be transmitted from the image encoding device 100 to the image decoding device 200 via a transmission path. Alternatively, the encoded data may be provided from the image encoding device 100 to the image decoding device 200 after being stored in a storage medium.

[0018] (Image Decoding Device 200) Hereinafter, the image decoding device 200 according to this embodiment will be described with reference to Fig. 2. Fig. 2 is a diagram showing an example of functional blocks of the image decoding device 200 according to this embodiment.

[0019] As shown in FIG. 2, the image decoding device 200 includes a decoding unit 210, an inverse transform / inverse quantization unit 220, an adder 230, an inter prediction unit 241, an intra prediction unit 242, an in-loop filtering unit 250, and a frame buffer 260.

[0020] The decoding unit 210 is configured to decode the coded data generated by the image coding device 100, and to decode the coefficient level values.

[0021] Here, for example, the decoding is of entropy coded data that assigns codes of different lengths based on the probability of occurrence of coefficient level values.

[0022] The decoding unit 210 may be configured to obtain the control data by a decoding process of the encoded data.

[0023] Here, the control data may include size data such as a coding block (CU: Coding Unit) size, a prediction block (PU: Prediction Unit) size, a transform block (TU: Transform Unit) size, and the like.

[0024] The inverse transform / inverse quantization unit 220 is configured to perform an inverse transform process on the coefficient level values ​​output from the decoding unit 210. Here, the inverse transform / inverse quantization unit 220 may be configured to perform an inverse quantization process on the coefficient level values ​​prior to the inverse transform process.

[0025] The adder 230 is configured to add the prediction signal to the prediction residual signal output from the inverse transform / inverse quantization unit 220 to generate a pre-filtered decoded signal, and output the pre-filtered decoded signal to the intra prediction unit 242 and the in-loop filter processing unit 250.

[0026] Here, the unfiltered decoded signal forms a reference block used by the intra prediction unit 242 .

[0027] The inter prediction unit 241 is configured to generate a prediction signal by inter prediction (inter-frame prediction).

[0028] Specifically, the inter prediction unit 241 is configured to generate a prediction signal for each prediction block based on a motion vector decoded from encoded data and a reference signal included in a reference frame. The inter prediction unit 241 is configured to output the prediction signal to the adder 230.

[0029] The intra prediction unit 242 is configured to generate a prediction signal by intra prediction (intra-frame prediction).

[0030] Specifically, the intra prediction unit 242 is configured to identify a reference block included in the target frame, and generate a prediction signal for each prediction block based on the identified reference block. The intra prediction unit 242 is configured to output the prediction signal to the adder 230.

[0031] The in-loop filtering unit 250 is configured to perform filtering on the unfiltered decoded signal output from the adder 230 , and to output the filtered decoded signal to the frame buffer 260 .

[0032] Here, for example, the filtering process is a deblocking filtering process that reduces distortion occurring at the boundary between blocks (encoded blocks, predicted blocks, transformed blocks, or subblocks obtained by dividing these blocks).

[0033] The frame buffer 260 is configured to store reference frames used by the inter prediction unit 241 .

[0034] Here, the filtered decoded signal forms a reference frame used by the inter prediction unit 241.

[0035] (Intra prediction unit 242) Hereinafter, the intra prediction unit 242 according to this embodiment will be described with reference to Fig. 3. Fig. 3 is a diagram showing an example of functional blocks of the intra prediction unit 242 of the image decoding device 200 according to this embodiment.

[0036] As shown in FIG. 3, the intra prediction unit 242 has an intra prediction mode decoding unit 242B and a prediction signal generation unit 242C.

[0037] The intra prediction unit 242 is an example of a prediction unit configured to generate a prediction signal by intra prediction (prediction within a frame).

[0038] The intra prediction mode decoding unit 242B decodes information necessary for performing intra prediction for each block.

[0039] 4 is a flowchart showing an example of a processing procedure of the intra prediction mode decoding unit 242B. Hereinafter, an example of a processing procedure of the intra prediction mode decoding unit 242B will be described with reference to this flowchart.

[0040] 4, in step S41, the intra prediction mode decoding unit 242B decodes information (ref_idx) on a pixel position to be referenced when performing intra prediction. The value that "ref_idx" can take is any one of "0", "1", and "2".

[0041] When "ref_idx=0", the intra prediction mode decoding unit 242B performs intra prediction by referring to pixel values ​​located at the block boundary with the block currently being processed (hereinafter, the target block) among pixels of a decoded block adjacent to the block currently being processed (hereinafter, the target block).

[0042] When "ref_idx=1", the intra prediction mode decoding unit 242B performs intra prediction by referring to a pixel that is one row or one column away from the pixel located at the above-mentioned block boundary.

[0043] When "ref_idx=2", the intra prediction mode decoding unit 242B performs intra prediction by referring to pixels three rows or three columns away from the pixel located at the above-mentioned block boundary.

[0044] In step S42, the intra prediction mode decoding unit 242B determines whether or not a predetermined condition 1 is satisfied.

[0045] Here, the predetermined condition 1 may include a condition that the value of "ref_idx" is "0". The predetermined condition 1 may also include a condition that the size of the block to be processed (the number of pixels contained in the block) is equal to or greater than a predetermined number of pixels. Such a number of pixels may be set to, for example, 32 pixels. The predetermined condition 1 may also include a condition that the vertical size (number of pixels) of the block to be processed is equal to or greater than 8 pixels. Furthermore, the predetermined condition 1 may also include a condition that the horizontal size of the block to be processed is equal to or greater than 8 pixels.

[0046] If the predetermined condition 1 is satisfied, the process proceeds to step S43, and if the predetermined condition 1 is not satisfied, the process proceeds to step S46.

[0047] In step S43, the intra prediction mode decoding unit 242B decodes the "ISP mode flag."

[0048] When "ISP mode flag=0", the intra-prediction mode decoding unit 242B does not apply the ISP to the block to be processed, and when "ISP mode flag=1", it applies the ISP to the block to be processed.

[0049] In step S44, the intra prediction mode decoding unit 242B determines whether or not a predetermined condition 2 is satisfied. If the predetermined condition 2 is satisfied, the process proceeds to step S45, and if the predetermined condition 2 is not satisfied, the process proceeds to step S46. Here, the predetermined condition 2 may include a condition that the value of the "ISP mode flag" is "1." Furthermore, the predetermined condition 2 may include a condition that the shape of the processing target block is neither horizontally nor vertically long.

[0050] Note that "landscape" can be defined as, for example, the horizontal size (number of pixels) of the target block being at least twice as large as the vertical size (number of pixels). Similarly, "portrait" can be defined as, for example, the vertical size of the target block being at least twice as large as the horizontal size.

[0051] For example, "landscape" can be defined as the horizontal size of the target block being larger than its vertical size, and "portrait" can be defined as the vertical size of the target block being larger than its horizontal size.

[0052] When the predetermined condition 2 includes the above-mentioned condition, the intra prediction mode decoding unit 242B omits decoding of the "ISP split flag" in step S45 for a block determined to be horizontally or vertically long. In this case, the intra prediction mode decoding unit 242B can determine the ISP split direction according to the block shape, such as horizontal division for a horizontally long block and vertical division for a vertically long block.

[0053] In step S45, the intra prediction mode decoding unit 242B decodes the "ISP split flag."

[0054] Here, when "ISP split flag=0", it indicates that the split direction in the ISP is horizontal (i.e., horizontal split), and when "ISP split flag=1", it indicates that the split direction in the ISP is vertical (i.e., vertical split).

[0055] ISP is a technique for dividing a processing target block into a plurality of sub-blocks, and performing intra prediction and generating a decoded image for each sub-block.

[0056] An example of such a division method is shown in Fig. 5. As shown in Fig. 5, in the case of horizontal division in this embodiment, the image is divided into sub-blocks in which the number of pixels in the horizontal direction is the same as before the division, and the number of pixels in the vertical direction is 1 / 2 or 1 / 4 of before the division (horizontal 2 division or horizontal 4 division). Similarly, in the case of vertical division, the image is divided into sub-blocks in which the number of pixels in the vertical direction is the same as before the division, and the number of pixels in the horizontal direction is 1 / 2 or 1 / 4 of before the division (vertical 2 division or vertical 4 division).

[0057] Here, a procedure for determining the division direction and the number of divisions in the ISP used in this embodiment will be described with reference to Fig. 6. Fig. 6 is a flowchart showing an example of a procedure for determining the division direction and the number of divisions in the ISP used in this embodiment.

[0058] As shown in FIG. 6, in step S61, based on the "ISP split flag" decoded in step S45, if vertical split is determined, the procedure proceeds to step S62, and if horizontal split is determined, the procedure proceeds to step S63.

[0059] In step S62, the block size of the block to be processed is determined. Here, the block size can be defined as the number of pixels in the block, for example.

[0060] If the specified condition 1 includes a condition that the size of the block to be processed is 32 pixels or more, in step S62, the number of divisions in the ISP can be determined, for example, as "vertical 2 division" when the block size is 32 pixels, and "vertical 4 division" when the block size is greater than 32 pixels.

[0061] Similarly, in step S63, the number of divisions in the ISP can be determined, for example, "horizontal 2 divisions" when the block size is 32 pixels, and "horizontal 4 divisions" when the block size is greater than 32 pixels.

[0062] The division direction and the number of divisions in the ISP can also be determined by the procedure shown in Fig. 7. Fig. 7 shows an example of the determination procedure when the predetermined condition 1 includes both a condition that the vertical size of the processing target block is 8 pixels or more and a condition that the horizontal size of the processing target block is 8 pixels or more.

[0063] As shown in FIG. 7, in step S71, in the same manner as in step S61, if vertical division is performed, the procedure proceeds to step S72, and if horizontal division is performed, the procedure proceeds to step S73.

[0064] In step S72, the number of divisions in the ISP can be determined such that if the horizontal size of the target block is 8 pixels, then "vertical 2 division" is selected, and if the horizontal size of the target block is greater than 8 pixels, then "vertical 4 division" is selected.

[0065] Here, if the horizontal and vertical sizes of a block are defined only as sizes in pixels that are powers of 2, then the above-mentioned size being greater than 8 pixels is equivalent to the size being 16 pixels or greater.

[0066] Similarly, in step S73, the number of divisions in the ISP can be determined such that if the vertical size of the block to be processed is 8 pixels, then "horizontal 2 division" is selected, and if the vertical size of the block to be processed is greater than 8 pixels, then "horizontal 4 division" is selected.

[0067] By adopting the procedure shown in Fig. 7, it is possible to ensure that the horizontal and vertical sizes of the sub-blocks after division are always 4 pixels or more. Furthermore, if the minimum horizontal and vertical sizes without using an ISP are 4 pixels, then by performing the above-described processing, it is guaranteed that the size of the sub-blocks will be equal to or greater than the minimum size without using an ISP (the minimum size of a block without division). As a result, it is possible to keep the processing complexity when using an ISP below that when not using an ISP.

[0068] It should be noted that the procedure shown in FIG. 7 is merely one example, and the same effect can be obtained as long as the division method is such that the size of the sub-blocks after division is always equal to or larger than the minimum size when no division is performed.

[0069] As described above, after the intra prediction mode decoding unit 242B decodes the "ISP split flag" indicating the split direction of the ISP, the process proceeds to step S46.

[0070] In step S46, the intra-prediction mode decoding unit 242B determines whether or not predetermined condition 3 is satisfied.

[0071] Here, the predetermined condition 3 may include a condition that the value of the "ISP mode flag" is "0." When the intra prediction mode decoding unit 242B does not decode the "ISP mode flag," it can consider the value of the "ISP mode flag" to be "0."

[0072] If the predetermined condition 3 is satisfied, the process proceeds to step S47, and if the predetermined condition 3 is not satisfied, the process proceeds to step S48.

[0073] In step S47, the intra prediction mode decoding unit 242B decodes the "MPM flag."

[0074] Here, "MPM flag=1" means that the intra prediction mode of the current block is one of the prediction modes present in the MPM list, which will be described later.

[0075] On the other hand, "MPM flag=0" means that a prediction mode that does not exist in the MPM list is applied to the current block.

[0076] It should be noted that if the "MPM flag" is not decoded, that is, if the predetermined condition 3 is not satisfied, for example, the intra prediction mode decoding unit 242B can regard the value of the "MPM flag" as being "1".

[0077] In step S48, the intra prediction mode decoding unit 242B decodes the intra prediction mode based on the information decoded up to that point. The intra prediction mode includes, for example, a DC mode, a planar mode, a directional prediction mode, and the like, as described in Non-Patent Document 1.

[0078] To decode the intra prediction mode, the intra prediction mode decoding unit 242B first creates an MPM list. The MPM list can be created using a known method such as that disclosed in Non-Patent Document 1. The MPM list creation method may be switched depending on the value of the above-mentioned "ISP mode flag."

[0079] Fig. 8 is a flowchart showing an example of a method for creating an MPM list when the value of the "ISP mode flag" is "1." As shown in Fig. 8, a default mode corresponding to the division direction is defined in advance.

[0080] Next, the intra-prediction mode decoding unit 242B constructs an MPM list using the intra-prediction modes used in the two blocks, the decoded blocks A and B, adjacent to the current block.

[0081] Here, the decoded block A is the adjacent block above the block to be processed, and the decoded block B is the adjacent block to the left of the block to be processed. For a strict method of determining the decoded blocks A and B, the method described in Non-Patent Document 1 can be used.

[0082] FIG. 8 shows an example of a procedure for constructing an MPM list when the intra prediction modes used in the decoded blocks A and B are both directional predictions with different prediction directions.

[0083] If the prediction direction used in the decoded block A has a smaller angular difference from the default mode than the prediction direction used in the decoded block AB, the intra-prediction mode decoding unit 242B sets the prediction mode used in the decoded block A to the 0th position in the MPM list, and sets the prediction mode used in the decoded block B to the 1st position in the MPM list.

[0084] Thereafter, the intra-prediction mode decoding unit 242B sets a prediction mode that differs slightly in angle from the prediction mode of the decoded block A to the second and third positions in the MPM list, and sets a prediction mode that differs slightly in angle from the prediction mode of the decoded block B to the fourth and fifth positions in the MPM list.

[0085] Here, A and B in FIG. 8 are defined as numerical values ​​corresponding to the angle of the prediction direction, with larger numerical values ​​indicating a larger angle and smaller numerical values ​​indicating a smaller angle.

[0086] Similarly, if the prediction direction used in decoded block B has a smaller angular difference from the default mode than the prediction direction used in decoded block A, the intra-prediction mode decoding unit 242B sets the prediction mode used in decoded block B to number 0 in the MPM list, and sets the prediction mode used in decoded block A to number 1 in the MPM list.

[0087] Thereafter, the intra-prediction mode decoding unit 242B sets a prediction mode that differs slightly in angle from the prediction mode of the decoded block B to the second and third positions in the MPM list, and sets a prediction mode that differs slightly in angle from the prediction mode of the decoded block A to the fourth and fifth positions in the MPM list.

[0088] If code words are assigned so that prediction modes with smaller index (Idx) values ​​in the MPM list can be encoded with less code, then by setting the index of the mode with a small angle difference from the default mode to a smaller value, the prediction mode for the prediction direction appropriate for the split direction can be expressed with less code, thereby improving encoding efficiency.

[0089] Note that the method for constructing an MPM list described here is merely one example. Similar effects can be achieved by using a method for constructing an MPM list that takes into account the division direction and reduces the index of the mode that is considered more appropriate.

[0090] In the example of FIG. 8, the setting of the default mode is defined according to the division method of the ISP, but as shown in the example of FIG. 9, it can also be defined based on the shape of the sub-blocks after division.

[0091] As shown in FIG. 9, a default mode is defined depending on whether the shape of the sub-blocks after division is horizontal, vertical, or other, and the method of constructing the MPM list can be switched depending on the default mode.

[0092] Here, the term "landscape" can be defined as, for example, the horizontal size (number of pixels) of the target block being at least twice as large as the vertical size (number of pixels).Similarly, the term "portrait" can be defined as, for example, the vertical size of the target block being at least twice as large as the horizontal size. Also, for example, landscape can be defined as the horizontal size of the target block being larger than the vertical size, and portrait can be defined as the vertical size of the target block being larger than the horizontal size.

[0093] In this case, by including the case where the horizontal and vertical sizes are equal in either portrait or landscape, the default mode can be defined in two patterns as in the example of FIG.

[0094] In addition, in FIG. 9, the default mode is defined based on the shape of the block after division, but the same processing is possible even if the default mode is defined using the shape of the block before division.

[0095] As described above, after the intra prediction mode decoding unit 242B decodes the "MPM flag", the process proceeds to step S48.

[0096] In step S48, if "MPM flag = 1", the intra-prediction mode decoding unit 242B decodes the index of the prediction mode to be used in the block to be processed, and determines the prediction mode corresponding to the decoded index in the constructed MPM list as described above, as the prediction mode of the block to be processed.

[0097] On the other hand, in step S48, if "MPM flag = 0", the intra prediction mode decoding unit 242B decodes information regarding which prediction mode to use among the prediction modes not included in the above-mentioned MPM list, and determines the decoded prediction mode as the prediction mode of the block to be processed.

[0098] Since the prediction mode used in the current block can be decoded through the above procedure, the process proceeds to step S49 and ends.

[0099] The prediction signal generating unit 242C shown in Fig. 3 is configured to generate a prediction signal based on the prediction mode of the processing target block decoded by the intra prediction mode decoding unit 242B. Here, the prediction signal generating method can use, for example, a known method described in Non-Patent Document 1.

[0100] FIG. 10 shows an example of an arrangement of reference pixels during intra prediction when vertical 4-division is applied in the ISP.

[0101] In the ISP, when a block T is divided in the vertical direction (when vertical division is performed), prediction processing and decoded image generation processing are performed sequentially from sub-block #1 on the left side.

[0102] As shown in Figure 10, when intra prediction is performed on the second sub-block #2 from the left, pixel X1 of the decoded block A adjacent to the upper side of sub-block #2 and pixel Y1 of the decoded sub-block #1 adjacent to the left side of sub-block #2 are referenced.

[0103] The arrangement of the reference pixels shown in Fig. 10 is merely an example. For example, when intra prediction is performed on subblock #2 in vertical division, pixels other than pixel Y1 in the arrangement shown in Fig. 10 may be referenced as long as they are decoded pixels in subblock #1.

[0104] Similarly, in the ISP, when a block is divided in the horizontal direction (when horizontal division is performed), prediction processing and decoded image generation processing are performed sequentially from the upper sub-block.

[0105] Here, the positions of the reference pixels when intra prediction is performed for each sub-block #1 to #4 may be, for example, pixel X2 of decoded block A adjacent to the upper side of sub-block #1 to #4 (block T) and pixel Y2 of decoded block B / C adjacent to the left side of sub-block #1 to #4 (block T), as shown in Figure 11.

[0106] In other words, the decoded pixels of the sub-blocks #1 to #3 in the current block T may not be referenced for intra prediction of the second and subsequent sub-blocks #2 to #4.

[0107] The pixel arrangement shown in Fig. 11 is merely an example. Pixels in a decoded block adjacent to the current block T may be referenced other than pixels X2 / Y2 in the arrangement shown in Fig. 11.

[0108] 11 shows an example in which vertical division is performed, but the same arrangement of reference pixels can be used in the case of horizontal division. In this way, by not referring to the decoded pixels in the block, the prediction process and the decoded image generation process for each sub-block can be performed in parallel.

[0109] In this embodiment, the prediction mode of each sub-block in the same block is common. Therefore, the generation process of the prediction signal when the reference pixel arrangement shown in Fig. 11 is used can be the same as the generation process of normal intra-prediction without dividing the processing target block T into sub-blocks.

[0110] Therefore, by adopting the arrangement of reference pixels shown in Fig. 11, it is possible to realize a configuration in which the generation process of a prediction signal is executed on a block-by-block basis, regardless of the suitability of the ISP. In such a case, the generation process of a decoded image in an ISP-applied block (the generation process of a pre-filtering decoded signal) can be executed in parallel for each sub-block.

[0111] The above-mentioned arrangement of reference pixels can also be switched depending on the block size. For example, the arrangement of reference pixels can be switched depending on whether the width and height of a sub-block are less than a predetermined threshold value.

[0112] For example, as shown in FIG. 6, when the width of the sub-block is less than four pixels, the reference pixels can be arranged as shown in FIG. 11, and when the width of the sub-block is four pixels or more, the reference pixels can be arranged as shown in FIG. 10.

[0113] Similarly, for example, if the height of the sub-block is less than four pixels, the reference pixels can be arranged as shown in FIG. 11, and if the height of the sub-block is four pixels or more, the reference pixels can be arranged as shown in FIG. 10.

[0114] That is, for example, when the size of sub-blocks #1 to #4 is less than a first threshold, the prediction signal generation unit 242C may be configured to refer to pixels (e.g., reference pixels X2 / Y2) of decoded blocks A to C adjacent to the current block T, as shown in FIG. 11, when generating prediction signals for all sub-blocks #1 to #4 in the current block T.

[0115] On the other hand, for example, when the size of sub-block #2 is equal to or greater than the first threshold, the prediction signal generation unit 242C may be configured to refer to a decoded pixel (e.g., reference pixel Y1) of the sub-block #1 processed immediately before when generating a prediction signal for sub-block #2, as shown in FIG. 10.

[0116] In addition, when at least one of the width or height of the sub-block is less than a first threshold, the prediction signal generation unit 242C may be configured to refer to pixels (e.g., reference pixels X2 / Y2) of decoded blocks A to C adjacent to the block T to be processed, as shown in FIG. 11, when generating prediction signals for all sub-blocks #1 to #4 in the block T to be processed.

[0117] On the other hand, for example, when both the width and height of sub-block #2 are equal to or greater than the first threshold, the prediction signal generation unit 242C may be configured to refer to a decoded pixel (e.g., reference pixel Y1) of the sub-block #1 processed immediately before when generating a prediction signal for sub-block #2, as shown in FIG. 10.

[0118] Furthermore, when the widths of sub-blocks #1 to #4 are less than a first threshold, the prediction signal generation unit 242C may be configured to refer to pixels (e.g., reference pixels X2 / Y2) of decoded blocks A to C adjacent to the current block T, as shown in FIG. 11, when generating prediction signals for all sub-blocks #1 to #4 in the current block T.

[0119] On the other hand, for example, when the width of sub-block #2 is equal to or greater than the first threshold, the prediction signal generation unit 242C may be configured to refer to a decoded pixel (e.g., reference pixel Y1) of the sub-block #1 processed immediately before when generating a prediction signal for sub-block #2, as shown in FIG. 10.

[0120] In this way, by switching the arrangement of reference pixels depending on the block size, for example, in small sub-blocks, an increase in implementation difficulty can be prevented by using an arrangement of reference pixels that can be processed in parallel, and in large sub-blocks, coding efficiency can be improved by referring to the decoded pixels of the previous sub-block.

[0121] Parallel processing techniques for intra prediction are also disclosed in, for example, Patent Document 1 and Non-Patent Document 3. These Non-Patent Documents support the case where different prediction modes are used for processing units corresponding to the sub-blocks according to the present embodiment.

[0122] On the other hand, in this embodiment, the prediction mode of each sub-block in the same block is common.By configuring in this way, the generation process of predicted value is exactly the same process when divided into sub-blocks and when not divided into sub-blocks.Therefore, it is not necessary to implement different processes depending on whether or not division is performed, so that it is possible to prevent the difficulty of implementation in hardware from increasing.

[0123] (Image encoding device 100) Hereinafter, the image encoding device 100 according to this embodiment will be described with reference to Fig. 12. Fig. 12 is a diagram showing an example of functional blocks of the image encoding device 100 according to this embodiment.

[0124] As shown in FIG. 12, the image encoding device 100 includes an inter prediction unit 111, an intra prediction unit 112, a subtractor 121, an adder 122, a transform / quantization unit 131, an inverse transform / inverse quantization unit 132, an encoding unit 140, an in-loop filter processing unit 150, and a frame buffer 160.

[0125] Like the inter prediction unit 241, the inter prediction unit 111 is configured to generate a prediction signal by inter prediction (inter-frame prediction).

[0126] Specifically, the inter prediction unit 111 is configured to identify a reference block contained in the reference frame by comparing a frame to be encoded (hereinafter, the target frame) with a reference frame stored in the frame buffer 160, and to determine a motion vector for the identified reference block.

[0127] Moreover, the inter prediction unit 111 is configured to generate a prediction signal included in the prediction block for each prediction block based on the reference block and the motion vector. The inter prediction unit 111 is configured to output the prediction signal to the subtractor 121 and the adder 122. Here, the reference frame is a frame different from the target frame.

[0128] Like the intra prediction unit 242, the intra prediction unit 112 is configured to generate a prediction signal by intra prediction (intra-frame prediction).

[0129] Specifically, the intra prediction unit 112 is configured to identify a reference block included in a target frame, and generate a prediction signal for each prediction block based on the identified reference block. The intra prediction unit 112 is also configured to output the prediction signal to the subtractor 121 and the adder 122.

[0130] Here, the reference block is a block that is referred to for a block to be predicted (hereinafter, a target block). For example, the reference block is a block adjacent to the target block.

[0131] The subtractor 121 is configured to subtract the prediction signal from the input image signal, and output the prediction residual signal to the transformation and quantization unit 131. Here, the subtractor 121 is configured to generate a prediction residual signal that is the difference between the prediction signal generated by intra prediction or inter prediction and the input image signal.

[0132] The adder 122 is configured to add the prediction signal to the prediction residual signal output from the inverse transform / inverse quantization unit 132 to generate a pre-filtered decoded signal, and output the pre-filtered decoded signal to the intra prediction unit 112 and the in-loop filter processing unit 150.

[0133] Here, the unfiltered decoded signal forms a reference block used by the intra prediction unit 112 .

[0134] The transform / quantization unit 131 is configured to perform a transform process on the prediction residual signal and to obtain a coefficient level value. Furthermore, the transform / quantization unit 131 may be configured to quantize the coefficient level value.

[0135] Here, the transform process is a process of transforming a prediction residual signal into a frequency component signal. In the transform process, a basis pattern (transformation matrix) corresponding to a discrete cosine transform (DCT) may be used, or a basis pattern (transformation matrix) corresponding to a discrete sine transform (DST) may be used.

[0136] The inverse transform / inverse quantization unit 132 is configured to perform an inverse transform process on the coefficient level values ​​output from the transform / quantization unit 131. Here, the inverse transform / inverse quantization unit 132 may be configured to perform inverse quantization of the coefficient level values ​​prior to the inverse transform process.

[0137] Here, the inverse transformation process and inverse quantization are performed in the reverse order to the transformation process and quantization performed by the transformation and quantization unit 131 .

[0138] The encoding unit 140 is configured to encode the coefficient level values ​​output from the transform / quantization unit 131, and output encoded data.

[0139] Here, for example, the coding is entropy coding, which assigns codes of different lengths based on the probability of occurrence of coefficient level values.

[0140] Furthermore, the encoding unit 140 is configured to encode control data for use in the decoding process in addition to the coefficient level values.

[0141] As mentioned above, the control data may include size data such as the coding block size, the prediction block size, and the transform block size.

[0142] Similar to in-loop filter processing unit 250, in-loop filter processing unit 150 is configured to perform filtering on the unfiltered decoded signal output from adder 122 and to output the filtered decoded signal to frame buffer 160.

[0143] Here, for example, the filtering process is a deblocking filtering process that reduces distortion occurring at the boundary portions of blocks (encoded blocks, predicted blocks, or transformed blocks).

[0144] The frame buffer 160 is configured to store reference frames used by the inter prediction unit 111 .

[0145] Here, the filtered decoded signal forms a reference frame used in the inter prediction unit 111.

[0146] (Intra prediction unit 112) Hereinafter, the intra prediction unit 112 according to this embodiment will be described with reference to Fig. 13. Fig. 13 is a diagram showing an example of functional blocks of the intra prediction unit 112 of the image encoding device 112 according to this embodiment.

[0147] As shown in FIG. 13, the intra prediction unit 112 has an intra prediction mode determination unit 112A, an intra prediction mode encoding unit 112B, and a prediction signal generation unit 112C.

[0148] The intra prediction unit 112 is an example of a prediction unit configured to generate a prediction signal by intra prediction (prediction within a frame).

[0149] The intra prediction mode determination unit 112A is configured to determine information to be coded by the subsequent intra prediction mode coding unit 112B, such as the intra prediction mode of the block and whether or not the ISP is applied. Since a known method can be used as a method of determination, details are omitted.

[0150] The intra-prediction mode encoding unit 112B is configured to encode the prediction mode of the current block. The processing content is the same as that of the intra-prediction mode decoding unit 242B. Specifically, for example, the processing is the same as that of the intra-prediction mode decoding unit 242B shown in FIG. 4 except that the "decoding" part is changed to "encoding".

[0151] The prediction signal generation unit 112C is configured to generate a prediction signal of the block in accordance with the prediction mode encoded in the intra prediction mode 112B. The method of generating a prediction signal is the same as that of the prediction signal generation unit 242C.

[0152] According to the image processing system 1 of this embodiment, even for a block to which an ISP is applied, intra prediction can refer to pixels of adjacent blocks that have already been encoded (decoded), so that processing of each sub-block can be performed in parallel, thereby reducing the number of cycles required to process one block in hardware.

[0153] The above-described image encoding device 100 and image decoding device 200 may be realized as a program that causes a computer to execute each function (each process).

[0154] In each of the above embodiments, the present invention has been described by taking as an example its application to the image encoding device 100 and the image decoding device 200. However, the present invention is not limited to this and can be similarly applied to an image encoding / decoding system having the functions of the image encoding device 100 and the image decoding device 200. [Explanation of symbols]

[0155] 10. Image processing system 100...Image encoding device 111, 241…Inter prediction section 112, 242...Intra prediction section 112A... Intra prediction mode determination unit 112B... Intra prediction mode encoding unit 112C, 242C...Prediction signal generation unit 121…Subtractor 122, 230…Adder 131...Transformation and quantization section 132, 220...Inverse transform and inverse quantization unit 140...encoding section 150, 250...In-loop filter processing section 160, 260... frame buffer 200...Image decoding device 210…Decoding section 242B... Intra prediction mode decoding unit

Claims

1. An image decoding device, comprising: A prediction signal generation unit that generates a prediction signal by intra prediction, the prediction signal generation unit, in a block to which intra sub-partition is applied, when generating prediction signals of all sub-blocks in the block, refers to pixels of a decoded block adjacent to the block when a width of the sub-block is less than a first threshold, and refers to decoded pixels of a sub-block processed immediately before when a width of the sub-block is equal to or greater than the first threshold; If the block size of the block is 32 pixels, the block is divided into two, 2. An image decoding device according to claim 1, wherein when the block size of the block is greater than 32 pixels, the block is divided into four.

2. An image encoding device, comprising: A prediction signal generation unit that generates a prediction signal by intra prediction, the prediction signal generation unit, in a block to which intra sub-partition is applied, when generating prediction signals of all sub-blocks in the block, refers to pixels of a decoded block adjacent to the block when a width of the sub-block is less than a first threshold, and refers to decoded pixels of a sub-block processed immediately before when a width of the sub-block is equal to or greater than the first threshold; If the block size of the block is 32 pixels, the block is divided into two, 2. An image encoding device according to claim 1, wherein when the block size of the block is greater than 32 pixels, the block is divided into four.

3. generating a prediction signal by intra prediction, In the step, when generating prediction signals of all sub-blocks in a block to which intra sub-division is applied, if the width of the sub-block is less than a first threshold, pixels of a decoded block adjacent to the block are referenced, and if the width of the sub-block is equal to or greater than the first threshold, decoded pixels of a sub-block processed immediately before are referenced; If the block size of the block is 32 pixels, the block is divided into two, 2. An image decoding method according to claim 1, wherein, if the block size of the block is greater than 32 pixels, the block is divided into four.

4. A program for use in an image decoding device, comprising: Executing a step of generating a prediction signal by intra prediction, In the step, when generating prediction signals of all sub-blocks in a block to which intra sub-division is applied, if the width of the sub-block is less than a first threshold, pixels of a decoded block adjacent to the block are referenced, and if the width of the sub-block is equal to or greater than the first threshold, decoded pixels of a sub-block processed immediately before are referenced; If the block size of the block is 32 pixels, the block is divided into two, When the block size of the block is greater than 32 pixels, the block is divided into four.

Images