Image decoding device, image encoding device, image decoding method, and program product

Abstract

Provided are an image decoding device, an image encoding device, an image decoding method, and a program product, the image decoding device (200) being provided with a prediction signal generation unit (242C) that generates a prediction signal based on intra prediction, and generates a prediction signal based on intra prediction. A prediction signal generation unit (242C) refers to pixels of a decoded block adjacent to a block in which intra-frame sub-division is applied, when generating prediction signals for all sub-blocks in the block. According to the image decoding device, the image encoding device, the image decoding method and the program product of the present invention, even a block to which ISP is applied, the processing of each sub-block can be executed in parallel.

Description

[0001] This application is a division of an invention patent application with an application date of March 2, 2020, application number 202080019709.X, and invention name “Image decoding device, image decoding method and program”. Technical Field

[0002] The present invention relates to an image decoding device, an image encoding device, an image decoding method and a program product. Background Art

[0003] Conventionally, there is known an “Intra Sub-Partitions (ISP)” technique that divides a block into sub-blocks and sequentially performs intra prediction on each sub-block (for example, refer to Non-Patent Documents 1 and 2).

[0004] Prior art literature

[0005] Patent Literature

[0006] Patent Document 1: Japanese Patent Application Publication No. 2018-037936

[0007] Non-patent literature

[0008] Non-patent document 1: Versatile Video Coding (Draft 4), JVET-M1001

[0009] Non-patent literature 2: CE3: Intra Sub-Partitions Coding Mode (Tests 1.1.1 and 1.1.2), JVET-M0102

[0010] Non-patent document 3: Jiang Jie, Guo Balong, Mo Wei, and Fan Kefeng, Block-Based Parallel Intra Prediction Scheme for HEVC, Journal of Multimedia, Vol.7, No.4, August 2012 Summary of the invention

[0011] Problems to be solved by the invention

[0012] However, in the previous ISP, when performing intra-frame prediction of a sub-block, the decoded image of the sub-block processed immediately before in the same block is referenced. Therefore, the processing of each sub-block needs to be performed serially. In particular, when implemented by hardware, there is a problem that such processing may become a bottleneck of processing speed.

[0013] Therefore, the present invention is completed in view of the above-mentioned problems, and its purpose is to provide an image decoding device, an image encoding device, an image decoding method and a program product that can perform processing of each sub-block in parallel even for a block to which ISP is applied.

[0014] Means for solving problems

[0015] The main purpose of the first technical solution of the present invention is an image decoding device, which has a prediction signal generating unit, which generates a prediction signal based on intra-frame prediction. The prediction signal generating unit refers to the pixels of a decoded block adjacent to the block when generating prediction signals for all sub-blocks in the block in which intra-frame sub-division is applied.

[0016] The main purpose of the second technical solution of the present invention is an image decoding device, which has a prediction signal generating unit, which generates a prediction signal based on intra-frame prediction, and the prediction signal generating unit divides the block only when the size of the divided sub-block is greater than the minimum size of the block when no division is performed in the block applying intra-frame sub-division.

[0017] The main purpose of the third technical solution of the present invention is an image decoding device, which has an intra-frame prediction mode decoding unit, which decodes information related to intra-frame prediction, and omits the decoding of the division direction of the intra-frame sub-division according to the shape of the block to which the intra-frame sub-division is applied.

[0018] The main purpose of the fourth technical solution of the present invention is an image decoding device, which has an intra-frame prediction mode decoding unit, which constructs an MPM list in the block applying the intra-frame sub-division considering the shape of the divided sub-block, and uses the MPM list to decode the intra-frame prediction mode.

[0019] The main purpose of the fifth technical solution of the present invention is an image decoding device, which has an intra-frame prediction mode decoding unit, which constructs an MPM list in a block to which intra-frame sub-division is applied, considering the shape of the block before division, and uses the MPM list to decode the intra-frame prediction mode.

[0020] The main purpose of the sixth technical solution of the present invention is an image decoding method, which has a step of generating a prediction signal based on intra-frame prediction, in which the pixels of the decoded blocks adjacent to the block are referenced when generating the prediction signals of all sub-blocks in the block in the block to which intra-frame sub-division is applied.

[0021] The subject matter of the seventh technical solution of the present invention is a program product, which includes a program used in an image decoding device, which enables a computer to execute the steps of generating a prediction signal based on intra-frame prediction, in which the pixels of a decoded block adjacent to the block are referenced when generating the prediction signals of all sub-blocks in the block in the block applying intra-frame sub-division.

[0022] Effects of the Invention

[0023] According to the present invention, it is possible to provide an image decoding device, an image encoding device, an image decoding method, and a program product that can perform processing of each sub-block in parallel even for a block to which ISP is applied. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 FIG. 1 is a diagram showing an example of a configuration of an image processing system 10 according to an embodiment.

[0025] Figure 2 This is a diagram showing an example of functional blocks of an image decoding device 200 according to an embodiment.

[0026] Figure 3 FIG. 1 is a diagram showing an example of a functional block of the intra prediction unit 242 of the image decoding device 200 according to an embodiment.

[0027] Figure 4 1 is a flowchart showing an example of a processing procedure of the intra prediction mode decoding unit 242B of the intra prediction unit 242 of the image decoding device 200 according to one embodiment.

[0028] Figure 5 FIG. 1 is a diagram showing an example of a partitioning method into sub-blocks used in one embodiment.

[0029] Figure 6 1 is a flowchart showing an example of a process for determining a division direction and a division number in an ISP used in one embodiment.

[0030] Figure 7 1 is a flowchart showing an example of a process for determining a division direction and a division number in an ISP used in one embodiment.

[0031] Figure 8 is a flowchart illustrating an example of a method for constructing an MPM list used in one embodiment.

[0032] Fig. 9 is a flowchart illustrating an example of a method for constructing an MPM list used in one embodiment.

[0033] Fig.10The configuration of reference pixels during intra prediction when vertical 4-division in ISP is applied in one embodiment is shown.

[0034] Fig.11 An example of the configuration of reference pixels during intra prediction when vertical 4-division in ISP is applied in one embodiment is shown.

[0035] Fig.12 FIG. 1 is a diagram showing an example of functional blocks of an image encoding device 100 according to an embodiment.

[0036] Fig.13 FIG. 1 is a diagram showing an example of a functional block of the intra prediction unit 112 of the image encoding device 100 according to an embodiment. DETAILED DESCRIPTION

[0037] Hereinafter, the embodiments of the present invention will be described with reference to the accompanying drawings. In addition, the constituent elements in the following embodiments can be appropriately replaced with existing constituent elements, etc., and various modifications including combinations with other existing constituent elements can be performed. Therefore, the description of the following embodiments is not intended to limit the content of the invention described in the claims.

[0038] (First Embodiment)

[0039] Below, refer to Figures 1 to 13 An image processing system 10 according to a first embodiment of the present invention will be described. Figure 1 1 is a diagram showing an image processing system 10 according to the present embodiment.

[0040] like Figure 1 As shown, the image processing system 10 includes an image encoding device 100 and an image decoding device 200 .

[0041] The image encoding device 100 is configured to generate encoded data by encoding an input image signal. The image decoding device 200 is configured to generate an output image signal by decoding the encoded data.

[0042] Such coded data may be transmitted from the image coding apparatus 100 to the image decoding apparatus 200 via a transmission path. Alternatively, the coded data may be provided from the image coding apparatus 100 to the image decoding apparatus 200 after being stored in a storage medium.

[0043] (Image Decoding Device 200)

[0044] Below, refer to Figure 2 The image decoding device 200 according to this embodiment is described. Figure 2 This is a diagram showing an example of functional blocks of the image decoding device 200 according to the present embodiment.

[0045] like Figure 2 As shown, the image decoding device 200 includes a decoding unit 210 , an inverse transform and inverse quantization unit 220 , an adder 230 , an inter prediction unit 241 , an intra prediction unit 242 , an in-loop filter processing unit 250 , and a frame buffer 260 .

[0046] The decoding section 210 is configured to decode the encoded data generated by the image encoding device 100 and decode the coefficient level values.

[0047] Among them, for example, decoding is decoding of entropy-coded data to which codes of different lengths are assigned based on the probability of occurrence of coefficient level values.

[0048] The decoding unit 210 may also be configured to obtain control data by decoding the encoded data.

[0049] The control data may include size data such as the coding block (CU: Coding Unit) size, the prediction block (PU: Prediction Unit) size, and the transform block (TU: Transform Unit) size.

[0050] The inverse transform and inverse quantization unit 220 is configured to perform an inverse transform process on the coefficient level values ​​output from the decoding unit 210. The inverse transform and inverse quantization unit 220 may also be configured to perform an inverse quantization process on the coefficient level values ​​before the inverse transform process.

[0051] The adder 230 is configured to add the prediction signal and the prediction residual signal output from the inverse transform and inverse quantization section 220 to generate a pre-filtering decoded signal, and output the pre-filtering decoded signal to the intra prediction section 242 and the in-loop filter processing section 250 .

[0052] Among them, the decoded signal before filtering constitutes a reference block used in the intra prediction unit 242.

[0053] The inter-frame prediction unit 241 is configured to generate a prediction signal through inter-frame prediction.

[0054] Specifically, the inter prediction section 241 is configured to generate a prediction signal for each prediction block based on a motion vector decoded from the encoded data and a reference signal included in a reference frame. The inter prediction section 241 is configured to output the prediction signal to the adder 230 .

[0055] The intra prediction unit 242 is configured to generate a prediction signal by intra-frame prediction.

[0056] Specifically, the intra prediction unit 242 is configured to determine a reference block included in the target frame, and generate a prediction signal for each prediction block based on the determined reference block. The intra prediction unit 242 is configured to output the prediction signal to the adder 230.

[0057] The in-loop filter processing section 250 is configured to perform a filter process on the decoded signal before filtering outputted from the adder 230 , and output the decoded signal after filtering to the frame buffer 260 .

[0058] Among them, for example, the filtering process is a deblocking filtering process that reduces distortion generated at a boundary portion of a block (a coding block, a prediction block, a transform block, or a sub-block obtained by dividing these blocks).

[0059] The frame buffer 260 is configured to accumulate reference frames used in the inter prediction section 241 .

[0060] The decoded signal after filtering constitutes a reference frame used in the inter-frame prediction unit 241 .

[0061] (Intra-frame prediction unit 242)

[0062] Below, refer to Figure 3 The intra prediction unit 242 of this embodiment will be described. Figure 3 FIG. 1 is a diagram showing an example of functional blocks of the intra prediction unit 242 of the image decoding device 200 according to the present embodiment.

[0063] like Figure 3 As shown, the intra prediction unit 242 includes an intra prediction mode decoding unit 242B and a prediction signal generating unit 242C.

[0064] The intra prediction section 242 is an example of a prediction section configured to generate a prediction signal by intra-frame prediction.

[0065] The intra prediction mode decoding unit 242B decodes information necessary for performing intra prediction for each block.

[0066] Figure 4 242B is a flowchart showing an example of the processing procedure of the intra prediction mode decoding unit 242B. Hereinafter, an example of the processing procedure of the intra prediction mode decoding unit 242B will be described using such a flowchart.

[0067] like Figure 4 As shown, in step S41, the intra prediction mode decoding unit 242B decodes information (ref_idx) related to the pixel position referenced when performing intra prediction. The value that "ref_idx" can take is any one of "0", "1" and "2".

[0068] In the case of "ref_idx=0", the intra prediction mode decoding unit 242B performs intra prediction with reference to pixel values ​​located at the block boundary with the processing target block among pixels of decoded blocks adjacent to the currently processed block (hereinafter referred to as the processing target block).

[0069] In the case of "ref_idx=1", the intra prediction mode decoding unit 242B performs intra prediction with reference to pixels that are one row or one column away from the pixels located at the block boundary.

[0070] In the case of "ref_idx=2", the intra prediction mode decoding unit 242B performs intra prediction with reference to pixels that are three rows or three columns away from the pixels located at the block boundary.

[0071] In step S42 , the intra prediction mode decoding unit 242B determines whether or not predetermined condition 1 is satisfied.

[0072] Among them, the predetermined condition 1 may include the condition that the value of "ref_idx" is "0". In addition, the predetermined condition 1 may also include the condition that the size of the processing target block (the number of pixels contained in the block) is greater than a predetermined number of pixels. Such a number of pixels can be set to 32 pixels, for example. In addition, the predetermined condition 1 may also include the condition that the size (number of pixels) in the vertical direction of the processing target block is greater than 8 pixels. Further, the predetermined condition 1 may also include the condition that the size in the horizontal direction of the processing target block is greater than 8 pixels.

[0073] When the predetermined condition 1 is satisfied, the present process proceeds to step S43 , and when the predetermined condition 1 is not satisfied, the present process proceeds to step S46 .

[0074] In step S43, the intra prediction mode decoding unit 242B decodes the "ISP mode flag".

[0075] The intra prediction mode decoding section 242B does not apply ISP to the processing target block in the case of “ISP mode flag=0”, and applies ISP to the processing target block in the case of “ISP mode flag=1”.

[0076] In step S44, the intra prediction mode decoding unit 242B determines whether or not predetermined condition 2 is satisfied. If predetermined condition 2 is satisfied, the process proceeds to step S45, and if predetermined condition 2 is not satisfied, the process proceeds to step S46.

[0077] The predetermined condition 2 may include a condition that the value of the "ISP mode flag" is "1". Further, the predetermined condition 2 may also include a condition that the shape of the processing target block is not horizontally long or vertically long.

[0078] In addition, the so-called "horizontal length" can be defined, for example, as the size of the processing target block in the horizontal direction (number of pixels) being more than twice the size of the processing target block in the vertical direction (number of pixels). Similarly, the so-called "vertical length" can be defined, for example, as the size of the processing target block in the vertical direction being more than twice the size of the processing target block in the horizontal direction.

[0079] In addition, for example, "horizontally long" may be defined as a size of the processing target block in the horizontal direction being larger than a size in the vertical direction, and "longitudinally long" may be defined as a size of the processing target block in the vertical direction being larger than a size in the horizontal direction.

[0080] When the predetermined condition 2 includes the above condition, the intra-frame prediction mode decoding unit 242B omits decoding of the "ISP split flag" in step S45 for the block determined to be horizontally long or vertically long. In this case, the intra-frame prediction mode decoding unit 242B can determine the ISP split direction according to the block shape, so that the horizontally long block is horizontally split and the vertically long block is vertically split.

[0081] In step S45, the intra prediction mode decoding unit 242B decodes the "ISP split flag".

[0082] In the case of "ISP split flag = 0", it indicates that the ISP split direction is horizontal (ie, horizontal split), and in the case of "ISP split flag = 1", it indicates that the ISP split direction is vertical (ie, vertical split).

[0083] ISP is a technology for dividing a processing target block into a plurality of sub-blocks to perform intra-frame prediction on each sub-block and generate a decoded image.

[0084] Figure 5 An example of such a segmentation method is shown. Figure 5 As shown, in the present embodiment, in the case of horizontal division, the sub-blocks are divided into the same number of pixels in the horizontal direction as before the division and the number of pixels in the vertical direction is 1 / 2 or 1 / 4 of the number of pixels before the division (horizontal 2 division or horizontal 4 division). Similarly, in the case of vertical division, the sub-blocks are divided into the same number of pixels in the vertical direction as before the division and the number of pixels in the horizontal direction is 1 / 2 or 1 / 4 of the number of pixels before the division (vertical 2 division or vertical 4 division).

[0085] Among them, refer to Figure 6 The process of determining the division direction and number of divisions of the ISP used in this embodiment is described. Figure 6 This is a flowchart showing an example of a process for determining the division direction and the number of divisions of the ISP used in this embodiment.

[0086] like Figure 6 As shown, in step S61, based on the "ISP split flag" decoded in step S45, in the case of vertical splitting, the process proceeds to step S62, and in the case of horizontal splitting, the process proceeds to step S63.

[0087] In step S62, the block size of the processing target block is determined, where the block size can be defined as the number of pixels in the block, for example.

[0088] When the predetermined condition 1 includes the condition that the size of the processing target block is greater than 32 pixels, in step S62, the number of ISP divisions can be determined, for example, as "vertical 2 divisions" when the block size is 32 pixels, or as "vertical 4 divisions" when the block size is greater than 32 pixels.

[0089] Similarly, in step S63, the number of divisions of the ISP can be determined, for example, as “horizontal 2 divisions” when the block size is 32 pixels, and as “horizontal 4 divisions” when the block size is larger than 32 pixels.

[0090] In addition, it is also possible to Figure 7 The process shown determines the direction and number of ISP segmentation. In addition, Figure 7 This is an example of a decision process in a case where the predetermined condition 1 includes both the condition that the size of the processing target block in the vertical direction is 8 pixels or more and the condition that the size of the processing target block in the horizontal direction is 8 pixels or more.

[0091] like Figure 7 As shown, in step S71, similarly to step S61, in the case of vertical segmentation, the process proceeds to step S72, and in the case of horizontal segmentation, the process proceeds to step S73.

[0092] In step S72, the number of ISP divisions can be determined as "vertical 2 divisions" when the horizontal size of the processing target block is 8 pixels, and "vertical 4 divisions" when the horizontal size of the processing target block is greater than 8 pixels.

[0093] However, when only the size of the number of pixels which is a power of 2 is defined as the size of the block in the horizontal direction and the vertical direction, the case where the above-mentioned size is larger than 8 pixels is equivalent to the case where such a size is larger than 16 pixels.

[0094] Similarly, in step S73, the number of ISP divisions can be determined as "horizontal 2 divisions" when the vertical size of the processing target block is 8 pixels, and "horizontal 4 divisions" when the vertical size of the processing target block is greater than 8 pixels.

[0095] By adopting Figure 7 The process shown can be segmented so that the size of the sub-block after segmentation in the horizontal and vertical directions must be more than 4 pixels. In addition, when the minimum value of the size in the horizontal and vertical directions when the ISP is not used is set to 4 pixels, the size of the sub-block can be guaranteed to be more than the minimum size when the ISP is not used (the minimum size of the block when not segmented) through the above-mentioned processing. As a result, the complexity of the processing when the ISP is used can be suppressed below the complexity of the processing when the ISP is not used.

[0096] in addition, Figure 7 The process shown is only an example, and the same effect can be obtained as long as the division method is used so that the size of the sub-block after division is definitely equal to or larger than the minimum size in the case of no division.

[0097] As described above, after the intra prediction mode decoding unit 242B decodes the "ISP split flag" indicating the ISP split direction, the present process proceeds to step S46.

[0098] In step S46 , the intra prediction mode decoding unit 242B determines whether or not predetermined condition 3 is satisfied.

[0099] The predetermined condition 3 may include a condition that the value of the "ISP mode flag" is "0". In addition, when the intra-frame prediction mode decoding unit 242B does not decode the "ISP mode flag", it can be regarded that the value of the "ISP mode flag" is "0".

[0100] In addition, when the predetermined condition 3 is satisfied, the present process proceeds to step S47, and when the predetermined condition 3 is not satisfied, the present process proceeds to step S48.

[0101] In step S47 , the intra prediction mode decoding unit 242B decodes “MPM flag”.

[0102] However, when "MPM flag=1", it means that the intra prediction mode of the processing target block is any one of the prediction modes existing in the MPM list described later.

[0103] On the other hand, in the case of "MPM flag = 0", it means that a prediction mode that does not exist in the MPM list is applied to the processing target block.

[0104] In addition, when the “MPM flag” is not decoded, that is, when the predetermined condition 3 is not satisfied, for example, the intra-frame prediction mode decoding unit 242B can regard that the value of the “MPM flag” is “1”.

[0105] In step S48, the intra prediction mode decoding unit 242B decodes the intra prediction mode based on the information decoded so far. Intra prediction modes include, for example, a DC mode, a Planar mode, a directional prediction mode, and the like as described in Non-Patent Document 1.

[0106] In order to decode the intra prediction mode, the intra prediction mode decoding unit 242B first creates an MPM list. The MPM list creation method can use the well-known method disclosed in Non-Patent Document 1. In addition, the MPM list creation method can also be switched according to the value of the above-mentioned "ISP mode flag".

[0107] Figure 8 2 is a flowchart showing an example of a method for creating an MPM list when the value of the "ISP mode flag" is "1". Figure 8 As shown, a default mode corresponding to the segmentation direction is predefined.

[0108] Next, the intra prediction mode decoding section 242B constructs an MPM list using the intra prediction modes used in two blocks of the decoded blocks A and B adjacent to the processing target block.

[0109] The decoded block A is an upper neighboring block of the processing target block, and the decoded block B is a left neighboring block of the processing target block. As for a strict method of determining the decoded blocks A and B, the method described in Non-Patent Document 1 can be used.

[0110] Figure 8 This is an example of the construction process of the MPM list when the intra prediction modes used in the decoded blocks A and B are both directional predictions and the prediction directions are different.

[0111] When the difference in angle between the prediction direction used in decoded block A and the default mode is smaller than the difference in angle between the prediction direction used in decoded block AB and the default mode, the intra-frame prediction mode decoding unit 242B sets the prediction mode used in decoded block A to the 0th item in the MPM list, and sets the prediction mode used in decoded block B to the 1st item in the MPM list.

[0112] Thereafter, the intra-frame prediction mode decoding unit 242B sets prediction modes for the 2nd and 3rd items in the MPM list that are slightly different from the prediction mode of the decoded block A, and sets prediction modes for the 4th and 5th items in the MPM list that are slightly different from the prediction mode of the decoded block B.

[0113] in, Figure 8 A and B are defined by numerical values ​​corresponding to the angle of the predicted direction. The larger the value, the larger the angle, and the smaller the value, the smaller the angle.

[0114] Similarly, when the difference in angle between the prediction direction used in the decoded block B and the default mode is smaller than the difference in angle between the prediction direction used in the decoded block A and the default mode, the intra-frame prediction mode decoding unit 242B sets the prediction mode used in the decoded block B to the 0th one in the MPM list, and sets the prediction mode used in the decoded block A to the 1st one in the MPM list.

[0115] Afterwards, the intra-frame prediction mode decoding unit 242B sets prediction modes for the 2nd and 3rd items in the MPM list that are slightly different from the prediction mode of the decoded block B, and sets prediction modes for the 4th and 5th items in the MPM list that are slightly different from the prediction mode of the decoded block A.

[0116] In the case where a prediction mode with a smaller value of the index (Idx) assigned to the MPM list can encode a codeword with a smaller amount of code, the index of the mode with a smaller angle difference from the default mode is set to be smaller, thereby being able to express the prediction mode suitable for the prediction direction of the segmentation direction with a smaller amount of code, thereby improving the encoding efficiency.

[0117] The method of constructing the MPM list described here is only an example, and the same effect can be obtained as long as the method of constructing the MPM list is to make the index of the pattern considered to be more appropriate smaller by taking the division direction into consideration.

[0118] exist Figure 8 In the example, the default mode setting is defined by the ISP split method, but it can also be Fig. 9 As shown in the example of , it is defined based on the shape of the segmented sub-blocks.

[0119] like Fig. 9 As shown, a default mode can be defined according to whether the shape of the divided sub-block is horizontally long, vertically long, or any other shape, and the method of constructing the MPM list can be switched according to the default mode.

[0120] The horizontal length may be defined as, for example, that the size (number of pixels) of the processing target block in the horizontal direction is more than twice the size (number of pixels) of the processing target block in the vertical direction. Similarly, the vertical length may be defined as, for example, that the size of the processing target block in the vertical direction is more than twice the size of the processing target block in the horizontal direction.

[0121] In addition, for example, the horizontal length may be defined as the size of the processing target block in the horizontal direction being larger than the size in the vertical direction, and the vertical length may be defined as the size of the processing target block in the vertical direction being larger than the size in the horizontal direction.

[0122] In this case, the case where the sizes in the horizontal and vertical directions are equal is included in either the vertical length or the horizontal length, so that the definition of the default mode can also be combined with the Figure 7 The example is similarly set to two modes.

[0123] In addition, Fig. 9 In the embodiment, the default mode is defined based on the block shape after segmentation, but the same processing can be performed even when the default mode is defined using the block shape before segmentation.

[0124] As described above, after the intra prediction mode decoding unit 242B decodes the "MPM flag", the process proceeds to step S48.

[0125] In step S48, when "MPM flag = 1", the intra-frame prediction mode decoding unit 242B decodes the index of the prediction mode used in the processing target block, and determines the prediction mode corresponding to the decoded index in the constructed MPM list as the prediction mode of the processing target block as described above.

[0126] On the other hand, in step S48, when "MPM flag = 0", the intra-frame prediction mode decoding unit 242B decodes information indicating which prediction mode is used among the prediction modes not included in the above-mentioned MPM list, and determines the decoded prediction mode as the prediction mode of the processing target block.

[0127] Through the above process, the prediction mode used in the processing target block can be decoded, so this processing enters step S49 and ends.

[0128] Figure 3 The prediction signal generation unit 242C shown is configured to generate a prediction signal based on the prediction mode of the processing target block decoded by the intra prediction mode decoding unit 242B. As a method of generating the prediction signal, for example, a known method described in Non-Patent Document 1 can be used.

[0129] Fig.10An example of the configuration of reference pixels in intra prediction when vertical 4-division in ISP is applied is shown.

[0130] In ISP, when the block T is divided in the vertical direction (when vertical division is performed), prediction processing and decoded image generation processing are performed in order from the sub-block #1 on the left.

[0131] like Fig.10 As shown, when performing intra prediction on the second left subblock #2, pixel X1 of the decoded block A adjacent to the upper side of subblock #2 and decoded pixel Y1 of the subblock #1 adjacent to the left side of subblock #2 are referenced.

[0132] Fig.10 The configuration of reference pixels shown is only an example. For example, when performing intra prediction for sub-block #2 in vertical segmentation, if it is a decoded pixel in sub-block #1, it can refer to Fig.10 Pixels other than pixel Y1 in the configuration shown.

[0133] Similarly, in ISP, when a block is divided in the horizontal direction (when horizontal division is performed), prediction processing and decoded image generation processing are performed in order from the upper sub-block.

[0134] Among them, Fig.11 In that case, the position of the reference pixel when performing intra-frame prediction for each sub-block #1 to #4 can also be set, for example, to the pixel X2 of the decoded block A adjacent to the upper side of sub-blocks #1 to #4 (block T), and the pixel Y2 of the decoded block B / C adjacent to the left side of sub-blocks #1 to #4 (block T).

[0135] That is, regarding the intra prediction for the second and subsequent sub-blocks #2 to #4, it is not necessary to refer to the decoded pixels of the sub-blocks #1 to #3 in the processing target block T.

[0136] Fig.11 The pixel configuration shown is only an example. If the pixels are of decoded blocks adjacent to the processing target block T, you can also refer to Fig.11 Pixels other than pixel X2 / Y2 in the configuration shown.

[0137] Fig.11 Although the example of vertical division is shown, the same reference pixel configuration can be adopted in the case of horizontal division. Thus, by setting not to refer to the decoded pixels in the block, the prediction process and the decoded image generation process of each sub-block can be performed in parallel.

[0138] In this embodiment, the prediction mode of each sub-block in the same block is common. Fig.11The generation process of the prediction signal in the case of the arrangement of reference pixels shown may be the same as the generation process of the normal intra prediction without dividing the processing target block T into sub-blocks.

[0139] Therefore, by adopting Fig.11 The configuration of reference pixels shown can realize a structure that performs prediction signal generation processing in units of blocks regardless of whether ISP is applied or not. In addition, in such a case, the generation processing of the decoded image in the ISP application block (the generation processing of the decoded signal before filtering) can be performed in parallel for each sub-block.

[0140] Furthermore, the arrangement of the reference pixels may be switched according to the block size. For example, the arrangement of the reference pixels may be switched according to whether the width and height of the sub-block are smaller than a predetermined threshold.

[0141] For example, Figure 6 As shown, it is possible to use the sub-block width less than 4 pixels. Fig.11 The configuration of the reference pixels shown is adopted when the width of the sub-block is 4 pixels or more. Fig.10 Configuration of reference pixels shown.

[0142] Likewise, for example, it is possible to use a sub-block with a height less than 4 pixels. Fig.11 The configuration of reference pixels shown is adopted when the height of the sub-block is 4 pixels or more. Fig.10 Configuration of reference pixels shown.

[0143] That is, for example, the prediction signal generating section 242C may also be configured to generate prediction signals for all sub-blocks #1 to #4 in the processing target block T when the sizes of the sub-blocks #1 to #4 are smaller than the first threshold value, as shown in FIG. Fig.11 As shown, pixels of decoded blocks A to C adjacent to the processing target block T are referenced (eg, reference pixels X2 / Y2 ).

[0144] On the other hand, for example, the prediction signal generating unit 242C may also be configured to generate the prediction signal of the sub-block #2 when the size of the sub-block #2 is greater than the first threshold, such as Fig.10 As shown, reference is made to a decoded pixel (eg, reference pixel Y1) of sub-block #1 that is processed immediately before.

[0145] In addition, the prediction signal generating section 242C may also be configured to generate prediction signals for all sub-blocks #1 to #4 in the processing target block T when at least one of the width or height of the sub-block is smaller than the first threshold value, such as Fig.11 As shown, pixels of decoded blocks A to C adjacent to the processing target block T are referenced (eg, reference pixels X2 / Y2 ).

[0146] On the other hand, for example, the prediction signal generating unit 242C may also be configured to generate the prediction signal of the sub-block #2 when the width and height of the sub-block #2 are both greater than the first threshold, such as Fig.10 As shown, reference is made to a decoded pixel (eg, reference pixel Y1) of sub-block #1 that is processed immediately before.

[0147] Further, the prediction signal generating unit 242C may also be configured to generate prediction signals of all sub-blocks #1 to #4 in the processing target block T when the width of the sub-blocks #1 to #4 is less than the first threshold value, such as Fig.11 As shown, pixels of decoded blocks A to C adjacent to the processing target block T are referenced (eg, reference pixels X2 / Y2 ).

[0148] On the other hand, for example, the prediction signal generating unit 242C may also be configured to generate the prediction signal of the sub-block #2 when the width of the sub-block #2 is greater than the first threshold, such as Fig.10 As shown, reference is made to a decoded pixel (eg, reference pixel Y1) of sub-block #1 that is processed immediately before.

[0149] Thus, by switching the configuration of reference pixels according to the block size, for example, adopting a configuration of reference pixels that can be processed in parallel in smaller sub-blocks to prevent an increase in implementation difficulty and referring to the decoded pixels of the immediately preceding sub-block in larger sub-blocks, the encoding efficiency can be improved.

[0150] Parallel processing techniques for intra prediction are also disclosed in, for example, Patent Document 1 and Non-Patent Document 3. These Non-Patent Documents support the case where different prediction modes are used in processing units corresponding to sub-blocks in the present embodiment.

[0151] On the other hand, in this embodiment, the prediction mode of each sub-block in the same block is common. By adopting such a structure, the generation process of the prediction value becomes completely the same process in the case of segmentation into sub-blocks and the case of non-segmentation into sub-blocks. Therefore, it is not necessary to implement different processes depending on the presence or absence of segmentation, so it is particularly possible to prevent an increase in the difficulty of implementation in hardware.

[0152] (Image Coding Device 100)

[0153] Below, refer to Fig.12 The image encoding device 100 according to this embodiment will be described. Fig.12 1 is a diagram showing an example of functional blocks of the image encoding device 100 according to the present embodiment.

[0154] like Fig.12As shown, the image encoding device 100 has an inter-frame prediction unit 111, an intra-frame prediction unit 112, a subtractor 121, an adder 122, a transform and quantization unit 131, an inverse transform and inverse quantization unit 132, an encoding unit 140, an in-loop filter processing unit 150 and a frame buffer 160.

[0155] The inter-frame prediction unit 111 is configured to generate a prediction signal by inter-frame prediction, similarly to the inter-frame prediction unit 241 .

[0156] Specifically, the inter-frame prediction unit 111 is configured to determine a reference block included in a reference frame by comparing a coding target frame (hereinafter referred to as a target frame) with a reference frame stored in a frame buffer 160 , and determine a motion vector for the determined reference block.

[0157] In addition, the inter prediction unit 111 is configured to generate a prediction signal included in the prediction block for each prediction block based on the reference block and the motion vector. The inter prediction unit 111 is configured to output the prediction signal to the subtractor 121 and the adder 122. The reference frame is a frame different from the target frame.

[0158] The intra prediction unit 112 is configured to generate a prediction signal by intra-frame prediction, similarly to the intra prediction unit 242 .

[0159] Specifically, the intra prediction section 112 is configured to determine a reference block included in the target frame and generate a prediction signal for each prediction block based on the determined reference block. In addition, the intra prediction section 112 is configured to output the prediction signal to the subtractor 121 and the adder 122.

[0160] The reference block is a block that is referenced for predicting a target block (hereinafter referred to as a target block). For example, the reference block is a block adjacent to the target block.

[0161] The subtractor 121 is configured to subtract the prediction signal from the input image signal and output the prediction residual signal to the transform and quantization section 131. The subtractor 121 is configured to generate a prediction residual signal which is a difference between the prediction signal generated by intra prediction or inter prediction and the input image signal.

[0162] The adder 122 is configured to add the prediction signal and the prediction residual signal output from the inverse transform and inverse quantization section 132 to generate a pre-filtering decoded signal, and output such a pre-filtering decoded signal to the intra prediction section 112 and the in-loop filter processing section 150 .

[0163] The decoded signal before filtering constitutes a reference block used in the intra prediction unit 112 .

[0164] The transform and quantization unit 131 is configured to perform a transform process on the prediction residual signal and obtain coefficient level values. Furthermore, the transform and quantization unit 131 may also be configured to perform quantization of the coefficient level values.

[0165] The transform process is a process of transforming the prediction residual signal into a frequency component signal. In such a transform process, a basic pattern (transformation matrix) corresponding to a discrete cosine transform (DCT) or a basic pattern (transformation matrix) corresponding to a discrete sine transform (DST) can be used.

[0166] The inverse transform and inverse quantization unit 132 is configured to perform an inverse transform process on the coefficient level values ​​output from the transform and quantization unit 131. The inverse transform and inverse quantization unit 132 may also be configured to perform an inverse quantization on the coefficient level values ​​before the inverse transform process.

[0167] Among them, the inverse transform processing and inverse quantization are performed according to the reverse process of the transform processing and quantization performed by the transform and quantization unit 131.

[0168] The encoding section 140 is configured to encode the coefficient level values ​​output from the transform and quantization section 131 and output the encoded data.

[0169] Among them, for example, the encoding is entropy encoding that assigns codes of different lengths based on the probability of occurrence of coefficient level values.

[0170] In addition, the encoding unit 140 is configured to encode control data used in the decoding process in addition to the coefficient level values.

[0171] In addition, as described above, the control data may also include size data such as coding block size, prediction block size, and transform block size.

[0172] Similar to the in-loop filter processing unit 250 , the in-loop filter processing unit 150 is configured to perform filtering on the pre-filtering decoded signal output from the adder 122 and output the post-filtering decoded signal to the frame buffer 160 .

[0173] Among them, for example, the filtering process is a deblocking filtering process that reduces distortion generated at a boundary portion of a block (coding block, prediction block, or transform block).

[0174] The frame buffer 160 is configured to accumulate reference frames used in the inter prediction section 111 .

[0175] The decoded signal after filtering constitutes a reference frame used in the inter-frame prediction unit 111 .

[0176] (Intra-frame prediction unit 112)

[0177] Below, refer to Fig.13 The intra prediction unit 112 according to this embodiment will be described. Fig.13 FIG. 1 is a diagram showing an example of functional blocks of the intra prediction unit 112 of the image encoding device 112 according to the present embodiment.

[0178] like Fig.13 As shown, the intra prediction unit 112 includes an intra prediction mode determination unit 112A, an intra prediction mode encoding unit 112B, and a prediction signal generation unit 112C.

[0179] The intra-frame prediction section 112 is an example of a prediction section configured to generate a prediction signal by intra-frame prediction.

[0180] The intra prediction mode determination unit 112A is configured to determine the intra prediction mode of the block, whether to apply ISP, and other information to be encoded by the subsequent intra prediction mode encoding unit 112B. A known method can be used for the determination method, so detailed description is omitted.

[0181] The intra-frame prediction mode encoding unit 112B is configured to encode the prediction mode of the target block. The processing content is the same as that of the intra-frame prediction mode decoding unit 242B. Specifically, for example, Figure 4 The “decoding” part in the process flow chart of the intra prediction mode decoding unit 242B shown in the figure is changed to the “encoding” process.

[0182] The prediction signal generation unit 112C is configured to generate a prediction signal for the block according to the prediction mode encoded by the intra prediction mode 112B. The prediction signal generation method is the same as that of the prediction signal generation unit 242C.

[0183] According to the image processing system 1 of this embodiment, even for a block to which ISP is applied, pixels of an adjacent block that has been encoded (decoded) can be referenced in intra-frame prediction, so that processing of each sub-block can be performed in parallel and the number of cycles required to process a block in hardware can be reduced.

[0184] The above-described image encoding device 100 and image decoding device 200 can be implemented by a program that causes a computer to execute each function (each step).

[0185] In addition, in the above-mentioned embodiments, the present invention is described by taking the application to the image encoding device 100 and the image decoding device 200 as an example, but the present invention is not limited to this and can also be applied to an image encoding / decoding system having the functions of the image encoding device 100 and the image decoding device 200.

[0186] Explanation of symbols

[0187] 10…image processing system; 100…image encoding device; 111, 241…inter-frame prediction unit; 112, 242…intra-frame prediction unit; 112A…intra-frame prediction mode determination unit; 112B…intra-frame prediction mode encoding unit; 112C, 242C…prediction signal generation unit; 121…subtractor; 122, 230…adder; 131…transformation and quantization unit; 132, 220…inverse transformation and inverse quantization unit; 140…encoding unit; 150, 250…in-loop filtering processing unit; 160, 260…frame buffer; 200…image decoding device; 210…decoding unit; 242B…intra-frame prediction mode decoding unit.

Claims

1. An image decoding device, characterized in that: The image decoding device includes a prediction signal generating unit that generates a prediction signal based on intra-frame prediction. When generating prediction signals for all sub-blocks in a block to which intra-frame sub-division is applied, the prediction signal generating unit refers to pixels of a decoded block adjacent to the block when the width of the sub-block is smaller than a first threshold, and refers to decoded pixels of a sub-block processed immediately before when the width of the sub-block is greater than the first threshold. The prediction mode of each sub-block in the same block is common.

2. An image encoding device, characterized in that: The image encoding device includes a prediction signal generating unit that generates a prediction signal based on intra-frame prediction. When generating prediction signals for all sub-blocks in a block to which intra-frame sub-division is applied, the prediction signal generating unit refers to pixels of a decoded block adjacent to the block when the width of the sub-block is smaller than a first threshold, and refers to decoded pixels of a sub-block processed immediately before when the width of the sub-block is greater than the first threshold. The prediction mode of each sub-block in the same block is common.

3. An image decoding method, characterized in that: The image decoding method comprises the steps of generating a prediction signal based on intra-frame prediction, In the step, when generating prediction signals of all sub-blocks in a block to which intra-frame sub-division is applied, when the width of the sub-block is less than a first threshold, reference is made to pixels of a decoded block adjacent to the block, and when the width of the sub-block is greater than the first threshold, reference is made to decoded pixels of a sub-block processed immediately before. The prediction mode of each sub-block in the same block is common.

4. A program product comprising a program for use in an image decoding device, characterized in that: The program causes a computer to execute the step of generating a prediction signal based on intra-frame prediction, In the step, when generating prediction signals of all sub-blocks in a block to which intra-frame sub-division is applied, when the width of the sub-block is less than a first threshold, reference is made to pixels of a decoded block adjacent to the block, and when the width of the sub-block is greater than the first threshold, reference is made to decoded pixels of a sub-block processed immediately before. The prediction mode of each sub-block in the same block is common.

Images