Video encoding or decoding device and video encoding or decoding method

Abstract

The invention relates to a video encoding or decoding device and a video encoding or decoding method. When the range of use of an image used in intra prediction is expanded, there is a problem that a required buffer size increases in response to the expanded range. To solve this problem, the range of use of an image for use in intra prediction is adaptively controlled. This intra prediction device (100) is provided with a control unit (115) that, on the basis of the relationship between the position of a candidate image used for intra prediction of a block to be processed and the position of a unit to which the block to be processed belongs, controls a partial range so as to be equal to or less than a predetermined maximum range. The partial range is used in the intra-picture prediction of an end portion exceeding the unit in a predetermined direction in a use range of the image used in the intra-picture prediction.

Description

[0001] This application is a divisional application, and the original application it is targeting is a Chinese invention patent with an application date of February 14, 2019, the applicant is "NEC Corporation", the invention name is "Video encoding or decoding device, video encoding or decoding method, program and recording medium", and the application number is 201980041057.7. Technical Field

[0002] The present invention relates to a video encoding or decoding device, a video encoding or decoding method, a program for video encoding or decoding processing, and a recording medium. Background Art

[0003] In intra-screen prediction coding, an intra-screen prediction image is generated from a reconstructed image adjacent to a block to be processed. For example, according to the HEVC (High Efficiency Video Coding) standard described in Non-Patent Document 1, an intra-screen prediction image is generated by configuring a reconstructed image corresponding to 1 pixel adjacent to the block to be processed in the left direction and 1 pixel similarly adjacent in the upper direction as a reference range.

[0004] Citation List

[0005] Non-patent literature

[0006] Non-Patent Literature 1: R. Joshi et al., “High Efficiency Video Coding (HEVC) Screen Content Coding: Draft 5,” document JCTVC-V1005, Joint Collaboration Group on Video Coding (JCT-VC) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11, 22nd Meeting: Geneva, Switzerland, October 15-21, 2015. Summary of the invention

[0007] Technical issues

[0008] However, there is a problem that when the use range of images for intra-screen prediction is expanded, the requirements for the size of line buffers used in video encoding devices and video decoding devices increase depending on how much the range is expanded. Therefore, it is desirable to adaptively control the use range of images for intra-screen prediction.

[0009] Therefore, an exemplary object of the present invention is to provide a video encoding or decoding apparatus, a video encoding or decoding method, a program, and a recording medium that make it possible to adaptively control the use range of an image used for intra-screen prediction.

[0010] Solution to the problem

[0011] According to an exemplary aspect of the present invention, a video encoding or video decoding device includes a control unit configured to control a partial range to be equal to or smaller than a predetermined maximum range based on a relationship between a position of a candidate image used in intra-picture prediction of a block to be processed and a position of a unit having the block to be processed, the partial range being used for intra-picture prediction beyond an end of the unit in a predetermined direction in a usage range of an image used for intra-picture prediction.

[0012] According to an exemplary aspect of the present invention, a video encoding or video decoding method includes controlling a partial range to be equal to or smaller than a predetermined maximum range based on a relationship between a position of a candidate image used in intra-picture prediction for a block to be processed and a position of a unit having the block to be processed, the partial range being used in intra-picture prediction beyond an end of the unit in a predetermined direction in a usage range of an image used in the intra-picture prediction.

[0013] According to an exemplary aspect of the present invention, a program for causing a processor to perform a video encoding or decoding process, the video encoding or decoding process including controlling a partial range to be equal to or less than a predetermined maximum range based on a relationship between a position of a candidate image used in intra-picture prediction for a block to be processed and a position of a unit having the block to be processed, the partial range being used for intra-picture prediction beyond an end of the unit in a predetermined direction in a usage range of an image for intra-picture prediction.

[0014] According to an exemplary aspect of the present invention, a non-transitory computer-readable recording medium stores a program that causes a processor to perform a video encoding or decoding process, the video encoding or decoding process including controlling a partial range to be equal to or smaller than a predetermined maximum range based on a relationship between a position of a candidate image used in intra-picture prediction for a block to be processed and a position of a unit having the block to be processed, the partial range being used for intra-picture prediction beyond an end of the unit in a predetermined direction in a use range of an image for intra-picture prediction.

[0015] Advantageous Effects of the Invention

[0016] According to an exemplary aspect of the present invention, it is possible to adaptively control the use range of images used for intra-picture prediction. Note that according to the present invention, other advantageous effects may also be brought about instead of or together with the above advantageous effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1is a diagram showing a specific example of a reconstructed image used in intra-screen prediction according to the above-mentioned HEVC standard for a block to be processed consisting of 4 horizontal pixels “bw” and 4 vertical pixels “Bh”;

[0018] Figure 2 is a diagram showing a specific example of an extended reference range for a block to be processed consisting of 4 horizontal pixels bw and 4 vertical pixels Bh;

[0019] Figure 3 is an explanatory diagram showing an example of a schematic configuration of an intra-screen prediction apparatus 100 according to an exemplary embodiment of the present invention;

[0020] Figure 4 is a block diagram showing an example of a schematic configuration of a region control processing section 110 according to the first exemplary embodiment;

[0021] Figure 5 is a diagram for explaining a specific example of processing related to the area control processing unit 110;

[0022] Figure 6 is a flowchart illustrating an example of the flow of processing performed by the intra-frame prediction apparatus 100;

[0023] Figure 7 (a) and 7(b) are diagrams for explaining the effects of the example according to the first exemplary embodiment;

[0024] Figure 8 is a block diagram showing a schematic configuration of a video encoding apparatus 800;

[0025] Fig. 9 is a block diagram showing a schematic configuration of a video decoding apparatus 900;

[0026] Fig.10 is a block diagram showing a schematic configuration of an information processing system 1000 to which the intra-screen prediction apparatus 100 is applied;

[0027] Fig.11 is a diagram showing a system in which the above-described video encoding device 800 and video decoding device 900 are connected via a transmission path 300 such as a wireless transmission path or a wired transmission path;

[0028] Fig.12 is a block diagram showing an example of a schematic configuration of a video encoding device 800 according to the second exemplary embodiment; and

[0029] Fig.13 is a block diagram showing an example of a schematic configuration of a decoding device 900 according to the second exemplary embodiment. DETAILED DESCRIPTION

[0030] Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. Note that in the specification and the drawings, elements described with the same or similar terms are assigned the same reference numerals, and repeated descriptions may be omitted.

[0031] The descriptions will be given in the following order:

[0032] 1. Related technologies

[0033] 2. Overview of Example Embodiments

[0034] 3. First exemplary embodiment

[0035] 3.1. Configuration of the intra-frame prediction device 100

[0036] 3.2 Technical features

[0037] 3.3. Specific examples

[0038] 3.4. Sample Application

[0039] 3.5. Example Variations

[0040] 4. Second exemplary embodiment

[0041] 4.1. Configuration

[0042] 4.2 Technical features

[0043] 5. Other exemplary embodiments

[0044] << 1. Related technologies>>

[0045] As a technology related to an exemplary embodiment of the present invention, intra-screen prediction performed in video encoding and video decoding processes will be described.

[0046] As described in Reference 1 below, for example, with intra-screen prediction encoding according to the HEVC (High Efficiency Video Coding) standard, an intra-screen prediction image is generated from a reconstructed image adjacent to a block to be processed.

[0047] Reference 1: R. Joshi et al., “High Efficiency Video Coding (HEVC) Screen Content Coding: Draft 5,” document JCTVC-V1005, Joint Collaboration Team on Video Coding (JCT-VC) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11, 22nd Meeting: Geneva, Switzerland, October 15-21, 2015.

[0048] Figure 1 : is a diagram showing a specific example of a reconstructed image used in intra-screen prediction according to the above-mentioned HEVC standard for a block to be processed consisting of 4 horizontal pixels "bw" and 4 vertical pixels "Bh". Figure 1 As shown, the reference range of the reconstructed image to be referred to in the intra-screen prediction is 1 pixel in the left direction Kleft and 1 pixel in the upper direction Kup.

[0049] Furthermore, in the following references 2 and 3, there are proposed suggestions for expanding a reference range for intra-screen prediction in order to improve prediction efficiency in intra-screen prediction.

[0050] Reference 2: J. Pfaff et al., “Intra-prediction modes based on neural networks,” JVET-J0037, Joint Video Experts Group (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC29 / WG 11, 10th Meeting: San Diego, USA, April 10-20, 2018.

[0051] Reference 3: P. Lin et al., “Multiple reference line intra-prediction based on JEM7.0”, JVET-J0070, Joint Video Experts Group (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11, 10th Meeting: San Diego, USA, April 10-20, 2018.

[0052] Figure 2 1 is a diagram showing a specific example of an extended reference range for a block to be processed consisting of 4 horizontal pixels bw and 4 vertical pixels Bh. Figure 2 As shown, the reference range of the reconstructed image to be referred to is 4 pixels in the left direction Kleft and 4 pixels in the upper direction Kup.

[0053] <2. Overview of Example Embodiments>

[0054] First, an overview of exemplary embodiments of the present invention will be described.

[0055] (1) Technical Problem As described above, there is a problem that, when a usage range (reference range) of an image used in intra-screen prediction is expanded, a requirement for the size of a line buffer used in a video encoding device and a video decoding device increases.

[0056] More specifically, setting the number of horizontal pixels of the picture to be encoded to w pixels and the pixel bit precision to bitDepth bits, if the range Kup of the reference image used in intra-picture prediction in the upward direction is extended from 1 pixel to 4 pixels, the requirement for the line buffer capacity size increases from w*bitDepth bits to w*bitDepth*4 bits. Therefore, it is desirable to adaptively control the use range of the image used in intra-picture prediction.

[0057] Therefore, an exemplary object of the present exemplary embodiment is to adaptively control the use range of an image used in intra-screen prediction.

[0058] (2) Technical characteristics

[0059] In an example embodiment according to an example aspect of the present invention, for example, based on the relationship between the position of a candidate image used in intra-picture prediction of a block to be processed and the position of a unit to which the block to be processed belongs, a partial range is controlled to be equal to or smaller than a predetermined maximum range, the partial range being a use range used in intra-picture prediction in an intra-picture prediction beyond an end of the unit in a predetermined direction.

[0060] In this way, for example, it is possible to adaptively control the usage range of images used in intra-screen prediction.

[0061] Note that the above-described technical features are one specific example pertaining to the exemplary embodiments of the present invention, and it is apparent that the exemplary embodiments of the present invention are not limited to the above-described technical features.

[0062] <3. First Exemplary Embodiment>

[0063] The following will refer to Figures 3 to 11 A first exemplary embodiment of the present invention is described.

[0064] <3.1. Configuration of Intra-frame Prediction Apparatus 100>

[0065] Will refer to Figure 3 An example configuration of the intra-screen prediction apparatus 100 according to the first exemplary embodiment is described. Figure 3 1 is an explanatory diagram showing an example of a schematic configuration of an intra-screen prediction apparatus 100 according to an exemplary embodiment of the present invention. Figure 3 The intra-frame prediction device 100 includes a region control processing unit 110 , a reference image generating unit 120 , and a predicted image generating unit 130 .

[0066] In the intra-screen prediction device 100 configured as described above, the region control processing unit 110 controls the use range (reference range) of the image (reconstructed image) used in the intra-screen prediction. The reference image generation unit 120 generates a reference image from the reconstructed image based on the use range controlled by the region control processing unit 110. The predicted image generation unit 130 generates a predicted image from the reference image generated by the reference image generation unit 120.

[0067] The following will refer to Figure 4 An example configuration of the area control processing section 110 is described. Figure 4 1 is a block diagram showing an example of a schematic configuration of the area control processing section 110 according to the first exemplary embodiment. Figure 4 , the area control processing section 110 includes a first derivation section 111, a second derivation section 113, and a control section 115. What specific operations are performed in each section will be described below.

[0068] < 3.2. Technical features>

[0069] Next, technical features of the first exemplary embodiment will be described below.

[0070] The area control processing unit 110 (control unit 115) controls a partial range to be equal to or smaller than a predetermined maximum range based on a relationship between a position of a candidate image used in intra-screen prediction of a block to be processed and a position of a unit to which the block to be processed belongs, the partial range being used in intra-screen prediction beyond an end of the unit in a predetermined direction in a usage range of an image used in intra-screen prediction.

[0071] The predetermined direction may be any direction, such as an upward direction in the picture or a left direction in the picture, but the following description will be given mainly assuming that the predetermined direction refers to the upward direction in the picture.

[0072] (1) Unit

[0073] A unit contains syntax structures for encoding pixel samples. Among other things, a unit is a coding tree unit included in a slice. Note that, as mentioned above, a unit includes a block to be processed and can therefore be considered a parent block. A unit can be referred to as a "parent block".

[0074] (2) Predetermined maximum range

[0075] More specifically, the predetermined maximum range is a maximum range beyond the end of the coding tree unit in a predetermined direction that can be used in intra-picture prediction.

[0076] For example, if the predetermined direction is an upward direction in a picture, the predetermined maximum range is a maximum range that can be used for (can be referenced for) intra-picture prediction from an upper end portion of a picture toward an upward direction, referring to an upper end portion of a coding tree unit.

[0077] (3) Derivation and application of the first boundary position

[0078] The region control processing unit 110 (first derivation unit 111) derives a first boundary position based on the position of the coding tree unit and a predetermined maximum range. The first boundary position is an image position that can be used for intra-picture prediction beyond the end of the coding tree unit in a predetermined direction and is farthest from the coding tree unit in the predetermined direction.

[0079] For example, if the predetermined direction is an upward direction in the picture, the region control processing section 110 (first derivation section 111) derives the farthest position in the upward direction in the picture as the first boundary position with reference to the upper end portion of the coding tree unit based on the position of the upper end portion of the coding tree unit and the predetermined maximum range.

[0080] When the first boundary position is derived in this manner, the region control processing section 110 (control section 115) controls a partial range for use in intra-screen prediction beyond an end of a coding tree unit in a predetermined direction to be equal to or smaller than a predetermined maximum range based on the relationship between the position of the candidate image for use in intra-screen prediction and the first boundary position.

[0081] For example, when candidate images for use in intra-screen prediction include candidate images farther than the first boundary position from the position of the block to be processed, the region control processing section 110 (control section 115 ) controls the partial range to be equal to or smaller than a predetermined maximum range.

[0082] (4) Derivation and application of the second boundary position

[0083] The region control processing section 110 (second deriving section 113) derives a second boundary position based on the position of the block to be processed and the candidate range of the candidate image for use in the intra-screen prediction in a predetermined direction, the second boundary position being the position of the candidate image for use in the intra-screen prediction and being farthest from the block to be processed in the predetermined direction.

[0084] For example, if the predetermined direction is the upward direction of the picture, the region control processing unit 110 (second derivation unit 113) derives the candidate position of the farthest candidate image in the upward direction in the picture as the second boundary position, with reference to the upper end portion of the block to be processed, based on the position of the upper end portion of the block to be processed and the candidate range of the candidate image used in the intra-screen prediction in the upward direction of the picture.

[0085] When the second boundary position is derived in this way, the region control processing section 110 (control section 115) controls the partial range used in intra-screen prediction beyond the end of the coding tree unit in a predetermined direction to be equal to or smaller than a predetermined maximum range based on the relationship between the first and second boundary positions.

[0086] More specifically, when the second boundary position is farther than the first boundary position in a predetermined direction relative to the position of the block to be processed, the region control processing unit 110 (control unit 115) controls the partial range used in the intra-screen prediction beyond the end of the coding tree unit in the predetermined direction to be equal to or smaller than a predetermined maximum range.

[0087] For example, in a case where the predetermined direction is an upward direction in the picture, the region control processing section 110 (control section 115) controls the partial range to be smaller than a predetermined maximum range when the second boundary position relative to the position of the upper end portion of the block to be processed is farther in the upward direction in the picture than the first boundary position.

[0088] Furthermore, when the second boundary position is not farther than the first boundary position in a predetermined direction relative to the position of the block to be processed, the region control processing section 110 (control section 115) controls a partial range used in intra-screen prediction beyond the end of the coding tree unit in the predetermined direction to be a candidate range.

[0089] For example, if the predetermined direction is the upward direction in the picture, when the second boundary position is not farther than the first boundary position in the predetermined direction relative to the position of the block to be processed, the region control processing section 110 (control section 115 ) controls the partial range as the candidate range.

[0090] < 3.3. Specific examples >

[0091] Next, a specific example of the process performed in the intra-screen prediction device 100 will be described below.

[0092] (1) Area control processing unit 110

[0093] A specific example of the processing related to the area control processing section 110 will be described below. Figure 5 It is a diagram for explaining a specific example of processing related to the area control processing unit 110.

[0094] First, the following variables will be defined.

[0095] like Figure 5As shown, in the upward direction of the picture, the predetermined maximum range that can be used (can be referenced) beyond the upper end portion of the coding tree unit 503 to which the target block 501 to be processed belongs is Kmax pixels. In addition, the candidate range of the candidate image 505 (reference image) used in the intra-screen prediction of the processing target block 501 in the upward direction in the picture is Kup pixels, and the candidate range in the left direction in the picture is Kleft pixels. Note that for this specific example, for ease of explanation, Kmax is smaller than Kup.

[0096] The use range adaptively controlled by the region control processing section 110 (control section 115 ), that is, the use range of the image (reconstructed image) used in the intra-screen prediction of the processing target block 501 in the upward direction in the picture is K pixels.

[0097] The picture will be defined as follows. The upper left end portion of the picture is the origin (x, y) = (0, 0) of the horizontal and vertical coordinate systems, the right direction in the picture is the +x direction, and the downward direction in the picture is the +y direction. In addition, the number of horizontal pixels in the picture is w pixels, and the number of vertical pixels is h pixels.

[0098] Next, the block to be processed will be defined as follows. First, the coordinates of the upper left corner position of the block to be processed are (cur_bx, cur_by). In addition, the number of horizontal pixels of the image block to be processed is cur_bw pixels, and the number of vertical pixels is cur_bh pixels.

[0099] Next, a coding tree unit will be defined as follows. The coordinates of the upper left corner position of the coding tree unit are (cur_cux, cur_cuy). The number of horizontal pixels of the coding tree unit is cuw pixels, and the number of vertical pixels is cuh pixels.

[0100] In addition, for the sake of convenience of explanation, cur_bw and cur_bh are respectively smaller than cuw and cuh. In addition, as mentioned above, the upper left end portion of the picture is the origin (0, 0) of the horizontal and vertical coordinate systems, so cur_cux and cur_cuy are respectively equal to or smaller than cur_bx and cur_by.

[0101] In this specific example, by using the variables defined above, the candidate range K of the image (reconstructed image) used (referenced) in the intra-picture prediction of the block to be processed is adaptively controlled based on the relationship between the candidate position of the image (reconstructed image) used (referenced) in the intra-picture prediction of the block to be processed and the position of the coding tree unit to which the block to be processed belongs.

[0102] First, the area control processing section 110 (first deriving section 111 ) derives the vertical coordinate ref_max_pos_y of the first boundary position by using the following formula 1:

[0103] ref_max_pos_y = cur_cuy – Kmax…(1)

[0104] Here, the vertical coordinate ref_max_pos_y of the first boundary position may be considered as a maximum value of a vertical position that may be used (may be referenced) beyond an upper end portion of the coding tree unit in an upward direction of the picture.

[0105] Furthermore, the area control processing section 110 (the second deriving section 113 ) derives the vertical coordinate cand_min_pos_y of the second boundary position by using the following formula 2:

[0106] cand_min_pos_y = cur_by – Kup…(2)

[0107] Here, as described above, in a picture with the upper end of the picture as the origin, the vertical coordinate axis y has a positive value in the downward direction, so that the vertical coordinate cand_min_pos_y of the second boundary position can be considered as the minimum value of the candidate position of the image (reconstructed image) used (referenced) in the intra-picture prediction of the block to be processed.

[0108] The region control processing section 110 (control section 115 ) adaptively controls the candidate range K by using the vertical coordinate ref_max_pos_y of the first boundary position and the vertical coordinate cand_min_pos_y of the second boundary position.

[0109] More specifically, when cand_min_pos_y < ref_max_pos_y holds, the region control processing section 110 (control section 115) calculates the use range K by the following formula 3, and controls the partial range used in intra-screen prediction beyond the upper end portion of the coding tree unit in the upward direction in the picture to be equal to or smaller than a predetermined range Kmax.

[0110] K = cur_by - cur_cuy + Kmax…(3)

[0111] Furthermore, when cand_min_pos_y ≥ ref_max_pos_y holds, the area control processing unit 110 (control unit 115 ) calculates the use range K by the following formula (4).

[0112] K = Kup … (4)

[0113] In this way, the area control processing section 110 (control section 115 ) can calculate the use range K based on the relative positional relationship between the vertical coordinate ref_max_pos_y at the first boundary position and the vertical coordinate cand_min_pos_y at the second boundary position, and control the partial range to be equal to or less than Kmax.

[0114] Note that in the above-mentioned horizontal and vertical coordinate systems, the upper left end of the picture is the origin, but this is by no means restrictive. For example, the upper end portion of the block to be processed may be the origin on the vertical coordinate axis. When the origin is determined in this way, the vertical coordinate of the second boundary position can be regarded as the maximum value of the candidate position of the image (reconstructed image) used (referenced) in the intra-picture prediction of the block to be processed.

[0115] (2) Reference Image Generating Unit 120

[0116] The reference image generation unit 120 generates an image (reference image) used (referenced) for intra-frame prediction based on the use range K calculated by the region control processing unit 110 .

[0117] More specifically, the reference image generation unit 120 copies an image whose vertical coordinate positions are from cur_by - Kmax to cur_by - 1 when K<Kup holds, and maps the copied image to a corresponding position corresponding to the vertical coordinate positions cur_by - Kup to cur_by - Kmax - 1 to generate an image (reference image) used (referenced) in intra-picture prediction.

[0118] In this way, the reference image generation unit 120 can copy the reconstructed image at the vertical position cur_by - Kmax - 1, instead of the reconstructed image at the vertical position cur_by - Kup to cur_by - Kmax - 1 which cannot be referenced beyond the boundary of the coding tree unit in the upward direction, and use the copied image as a reference image. Note that the reconstructed image from the vertical position cur_by - Kmax to cur_by - 1 can be used as a reference image based on the original.

[0119] In addition, when K = Kup, there is no vertical position that cannot be used (referenced) beyond the upper end of the coding tree unit in the upward direction in the picture, so that the reference image generation unit 120 can use the reconstructed image of the vertical positions cur_by - Kup to cur_by - 1 as a reference image as it is.

[0120] (3) Prediction Image Generating Unit 130

[0121] The prediction image generation unit 130 generates an intra-frame prediction image from the reference image provided by the reference image generation unit 120. Any intra-frame prediction image generation process may be applied to the generation of the intra-frame prediction image, for example, such as the intra-frame prediction image generation process described in any one of the above-mentioned references 1, 2, and 3.

[0122] (4) Processing Flow Executed by the Intra-frame Prediction Device 100

[0123] Figure 6 : is a flowchart explaining an example process flow performed by the intra-screen prediction apparatus 100 .

[0124] In step S601, the region control processing unit 110 adaptively controls the candidate range K of the image (reconstructed image) used (referenced) in the intra-picture prediction based on the relationship between the candidate position of the reconstructed image referenced in the intra-picture prediction of the block to be processed and the position of the coding tree unit to which the block to be processed belongs. Then, step S603 is performed.

[0125] In step S603, the reference image generation section 120 generates a reference image for generating an intra-screen prediction image of the block to be processed based on the relationship between Kmax and the K value calculated in the region control processing section 110. Then, step S605 is performed.

[0126] In step S605 , the predicted image generation unit 130 generates an intra-screen predicted image of the block to be processed from the reference image supplied from the reference image generation unit 120 .

[0127] (5) Effect

[0128] According to the above example, for example, Figure 7 As shown, by adaptively controlling the use range (reference range) of the image (reconstructed image) used in the intra-picture prediction of each block to be processed, the requirement for the line buffer size can be reduced.

[0129] Figure 7 1 is a diagram for explaining the effect of an example according to the first exemplary embodiment. Figure 7 (a) is a diagram showing an example (comparative example) in which the partial range for use beyond the end of the coding tree unit 701 is not limited to a predetermined maximum range Kmax or less. On the other hand, Figure 7 (b) is a diagram showing an example (this specific example) in which a partial range for use at an end beyond the coding tree unit 702 is limited to a predetermined maximum range Kmax or less.

[0130] exist Figure 7In the comparative example shown in (a), the use range of the image (reference image) in the upward direction of the picture is always Kup pixels on the line, so that the requirement for the line buffer size is w*bitDepth*Kup bits.

[0131] On the other hand, reference Figure 7 (b) shows an example in which the predetermined maximum range is set to Kmax = 1 to limit the partial range in the use range K to pixels on one line. Figure 7 In the specific example shown in (b), w*bitDepth*1 holds. Figure 7 In the specific example shown in (b), compared with the comparative example ( Figure 7 Compared with the comparative example shown in (a), the requirement for the line buffer size can be reduced to w*bitDepth*(Kup - 1).

[0132] < 3.4. Sample Application>

[0133] (1) Video encoding device 800

[0134] For example, the intra-picture prediction apparatus 100 can be applied to Figure 8 A video encoding apparatus 800 is shown.

[0135] Figure 8 8 is a block diagram showing a schematic configuration of a video encoding device 800. Figure 8 As shown, the video encoding device 800 includes a conversion / quantization unit 801, an entropy encoding unit 802, an inverse conversion / dequantization unit 803, a buffer 804, a prediction unit 805 including the intra-screen prediction device 100, and a multiplexing unit 806.

[0136] First, the prediction section 805 generates a prediction signal for the input image signal on a per-block basis. More specifically, when intra-screen prediction is performed on a block to be processed, the intra-screen prediction apparatus 100 generates a prediction signal for the block to be processed as described above.

[0137] The conversion / quantization section 801 frequency-converts a prediction error image obtained by subtracting a prediction signal from an input image signal. Furthermore, the conversion / quantization section 801 quantizes the prediction error image (conversion coefficient) after the frequency conversion.

[0138] The entropy encoding section 802 entropy encodes difference information and the like between the converted quantized value and a motion vector as a prediction parameter used by the prediction section 805 based on, for example, CABAC (Context-based Adaptive Binary Arithmetic Coding).

[0139] The inverse conversion / dequantization unit 803 dequantizes the converted quantized value. In addition, the inverse conversion / dequantization unit 803 performs inverse frequency conversion on the dequantized frequency conversion coefficient. The prediction signal is added to the reconstructed prediction error image that has been inversely frequency converted, and the result is provided to the buffer 804. The buffer 804 stores the reconstructed image.

[0140] The multiplexing unit 806 multiplexes the code words supplied from the entropy coding unit 802 into a bit stream.

[0141] While the video encoding device 800 generates a bit stream following the above operation, the intra-picture prediction device 100 included in the prediction section 805 adaptively controls the use range of the image (reference image) used in the intra-picture prediction of each block to be processed. This makes it possible to output a video-encoded bit stream while reducing the requirement for the line buffer size.

[0142] (2) Video decoding device 900

[0143] The intra-frame prediction apparatus 100 can be applied to, for example, Fig. 9 A video decoding apparatus 900 is shown.

[0144] Fig. 9 9 is a block diagram showing a schematic configuration of a video decoding device 900. Fig. 9 As shown, the video decoding device 900 includes a demultiplexing unit 901, an entropy decoding unit 902, an inverse conversion / dequantization unit 903, a prediction unit 904 including the above-mentioned intra-screen prediction device 100, a buffer 905 and a control information generation unit 906.

[0145] The demultiplexing unit 901 demultiplexes the input bit stream and extracts a codeword.

[0146] The entropy decoding section 902 entropy decodes the codeword extracted in the demultiplexing section 901 based on, for example, CABAC. The converted quantized value that has been entropy decoded in the entropy decoding section 902 is supplied to the inverse conversion / dequantization section 903. In addition, motion vector difference information and the like are supplied to the prediction section 904.

[0147] The inverse conversion / dequantization section 903 dequantizes the converted quantized value at the quantization step size. In addition, the inverse conversion / dequantization section 903 performs inverse frequency conversion on the dequantized frequency conversion coefficient.

[0148] The prediction section 904 generates a prediction signal on a per-block basis. When intra-screen prediction is performed on a block to be processed, as described above, the intra-screen prediction apparatus 100 generates a prediction signal for the block to be processed.

[0149] The prediction signal supplied from the prediction section 904 is added to the reconstructed prediction error image subjected to inverse frequency conversion in the inverse conversion / dequantization section 903, and the resulting reconstructed picture is supplied to the buffer 905. Then, the reconstructed picture stored in the buffer 905 is output as a decoded image.

[0150] While the video decoding device 900 generates a decoded image from a bit stream following the above-described operation, the intra-screen prediction device 100 included in the prediction section 904 adaptively controls the use range (reference range) of an image (reconstructed image) used in intra-screen prediction for each block to be processed. This makes it possible to generate a decoded image from a bit stream while reducing the requirement for the line buffer size.

[0151] (3) Information Processing System 1000

[0152] For example, the intra-frame prediction apparatus 100 may be composed of Fig.10 The information handling system 1000 is shown implemented.

[0153] Fig.10 is a block diagram showing a schematic configuration of an information processing system 1000 to which the intra-screen prediction apparatus 100 is applied.

[0154] like Fig.10 As shown, the information processing system 1000 has a processor 1001, a program memory 1002, a storage medium 1003 for storing video data, and a storage medium 1004 for storing a bit stream. The storage medium 1003 and the storage medium 1004 may be separate storage media, or may be storage areas formed by the same storage medium. A magnetic storage medium such as a hard disk may be used as the storage medium.

[0155] The information processing system 1000 adaptively controls the use range (reference range) of the image (reconstructed image) used in the intra-picture prediction for each block to be processed by installing a computer program for implementing the functions of the intra-picture prediction device 100 in the program memory 1002. This makes it possible to generate a decoded image from a bit stream while reducing the requirement for the line buffer size.

[0156] (4) Interoperability

[0157] Fig.11 is a diagram showing a system in which the above-mentioned video encoding device 800 and the above-mentioned video decoding device 900 are connected through a transmission path 300 such as a wireless transmission path or a wired transmission path.

[0158] exist Fig.11In the system shown, the video encoding device 800 and the video decoding device 900 follow common processing such as using common Kmax, Kup, Kleft, etc., and for each block to be processed, the range of the reference image used in intra-picture prediction is adaptively controlled, so that interoperability between the video encoding device 800 and the video decoding device 900 can be ensured.

[0159] More specifically, for the predetermined maximum value Kmax, a fixed value common to the video encoding device 800 and the video decoding device 900 may be used. In addition, the predetermined maximum value Kmax may be a variable value implicitly set based on the number of horizontal and vertical pixels of a picture, etc., and, for example, as the number of pixels in a picture increases, a smaller value may be set.

[0160] As described above, if the predetermined maximum value Kmax is a variable value, it is not necessary to implicitly set the predetermined maximum value Kmax. For example, the value can be explicitly signaled as a syntax element of the bitstream. That is, information for specifying the predetermined maximum range Kmax can be included in the bitstream as a syntax element. For example, information for specifying the predetermined maximum range Kmax can be included in each sequence, each picture, each slice, each unit, etc.

[0161] Furthermore, the candidate ranges Kup and Kleft may be variable values. In this case, the candidate ranges Kup and Kleft may be explicitly signaled as syntax elements of the bitstream.

[0162] < 3.5. Example Variations>

[0163] In this example embodiment, for an image used in intra-screen prediction, the usage range in the upward direction in the picture is adaptively controlled for each block to be processed, but this is by no means restrictive, and similarly, the usage range in the left direction in the picture can be adaptively controlled.

[0164] More specifically, the region control processing section 110 (control section 115) of the intra-screen prediction device 100 may control a partial range for use in intra-screen prediction that exceeds the left end of the unit in the left direction in the picture, among the use range of the image used in the intra-screen prediction, to be equal to or smaller than a predetermined maximum range. In this example variation, more specifically, the predetermined maximum range is a maximum range that can be used in intra-screen prediction that exceeds the left end of the coding tree unit in the left direction in the picture.

[0165] In addition, information for specifying the predetermined maximum range may be signaled from the video encoding device 800 to the video decoding device 900 as a syntax element of a bitstream.

[0166] <4. Second Exemplary Embodiment>

[0167] Next, we will refer to Fig.12 and Fig.13 A second exemplary embodiment of the present invention is described. While the above-described first exemplary embodiment is a specific exemplary embodiment, the second exemplary embodiment is a more generalized exemplary embodiment.

[0168] < 4.1. Configuration >

[0169] Fig.12 is a block diagram showing an example of a schematic configuration of a video encoding device 800 according to the second exemplary embodiment. Fig.12 , the video encoding device 800 includes a control unit 810.

[0170] Fig.13 is a block diagram showing an example of a schematic configuration of a video decoding device 900 according to the second exemplary embodiment. Fig.13 , the video decoding device 900 includes a control unit 910.

[0171] < 4.2. Technical features>

[0172] Next, technical features of the second exemplary embodiment will be described below.

[0173] With the second exemplary embodiment, the video encoding device 800 (control section 810) controls a partial range to be equal to or smaller than a predetermined maximum range, based on a relationship between a position of a candidate image used in intra-screen prediction for a block to be processed and a position of a unit to which the block to be processed belongs, the partial range being used in intra-screen prediction beyond an end of the unit in a predetermined direction in a usage range of an image used in intra-screen prediction.

[0174] For example, the video encoding device 800 may perform the operation of the intra-screen prediction device 100 according to the first exemplary embodiment.

[0175] In addition, the video decoding device 900 (control unit 910) controls a partial range to be equal to or smaller than a predetermined maximum range based on the relationship between the position of a candidate image used in intra-screen prediction for the block to be processed and the position of a unit to which the block to be processed belongs, the partial range being used in intra-screen prediction beyond an end of the unit in a predetermined direction in a usage range of an image used in intra-screen prediction.

[0176] For example, the video decoding device 900 may perform the operation of the intra-screen prediction device 100 according to the first exemplary embodiment.

[0177] The second exemplary embodiment has been described above. According to the second exemplary embodiment, for example, the use range of an image used in intra-screen prediction can be adaptively controlled.

[0178] < 5. Other Example Embodiments >

[0179] Although the exemplary embodiments of the present invention have been described above, the present invention is by no means limited to these exemplary embodiments. It will be appreciated by those skilled in the art that these exemplary embodiments are merely examples and that various changes may be made without departing from the scope and spirit of the present invention.

[0180] For example, the steps in the process described in this specification do not have to be performed in chronological order in the order described in the sequence diagram. For example, the steps in the process may be performed in an order different from the order shown in the sequence diagram, or the steps in the process may be performed in parallel. In addition, some steps in the process may be deleted, or additional steps may be added to the process.

[0181] In addition, a method may be provided to include processing of components (e.g., a first derivation unit, a second derivation unit, and / or a control unit) described in the specification, and a program may be provided to cause a processor to perform processing of these components. In addition, a non-transitory computer-readable recording medium recording the program may be provided. Obviously, such devices, modules, methods, programs, and non-transitory computer-readable recording media are included in the present invention.

[0182] Some or all of the example embodiments may be as described in the following supplementary notes, but are not limited to the following supplementary notes.

[0183] (Supplementary Note 1)

[0184] A video encoding or decoding device, comprising:

[0185] A control unit configured to control a partial range used in intra-picture prediction beyond an end of the unit in a predetermined direction in a usage range of an image used in intra-picture prediction to be equal to or smaller than a predetermined maximum range based on a relationship between a position of a candidate image used in intra-picture prediction of a block to be processed and a position of a unit having the block to be processed.

[0186] (Supplementary Note 2)

[0187] The video encoding or decoding apparatus according to Supplementary Note 1, wherein the unit includes a syntax structure for encoding pixel samples.

[0188] (Supplementary Note 3)

[0189] The video encoding or decoding apparatus according to Supplementary Note 1 or 2, wherein the unit is a coding tree unit included in a slice.

[0190] (Supplementary Note 4)

[0191] The video encoding or decoding device according to any one of Supplementary Notes 1 to 3, wherein the predetermined maximum range is a maximum range that can be used for intra-screen prediction beyond an end of the unit in a predetermined direction.

[0192] (Supplementary Note 5)

[0193] The video encoding or decoding device according to Supplementary Note 4 further includes a first deriving unit configured to derive a first boundary position based on the position of the unit and a predetermined maximum range, the first boundary position being an image position that can be used for intra-screen prediction beyond an end of the unit in a predetermined direction and is farthest from the unit in the predetermined direction, wherein

[0194] The control section is configured to control a partial range used in intra prediction beyond an end of the cell in a predetermined direction to be equal to or smaller than a predetermined maximum range based on a relationship between a position of the candidate image used in the intra prediction and the first boundary position.

[0195] (Supplementary Note 6)

[0196] The video encoding or decoding device according to Supplementary Note 5, further comprising a second deriving section configured to derive a second boundary position based on a position of the block to be processed and a candidate range of a candidate image used in intra-picture prediction in a predetermined direction, the second boundary position being a position of the candidate image used in intra-picture prediction and being farthest from the block to be processed in the predetermined direction,

[0197] The control unit is configured to control a partial range used in intra-screen prediction beyond an end of the cell in a predetermined direction to be equal to or smaller than a predetermined maximum range based on a relationship between the first boundary position and the second boundary position.

[0198] (Supplementary Note 7)

[0199] A video encoding or decoding device according to Supplementary Note 6, wherein the control unit is configured to control a partial range used in intra-screen prediction beyond the end of the unit in a predetermined direction to be equal to or smaller than a predetermined maximum range when the second boundary position is farther than the first boundary position in a predetermined direction relative to the position of the block to be processed.

[0200] (Supplementary Note 8)

[0201] A video encoding or decoding device according to Supplementary Note 6 or 7, wherein the control unit is configured to control a usage range of an image used in intra-screen prediction to a candidate range when the second boundary position is not farther than the first boundary position in a predetermined direction relative to the position of the block to be processed.

[0202] (Supplementary Note 9)

[0203] The video encoding or decoding device according to any one of Supplementary Notes 6 to 8, wherein:

[0204] The intended direction is the upward direction in the picture;

[0205] The predetermined maximum range is a maximum range Kmax that can be used for intra-picture prediction of a portion beyond the upper end of the unit in the upward direction in the picture;

[0206] The candidate range is a candidate range Kup of candidate images used in intra-screen prediction in the upward direction in the picture;

[0207] The first deriving section is configured to derive a vertical coordinate ref_max_pos_y of a first boundary position by using the following formula 1 based on a vertical coordinate axis having a positive value in a downward direction in the picture, wherein the upper end of the picture is the origin;

[0208] The second deriving section is configured to derive a vertical coordinate cand_min_pos_y of the second boundary position by using the following Formula 2 based on the vertical coordinate axis; and

[0209] The control section is configured to calculate a use range K of an image used in intra-picture prediction in an upward direction in a picture by using the following formula 3 when cand_min_pos_y < ref_max_pos_y holds, and control a partial range used in intra-picture prediction beyond an end of a unit in a predetermined direction to be equal to or smaller than a predetermined maximum range Kmax:

[0210] ref_max_pos_y = cur_by - Kup ... (Formula 1),

[0211] Where cur_by is the vertical coordinate of the upper part of the cell;

[0212] cand_min_pos_y = cur_cuy - Kmax ... (Formula 2),

[0213] where cur_cuy is the vertical coordinate of the upper portion of the cell; and

[0214] K = cur_by-cur_cuy + Kmax ... (Formula 3).

[0215] (Supplementary Note 10)

[0216] The video encoding or decoding device according to Supplementary Note 9, wherein the control section is configured to calculate a use range K of an image used in intra-screen prediction in an upward direction in the picture by using the following Formula 4 when cand_min_pos_y ≥ ref_max_pos_y:

[0217] K = Kup ... (Formula 4).

[0218] (Supplementary Note 11)

[0219] The video encoding or decoding device according to Supplementary Note 9 or 10, further including an image generating section configured to generate an image used in intra-screen prediction based on a usage range K of images used in intra-screen prediction in an upward direction in a picture.

[0220] (Supplementary Note 12)

[0221] A video encoding or decoding device according to Supplementary Note 11, wherein the image generation unit is configured to copy an image having a vertical coordinate position from cur_by - Kmax to cur_by - 1 when K < Kup holds, and map the copied image to respective positions corresponding to the vertical coordinate positions cur_by - Kup to cur_by - Kmax - 1 to generate an image used in intra-screen prediction.

[0222] (Supplementary Note 13)

[0223] A video encoding or decoding method, comprising:

[0224] Based on the relationship between the position of a candidate image used in the intra-picture prediction of the block to be processed and the position of a unit having the block to be processed, a partial range is controlled to be equal to or smaller than a predetermined maximum range, the partial range being used in the intra-picture prediction beyond the end of the unit in a predetermined direction in a usage range of the image used in the intra-picture prediction.

[0225] (Supplementary Note 14)

[0226] A program that causes a processor to perform a video encoding or decoding process, the video encoding or decoding process comprising:

[0227] Based on the relationship between the position of a candidate image used in the intra-picture prediction of the block to be processed and the position of a unit having the block to be processed, a partial range is controlled to be equal to or smaller than a predetermined maximum range, the partial range being used in the intra-picture prediction beyond the end of the unit in a predetermined direction in a usage range of the image used in the intra-picture prediction.

[0228] (Supplementary Note 15)

[0229] A non-transitory computer-readable recording medium storing a program, the program causing a processor to perform a video encoding or decoding process, the video encoding or decoding process comprising:

[0230] Based on the relationship between the position of a candidate image used in the intra-picture prediction of the block to be processed and the position of a unit having the block to be processed, a partial range is controlled to be equal to or smaller than a predetermined maximum range, the partial range being used in the intra-picture prediction beyond the end of the unit in a predetermined direction in a usage range of the image used in the intra-picture prediction.

[0231] This application claims priority based on Japanese Patent Application No. 2018-120872 filed on June 26, 2018, and the disclosure thereof including specification, drawings and abstract is incorporated herein by reference in its entirety.

[0232] Industrial Applicability

[0233] In a system for encoding or decoding video, it is possible to adaptively control the use range of an image used in intra-picture prediction.

[0234] Reference numerals list

[0235] 100 Intra-screen prediction device

[0236] 110 Area Control Processing Department

[0237] 111 first export section

[0238] 113 Second export section

[0239] 115, 810, 910 control unit

[0240] 120 reference image generation unit

[0241] 130 prediction image generation unit

[0242] 800 video encoding device

[0243] 900 video decoding device.

Claims

1. A video encoding device, comprising: a control device for controlling a use range in the upward direction of the image in use for intra-picture prediction of the block to be processed to be equal to or smaller than a first maximum range if the image used for intra-picture prediction of the block to be processed is located outside the end of the coding tree unit in the upward direction, or controlling the use range in the upward direction of the image in use for intra-picture prediction of the block to be processed to be equal to or smaller than a second maximum range if the image used for intra-picture prediction of the block to be processed is not located outside the end of the coding tree unit in the upward direction, wherein the second maximum range is wider than the first maximum range, wherein, when the use range is controlled to be equal to or less than the second maximum range, the use range includes a plurality of lines, and When the use range is controlled to be equal to or smaller than the second maximum range, all of the plurality of lines are located successively, and at least one of the plurality of lines is used for intra-screen prediction of the block.

2. The video encoding apparatus according to claim 1, further comprising: The entropy encoding device is used to perform entropy encoding on the quantization parameter of the block.

3. The video encoding apparatus according to claim 1, further comprising: Quantizing means for quantizing the transform coefficients of the block.

4. A video decoding device, comprising: a control device for controlling a use range in the upward direction of the image in use for intra-picture prediction of the block to be processed to be equal to or smaller than a first maximum range if the image used for intra-picture prediction of the block to be processed is located outside the end of the coding tree unit in the upward direction, or controlling the use range in the upward direction of the image in use for intra-picture prediction of the block to be processed to be equal to or smaller than a second maximum range if the image used for intra-picture prediction of the block to be processed is not located outside the end of the coding tree unit in the upward direction, wherein the second maximum range is wider than the first maximum range, wherein, when the use range is controlled to be equal to or less than the second maximum range, the use range includes a plurality of lines, and When the use range is controlled to be equal to or smaller than the second maximum range, all of the plurality of lines are located successively, and at least one of the plurality of lines is used for intra-screen prediction of the block.

5. The video decoding apparatus according to claim 4, further comprising: An entropy decoding device is used to perform entropy decoding on the quantization parameter of the block.

6. The video decoding apparatus according to claim 4, further comprising: Quantizing means for quantizing the transform coefficients of the block.

7. A video encoding method, comprising: if the image used for intra-picture prediction of the processed block is located outside the end of the coding tree unit in the upward direction, the use range in the upward direction of the image in use for intra-picture prediction of the block is controlled to be equal to or smaller than a first maximum range, or if the image used for intra-picture prediction of the block to be processed is not located outside the end of the coding tree unit in the upward direction, the use range in the upward direction of the image in use for intra-picture prediction of the block is controlled to be equal to or smaller than a second maximum range, wherein the second maximum range is wider than the first maximum range, wherein, when the use range is controlled to be equal to or less than the second maximum range, the use range includes a plurality of lines, and When the use range is controlled to be equal to or smaller than the second maximum range, all of the plurality of lines are located successively, and at least one of the plurality of lines is used for intra-screen prediction of the block. The video encoding method according to claim 7 , further comprising entropy encoding the quantization parameter of the block.

9. The video encoding method according to claim 7, further comprising quantizing transform coefficients of the block.

10. A video decoding method, comprising: if the image used for intra-picture prediction of the processed block is located outside the end of the coding tree unit in the upward direction, the use range in the upward direction of the image in use for intra-picture prediction of the block is controlled to be equal to or smaller than a first maximum range, or if the image used for intra-picture prediction of the block to be processed is not located outside the end of the coding tree unit in the upward direction, the use range in the upward direction of the image in use for intra-picture prediction of the block is controlled to be equal to or smaller than a second maximum range, wherein the second maximum range is wider than the first maximum range, wherein, when the use range is controlled to be equal to or less than the second maximum range, the use range includes a plurality of lines, and When the use range is controlled to be equal to or smaller than the second maximum range, all of the plurality of lines are located successively, and at least one of the plurality of lines is used for intra-screen prediction of the block. The video decoding method according to claim 10 , further comprising entropy decoding the quantization parameter of the block. 12 . The video decoding method according to claim 10 , further comprising quantizing transform coefficients of the block.

Images