Video encoding device, video decoding device, video encoding method, and video decoding method

Abstract

To provide a video encoding or video decoding device, a video encoding or video decoding method, a program, and a recording medium with which it is possible to adaptively control a use range of an image used for in-screen prediction.SOLUTION: An in-screen prediction device includes a control section that, on the basis of a relation between a position of a candidate image used for in-screen prediction for a processing target block and a position of a unit to which the processing target block belongs, controls a partial range used for the in-screen prediction over an end of the unit in a predetermined direction, within a use range of an image used for the in-screen prediction, to be equal to or less than a predetermined maximum range.SELECTED DRAWING: Figure 7

Description

Technical Field

[0001] The present invention relates to a video encoding or decoding apparatus, a video encoding or decoding method, a program for video encoding or decoding processing, and a recording medium.

Background Art

[0002] In intra prediction coding, an intra prediction image is generated from a reconstructed image adjacent to a processing target block. For example, in the HEVC (High Efficiency Video Coding) standard described in Non-Patent Document 1, by setting, as a reference range, the reconstructed images corresponding to one pixel adjacent to the processing target block in the left direction and one pixel adjacent to the processing target block in the upward direction, an intra prediction image is generated.

Prior Art Documents

Non-Patent Documents

[0003]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, expanding the range of images used for in-screen prediction presents a problem: the size requirements for the line buffers used in the video encoding and decoding devices increase proportionally to the expansion. Therefore, it is desirable to adaptively control the range of images used for in-screen prediction.

[0005] The object of the present invention is to provide a video encoding or decoding device, a video encoding or decoding method, a program, and a recording medium that enable adaptive control of the range of images used for in-screen prediction. [Means for solving the problem]

[0006] According to one aspect of the present invention, a video encoding or decoding device includes a control unit that controls the range of use of an image used for in-screen prediction for a block to be processed, the control unit controls the range of use to be less than or equal to a first maximum range that can be used for in-screen prediction, spanning the ends of the encoding tree unit upwards, and if the range of use is not controlled to be less than or equal to the first maximum range, the control unit controls the range of use to be less than or equal to a second maximum range that is greater than the first maximum range, and if the range of use is controlled to be less than or equal to the second maximum range, the range of use includes a plurality of lines, at least one of the plurality of lines is used for in-screen prediction for a block to be processed, and all of the plurality of lines are arranged consecutively.

[0007] According to one aspect of the present invention, a video encoding or decoding method includes controlling the range of use of an image used for in-screen prediction for a block to be processed, the control including controlling the range of use to be less than or equal to a first maximum range that is available for in-screen prediction spanning the ends of the encoding tree unit upwards, and, if the range of use is not controlled to be less than or equal to the first maximum range, controlling the range of use to be less than or equal to a second maximum range that is greater than the first maximum range, wherein, when the range of use is controlled to be less than or equal to the second maximum range, the range of use includes a plurality of lines, and at least one of the plurality of lines is used for in-screen prediction for a block to be processed, and all of the plurality of lines are arranged consecutively. [Effects of the Invention]

[0008] According to one aspect of the present invention, it becomes possible to adaptively control the range of images used for in-screen prediction. In addition, the present invention may produce other effects instead of, or in conjunction with, this effect. [Brief explanation of the drawing]

[0009] [Figure 1] Figure 1 shows a specific example of a reconstructed image used for in-screen prediction in the HEVC standard described above, for a processing block consisting of 4 horizontal pixels bw and 4 vertical pixels Bh. [Figure 2] Figure 2 shows a specific example of an expanded reference range for a processing target block consisting of 4 horizontal pixels bw and 4 vertical pixels Bh. [Figure 3] Figure 3 is an explanatory diagram showing an example of a schematic configuration of the in-screen prediction device 100 according to an embodiment of the present invention. [Figure 4] Figure 4 is a block diagram showing an example of a schematic configuration of the region control processing unit 110 according to the first embodiment. [Figure 5] Figure 5 is a diagram illustrating a specific example of processing related to the region control processing unit 110. [Figure 6] Figure 6 is a flowchart illustrating an example of the processing flow performed by the on-screen prediction device 100. [Figure 7] Figure 7 is a diagram illustrating the effects of the embodiment of the first embodiment. [Figure 8] Figure 8 is a block diagram showing the schematic configuration of the video encoding device 800. [Figure 9] Figure 9 is a block diagram showing the schematic configuration of the video decoding device 900. [Figure 10] Figure 10 is a block diagram showing the schematic configuration of the information processing system 100 to which the on-screen prediction device 100 is applied. [Figure 11]FIG. 11 is a diagram showing a system in which the above-described video encoding apparatus 800 and the above-described video decoding apparatus 900 are connected by a transmission path 300 such as a wireless transmission path or a wired transmission path. [Figure 12] FIG. 12 is a block diagram showing an example of a schematic configuration of a video encoding apparatus 800 according to the second embodiment. [Figure 13] FIG. 13 is a block diagram showing an example of a schematic configuration of a video decoding apparatus 900 according to the second embodiment.

Embodiments of the Invention

[0010] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the present specification and drawings, for elements that can be similarly described, duplicate description may be omitted by assigning the same reference numerals.

[0011] The description will be made in the following order. 1. Related Art 2. Outline of Embodiment 3. First Embodiment 3.1. Configuration of Intra-Prediction Device 100 3.2. Technical Features 3.3. Specific Example 3.4. Application Example 3.5. Modified Example 4. Second Embodiment 4.1. Configuration 4.2. Technical Features 5. Other Forms

[0012] <<1. Related Art>> As related art to the embodiments of the present invention, intra-prediction performed in video encoding processing and video decoding processing will be described.

[0013] As described in the following Reference 1, for example, in intra-prediction coding of the HEVC (High Efficiency Video Coding) standard, an intra-prediction image is generated from a reconstructed image adjacent to a processing target block. Reference 1: R. Joshi et al., "High Efficiency Video Coding (HEVC) Screen Content Coding: Draft 5" document JCTVC-V1005, Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11, 22nd Meeting: Geneva, CH, 15-21 Oct. 2015.

[0014] Figure 1 shows a specific example of a reconstructed image used for in-screen prediction in the HEVC standard described above, for a processing target block consisting of 4 horizontal pixels bw and 4 vertical pixels Bh. Also, as shown in Figure 1, the reference range of the reconstructed image referenced for in-screen prediction is 1 pixel to the left (Kleft) and 1 pixel to the top (Kup).

[0015] Furthermore, references 2 and 3 below propose expanding the reference range used for in-screen prediction in order to improve the prediction efficiency of in-screen prediction. Reference 2: J. Pfaff et al., "Intra prediction modes based on neural networks", JVET-J0037, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11 10th Meeting: San Diego, US, 10-20 Apr. 2018. Reference 3: P. Lin et al., "Multiple reference line intra prediction based on JEM7.0", JVET-J0070, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11 10th Meeting: San Diego, US, 10-20 Apr. 2018.

[0016] For example, Figure 2 shows a specific example of an expanded reference range for a processing target block consisting of 4 horizontal pixels bw and 4 vertical pixels Bh. As shown in Figure 2, the reference range for the reconstructed image consists of 4 pixels to the left (Kleft) and 4 pixels upward (Kup).

[0017] <<2. Overview of Embodiments>> First, an overview of the embodiments of the present invention will be described.

[0018] (1)Technical issues As mentioned above, expanding the range of images used for in-screen prediction (reference range) presents the problem of increasing the size requirements for the line buffers used in the video encoding and decoding devices.

[0019] Specifically, if the horizontal number of pixels in the picture to be encoded is denoted as w pixels, and the pixel bit precision as bitDepth bits, and the upward range Kup of the reference image used for in-screen prediction is expanded from 1 pixel to 4 pixels, the line buffer size requirement increases from w*bitDepth bits to w*bitDepth*4 bits. Therefore, it is desirable to adaptively control the range of the image used for in-screen prediction.

[0020] Therefore, the objective of this embodiment is to adaptively control the range of images used for in-plane prediction.

[0021] (2) Technical features In one embodiment of the present invention, for example, based on the relationship between the position of a candidate image used for in-screen prediction for a block to be processed and the position of the unit to which the block to be processed belongs, the portion of the image used for in-screen prediction that spans the end of the unit in a predetermined direction is controlled to be less than or equal to a predetermined maximum range.

[0022] This makes it possible, for example, to adaptively control the range of images used for in-screen prediction.

[0023] It should be noted that the technical features described above are just one specific example of embodiments of the present invention, and naturally, embodiments of the present invention are not limited to the technical features described above.

[0024] <<3. First Embodiment>> A first embodiment of the present invention will be described with reference to Figures 3 to 11.

[0025] <3.1. Configuration of the on-screen prediction device 100> Referring to Figure 3, an example of the configuration of the in-screen prediction device 100 according to the first embodiment will be described. Figure 3 is an explanatory diagram showing an example of a schematic configuration of the in-screen prediction device 100 according to an embodiment of the present invention. Referring to Figure 3, the in-screen prediction device 100 includes a region control processing unit 110, a reference image generation unit 120, and a prediction image generation unit 130.

[0026] In the in-screen prediction device 100 having the above configuration, the region control processing unit 110 controls the usage range (reference range) of the image (reconstructed image) used for in-screen prediction. The reference image generation unit 120 generates a reference image from the reconstructed image based on the usage range controlled by the region control processing unit 110. The prediction image generation unit 130 generates a prediction image from the reference image generated by the reference image generation unit 120.

[0027] Referring to Figure 4, an example of the configuration of the region control processing unit 110 will be described. Figure 4 is a block diagram showing a schematic example of the configuration of the region control processing unit 110 according to the first embodiment. Referring to Figure 4, the region control processing unit 110 includes a first derivation unit 111, a second derivation unit 113, and a control unit 115. The specific operations performed by each unit will be described below.

[0028] <3.2. Technical Features> Next, the technical features of the first embodiment will be described.

[0029] The region control processing unit 110 (control unit 115) controls the portion of the image used for in-screen prediction that spans the edges of the unit in a predetermined direction, based on the relationship between the position of the candidate image used for in-screen prediction for the block to be processed and the position of the unit to which the block to be processed belongs, to be less than or equal to a predetermined maximum range.

[0030] The specified direction mentioned above can be any direction, such as the top of the screen or the left side of the screen, but in the following explanation, we will mainly assume that it is the top of the screen. (1) Unit The above unit contains a syntax structure for encoding pixel samples. In particular, the above unit is one encoding tree unit contained in a slice. Note that, as mentioned above, the above unit contains the block to be processed, so it can also be considered as a parent block. The above unit may also be called a "parent block".

[0031] (2) A predetermined maximum range The above-mentioned predetermined maximum range is, specifically, the maximum range that can be used for in-screen prediction, spanning the ends of the above-mentioned coding tree unit in the above-mentioned predetermined direction.

[0032] For example, if the predetermined direction is the upward direction of the screen, the predetermined maximum range is the maximum range that can be used (referenced) for in-screen prediction, with respect to the upper end of the coding tree unit, extending upward from the upper end of the unit.

[0033] (3) Derivation and application of the first boundary position The region control processing unit 110 (first derivation unit 111) derives a first boundary position that is the image position usable for in-screen prediction, which straddles the end of the encoding tree unit in the predetermined direction, and is furthest from the encoding tree unit in the predetermined direction, based on the position of the encoding tree unit and the predetermined maximum range.

[0034] For example, if the predetermined direction is the upward direction of the screen, the region control processing unit 110 (first derivation unit 111) derives the position furthest from the upper end of the coding tree unit in the upward direction of the screen, based on the position of the upper end of the coding tree unit and the predetermined maximum range, as the first boundary position.

[0035] When the first boundary position is derived in this manner, the region control processing unit 110 (control unit 115) controls the portion range used for in-screen prediction, which spans the end of the coding tree unit in the predetermined direction, to be less than or equal to the predetermined maximum range, based on the relationship between the position of the candidate image used for in-screen prediction and the first boundary position.

[0036] The region control processing unit 110 (control unit 115) controls the partial range to be less than or equal to the predetermined maximum range if, for example, there is a candidate image among the candidate images used for in-screen prediction that is further away from the first boundary position when viewed from the position of the block to be processed.

[0037] (4) Derivation and application of the second boundary position The region control processing unit 110 (second derivation unit 113) derives a second boundary position, which is the position of the candidate image used for the in-screen prediction and is furthest from the processing target block in the predetermined direction, based on the position of the block to be processed and the candidate range of the candidate image used in the predetermined direction in the in-screen prediction.

[0038] For example, if the predetermined direction is the upward direction of the screen, the region control processing unit 110 (second derivation unit 113) derives the candidate position of the candidate image furthest from the upper end of the processing target block in the upward direction of the screen, based on the position of the upper end of the processing target block and the candidate range of the candidate image used in the upward direction of the screen in the in-screen prediction, as the second boundary position.

[0039] When the second boundary position is derived in this manner, the region control processing unit 110 (control unit 115) controls the portion range used for in-screen prediction, which spans the end of the coding tree unit in the predetermined direction, to be less than or equal to the predetermined maximum range, based on the relationship between the first boundary position and the second boundary position.

[0040] Specifically, the region control processing unit 110 (control unit 115) controls the portion range used for in-screen prediction, which spans the end of the coding tree unit in the predetermined direction, to be less than or equal to the predetermined maximum range, if the second boundary position is further away in the predetermined direction than the first boundary position with respect to the position of the block to be processed.

[0041] For example, if the predetermined direction is the upward direction of the screen, the region control processing unit 110 (control unit 115) controls the partial range to be less than or equal to the predetermined maximum range if the second boundary position is further upward in the screen direction than the first boundary position with respect to the upper end position of the block to be processed.

[0042] Furthermore, the region control processing unit 110 (control unit 115) controls the portion range used for in-screen prediction, which spans the end of the coding tree unit in the predetermined direction, to the candidate range, if the second boundary position is not further away in the predetermined direction than the first boundary position with respect to the position of the block to be processed.

[0043] For example, if the predetermined direction is the upward direction of the screen, the region control processing unit 110 (control unit 115) controls the partial range to the candidate range if the second boundary position is not further upward in the screen direction than the first boundary position with respect to the position of the block to be processed.

[0044] <3.3.Specific Examples> Next, we will explain a specific example of the processing performed by the on-screen prediction device 100.

[0045] (1) Region control processing unit 110 A specific example of the processing related to the region control processing unit 110 will be explained. Figure 5 is a diagram illustrating a specific example of the processing related to the region control processing unit 110.

[0046] First, define the variables as follows:

[0047] As shown in Figure 5, the maximum range that can be used (referenced) across the upper end of the coding tree unit 503 to which the processing target block 501 belongs is defined as Kmax pixels. Furthermore, the candidate range in the upper direction of the screen of the candidate image 505 (reference image) used for in-screen prediction of the processing target block 501 is defined as Kup pixels, and the candidate range in the left direction of the screen is defined as Kleft pixels. In this specific example, for the sake of simplicity, Kmax is assumed to be less than Kup.

[0048] The usage range that the region control processing unit 110 (control unit 115) adaptively controls, that is, the usage range in the upward direction of the image (reconstructed image) used for in-screen prediction of the processing target block 501, is defined as K pixels.

[0049] Regarding pictures, they are defined as follows: The upper left corner of the picture is the origin of the horizontal-vertical coordinate system (x,y)=(0,0), the rightward direction of the screen is +x, and the downward direction of the screen is +y. Also, the horizontal number of pixels in the picture is w pixels, and the vertical number of pixels is h pixels.

[0050] Next, the blocks to be processed are defined as follows. First, the coordinates of the upper left corner of the block to be processed are (cur_bx, cur_by). Also, the horizontal number of pixels of the image block to be processed is cur_bw pixels, and the vertical number of pixels is cur_bh pixels.

[0051] Next, the above coding tree unit is defined as follows: The coordinates of the upper left corner of the above coding tree unit are (cur_cux, cur_cuy). Also, the number of horizontal pixels of the above coding tree unit is cuw pixels, and the number of vertical pixels is cuh pixels.

[0052] For the sake of simplicity, cur_bw and cur_bh are assumed to be less than cuw and less than cuh, respectively. Also, as mentioned above, the upper left corner of the picture is the origin (0,0) of the horizontal and vertical coordinate system, so cur_cux and cur_cuy are less than or equal to cur_bx and less than or equal to cur_by, respectively.

[0053] In this specific example, the variables defined above are used to adaptively control the candidate range K of the image (reconstructed image) used (referenced) in the in-screen prediction of the above-mentioned block, based on the relationship between the candidate position of the image (reconstructed image) used (referenced) in the in-screen prediction of the above-mentioned block and the position of the coding tree unit to which the above-mentioned block belongs.

[0054] First, the region control processing unit 110 (first derivation unit 111) derives the vertical coordinate ref_max_pos_y of the first boundary position using the following equation (1). ref_max_pos_y=cur_cuy-Kmax ··· (1)

[0055] Here, the vertical coordinate ref_max_pos_y of the first boundary position can be considered as the maximum vertical position that can be used (referenced) across the upper end of the encoding tree unit in the upward direction of the screen.

[0056] Further, the area control processing unit 110 (second derivation unit 113) derives the vertical coordinate cand_min_pos_y of the second boundary position using the following formula (2). cand_min_pos_y = cur_by - Kup ··· (2)

[0057] Here, as described above, since the vertical coordinate axis y has the upper end of the picture as the origin and takes positive values in the downward direction of the screen, the vertical coordinate cand_min_pos_y of the second boundary position can be regarded as the minimum value of the candidate position of the image (reconstructed image) used (referenced) for in-screen prediction of the processing target block.

[0058] The area control processing unit 110 (control unit 115) adaptively controls the candidate range K using the vertical coordinate ref_max_pos_y of the first boundary position and the vertical coordinate cand_min_pos_y of the second boundary position.

[0059] Specifically, when cand_min_pos_y < ref_max_pos_y, the area control processing unit 110 (control unit 115) calculates the usage range K according to the following formula (3), and controls the partial range used for the in-screen prediction across the upper end of the encoding tree unit in the upward direction of the screen to be below the predetermined range Kmax. K = cur_by - cur_cuy + Kmax ··· (3)

[0060] Further, when cand_min_pos_y ≥ ref_max_pos_y, the area control processing unit 110 (control unit 115) calculates the usage range K according to the following formula (4). K = Kup ··· (4)

[0061] In this way, the area control processing unit 110 (control unit 115) can calculate the usage range K according to the relative positional relationship between the vertical coordinate ref_max_pos_y of the first boundary position and the vertical coordinate cand_min_pos_y of the second boundary position, and control the partial range to be below Kmax.

[0062] In the horizontal-vertical coordinate system as described above, the upper left corner of the picture is taken as the origin, but it is not limited to this. For example, the upper end of the processing target block may be taken as the origin of the vertical coordinate axis. When the origin is determined in this way, the vertical coordinate of the second boundary position can be regarded as the maximum value of the candidate positions of the image (reconstructed image) used (referenced) for the in-picture prediction of the processing target block.

[0063] (2) Reference Image Generation Unit 120 The reference image generation unit 120 generates an image (reference image) used (referenced) for the in-picture prediction based on the usage range K calculated by the above-described region control processing unit 110.

[0064] Specifically, when K < Kup, the reference image generation unit 120 copies the image whose vertical coordinate position belongs to cur_by - Kmax to cur_by - 1, and maps the copied image to the positions corresponding to cur_by - Kup to cur_by - Kmax - 1, thereby generating an image (reference image) used (referenced) for the in-picture prediction of the processing target block.

[0065] In this way, the reference image generation unit 120 can copy the reconstructed image at the vertical position cur_by - Kmax - 1 instead of the reconstructed images at the vertical positions cur_by - Kup to cur_by - Kmax - 1 that cannot be referenced across the upper boundary of the encoding tree unit, and use the copied image as the reference image. The reconstructed images at the vertical positions cur_by - Kmax to cur_by - 1 may be directly used as the reference images.

[0066] Also, when K = Kup, since there is no vertical position that cannot be used (referenced) across the upper end of the encoding tree unit in the upward direction of the screen, the reconstructed images at the vertical positions cur_by - Kup to cur_by - 1 can be directly used as the reference images.

[0067] (3) Predicted Image Generation Unit 130 The prediction image generation unit 130 generates an in-screen prediction image from the reference image supplied by the reference image generation unit 120. The generation of the in-screen prediction image may be performed using any in-screen prediction image generation process, such as the in-screen prediction image generation process described in any of the above-mentioned references 1, 2, or 3.

[0068] (4) Flow of processing performed by the on-screen prediction device 100 Figure 6 is a flowchart illustrating an example of the processing flow performed by the on-screen prediction device 100.

[0069] In step S601, the region control processing unit 110 adaptively controls the candidate range K of the image (reconstructed image) used (referenced) in the in-screen prediction, based on the relationship between the candidate position of the reconstructed image referenced in the in-screen prediction of the block to be processed and the position of the coding tree unit to which the block to be processed belongs. The process then proceeds to step S603.

[0070] In step S603, the reference image generation unit 120 generates a reference image to be used for generating the in-screen prediction of the processing target block, based on the relationship between Kmax and the value of K calculated by the region control processing unit 110. The process then proceeds to step S605.

[0071] In step S605, the prediction image generation unit 130 generates a predicted image of the screen for the processing target block from the reference image supplied by the reference image generation unit 120.

[0072] (5) Effects According to this embodiment, by adaptively controlling the usage range (reference range) of the image (reconstructed image) used for in-screen prediction for each processing block, the line buffer size requirement can be reduced, for example, as shown in Figure 7.

[0073] Figure 7 is a diagram illustrating the effects of an embodiment of the first embodiment. First, Figure 7(a) is a diagram illustrating an example (comparative example) in which the partial range used across the ends of the coding tree unit 701 is not limited to the predetermined maximum range Kmax or less. On the other hand, Figure 7(b) is a diagram illustrating an example (this specific example) in which the partial range used across the ends of the coding tree unit 702 is controlled to be less than or equal to the predetermined maximum range Kmax or less.

[0074] In the comparative example shown in Figure 7(a), the usable range of the image (reference image) in the upper direction of the screen is always the pixels on Kup lines, so the line buffer size requirement is w*bitDepth*Kup bits.

[0075] On the other hand, the example shown in Figure 7(b) shows an example in which the above-mentioned maximum range Kmax=1 restricts the above-mentioned portion of the usage range K to pixels on one line. Therefore, in this specific example shown in Figure 7(b), it becomes w*bitDepth bits*1. In other words, this specific example shown in Figure 7(b) can reduce the line buffer size requirement to w*bitDepth*(Kup-1) compared to the above comparative example (the comparative example shown in Figure 7(a)).

[0076] <3.4. Application Examples> (1) Video encoding device 800 The in-screen prediction device 100 described above can be applied to, for example, a video encoding device 800 as shown in Figure 8.

[0077] Figure 8 is a block diagram showing the schematic configuration of the video encoding device 800. As shown in Figure 8, the video encoding device 800 comprises a conversion / quantization unit 801, an entropy encoding unit 802, an inverse conversion / inverse quantization unit 803, a buffer 804, a prediction unit 805 that includes the in-screen prediction device 100, and a multiplexing unit 806.

[0078] First, the prediction unit 805 generates a prediction signal for each block of the input image signal. Specifically, when performing in-screen prediction for a block to be processed, the in-screen prediction device 100 generates a prediction signal for the block to be processed, as described above.

[0079] The conversion / quantization unit 801 frequency-converts the prediction error image obtained by subtracting the prediction signal from the input image signal. Furthermore, the conversion / quantization unit 801 quantizes the frequency-converted prediction error image (conversion coefficient).

[0080] The entropy coding unit 802 entropy codes the transformed quantized values ​​and the difference information of the motion vectors, which are prediction parameters used by the prediction unit 805, based on, for example, CABAC (Context-based Adaptive Binary Arithmetic Coding).

[0081] The inverse transform / inverse quantization unit 803 inversely quantizes the transformed quantized values. Furthermore, the inverse transform / inverse quantization unit 803 inversely transforms the inversely quantized frequency transform coefficients. The reconstructed prediction error image, which has undergone inverse frequency transformation, has the prediction signal added to it and is supplied to the buffer 804. The buffer 804 stores the reconstructed image.

[0082] The multiplexing unit 806 multiplexes the codewords supplied from the entropy coding unit 802 as a bitstream.

[0083] The video encoding device 800, which generates a bitstream through the operations described above, has its usage range of the image (reference image) used for in-screen prediction adaptively controlled for each processing block by the in-screen prediction device 100, which is included in the prediction unit 805. This allows the video encoded bitstream to be output while reducing the line buffer size requirement.

[0084] (2) Video decoding device 900 The in-screen prediction device 100 described above can be applied to, for example, a video decoding device 900 as shown in Figure 9.

[0085] Figure 9 is a block diagram showing the schematic configuration of the video decoding device 900. As shown in Figure 9, the video decoding device 900 comprises a demultiplexing unit 901, an entropy decoding unit 902, an inverse transform / inverse quantization unit 903, a prediction unit 904 that includes the in-screen prediction device 100 described above, a buffer 905, and a control information generation unit 906.

[0086] The demultiplexing unit 901 demultiplexes the input bitstream and extracts the codewords.

[0087] The entropy decoding unit 902 entropy-decodes the codeword extracted by the demultiplexing unit 901, for example, based on CABAC. The entropy-decoded transformed quantized values ​​from the entropy decoding unit 902 are supplied to the inverse transform / inverse quantization unit 903. Difference information of motion vectors, etc., are supplied to the prediction unit 904.

[0088] The inverse transform / inverse quantization unit 903 inversely quantizes the transformed quantized value by the quantization step width. Furthermore, the inverse transform / inverse quantization unit 903 inversely transforms the inversely quantized frequency transform coefficients.

[0089] The prediction unit 904 generates prediction signals for each block. When performing in-screen prediction for a block to be processed, the in-screen prediction device 100 generates prediction signals for the block to be processed, as described above.

[0090] The reconstructed prediction error image, which has been inverse frequency-converted by the inverse transform / inverse quantization unit 903, is combined with the prediction signal supplied from the prediction unit 904 and supplied to the buffer 905 as a reconstructed picture. The reconstructed picture stored in the buffer 905 is then output as a decoded image.

[0091] The video decoding device 900, which generates a decoded image from a bitstream through the operations described above, has its usage range (reference range) of the image (reconstructed image) used for in-screen prediction adaptively controlled for each processing block by the in-screen prediction device 100, which is included in the prediction unit 904. This makes it possible to generate a decoded image from a bitstream while reducing the line buffer size requirement.

[0092] (3) Information processing system 1000 The on-screen prediction device 100 described above may be implemented by an information processing system 1000, for example, as shown in Figure 10.

[0093] Figure 10 is a block diagram showing the schematic configuration of the information processing system 1000 to which the on-screen prediction device 100 is applied.

[0094] As shown in Figure 10, the information processing system 1000 includes a processor 1001, a program memory 1002, a storage medium 1003 for storing video data, and a storage medium 1004 for storing bitstreams. The storage mediums 1003 and 1004 may be separate storage mediums or may be storage areas consisting of the same storage medium. Magnetic storage media such as hard disks can be used as storage media.

[0095] The information processing system 1000 installs a computer program that implements the functions of the in-screen prediction device 100 into the program memory 1002, thereby adaptively controlling the usage range (reference range) of the image (reconstructed image) used for in-screen prediction for each processing block. This makes it possible to generate a decoded image from a bitstream while reducing the line buffer size requirement.

[0096] (4) Interoperability Figure 11 shows a system in which the aforementioned video encoding device 800 and the aforementioned video decoding device 900 are connected by a transmission path 300, such as a wireless transmission path or a wired transmission path.

[0097] In a system like the one shown in Figure 11, the video encoding device 800 and the video decoding device 900 can ensure interoperability by adaptively controlling the range of the reference image used for in-screen prediction for each processing block using a common procedure, such as using common Kmax, Kup, Kleft, etc.

[0098] Specifically, the predetermined maximum value Kmax can be a fixed value common to both the video encoding device 800 and the video decoding device 900. Alternatively, the predetermined maximum value Kmax may be a variable value implicitly set based on the horizontal and vertical pixel counts of a picture, for example, by decreasing the value as the number of pixels in the picture increases.

[0099] As described above, when the predetermined maximum value Kmax is a variable value, the predetermined maximum value Kmax is not limited to being set implicitly. For example, the value may be explicitly signaled as a syntax element of the bitstream. That is, information specifying the predetermined maximum range Kmax may be included in the bitstream as a syntax element. The information specifying the predetermined maximum range Kmax may be included, for example, for each sequence, each picture, each slice, or each unit.

[0100] Furthermore, the candidate ranges Kup and Kleft may also be variable values. In this case, the candidate ranges Kup and Kleft may be explicitly signaled as syntax elements of the bitstream.

[0101] <3.5. Variant Example> In this embodiment, the usage range of the image used for in-screen prediction for each processing block was adaptively controlled in the upward direction of the screen. However, the embodiment is not limited to this, and the usage range in the leftward direction of the screen may also be adaptively controlled in the same manner.

[0102] Specifically, the region control processing unit 110 (control unit 115) of the in-screen prediction device 100 may control the portion of the image used for in-screen prediction that spans the left end of the unit in the left direction of the screen to be less than or equal to a predetermined maximum range. In this modified example, the predetermined maximum range is specifically the maximum range that can be used for in-screen prediction that spans the left end of the encoding tree unit in the left direction of the screen.

[0103] Furthermore, information for identifying the predetermined maximum range may be signaled from the video encoding device 800 to the video decoding device 900 as a syntax element of the bitstream.

[0104] <<4. Second Embodiment>> Next, a second embodiment of the present invention will be described with reference to Figures 12 and 13. While the first embodiment described above is a specific embodiment, the second embodiment is a more generalized embodiment.

[0105] <4.1. Structure> Figure 12 is a block diagram showing an example of a schematic configuration of the video encoding device 800 according to the second embodiment. Referring to Figure 12, the video encoding device 800 includes a control unit 810.

[0106] Figure 13 is a block diagram showing an example of a schematic configuration of the video decoding device 900 according to the second embodiment. Referring to Figure 13, the video decoding device 900 includes a control unit 910.

[0107] <4.2. Technical Features> Next, the technical features of the second embodiment will be described.

[0108] In the second embodiment, the video encoding device 800 (control unit 810) controls the portion of the image used for in-screen prediction that spans the edges of the unit in a predetermined direction, based on the relationship between the position of the candidate image used for in-screen prediction for the block to be processed and the position of the unit to which the block to be processed belongs, to be less than or equal to a predetermined maximum range.

[0109] For example, the video encoding device 800 may perform the operation of the in-screen prediction device 100 according to the first embodiment.

[0110] Furthermore, the video decoding device 900 (control unit 910) controls the portion of the image used for in-screen prediction that spans the end of the unit in a predetermined direction, based on the relationship between the position of the candidate image used for in-screen prediction for the block to be processed and the position of the unit to which the block to be processed belongs, to be less than or equal to a predetermined maximum range.

[0111] For example, the video decoding device 900 may perform the operation of the in-screen prediction device 100 according to the first embodiment.

[0112] The second embodiment has now been described. According to the second embodiment, for example, it becomes possible to adaptively control the range of images used for in-screen prediction.

[0113] <<5. Other Forms>> Although embodiments of the present invention have been described above, the present invention is not limited to these embodiments. It will be understood by those skilled in the art that these embodiments are merely illustrative and that various modifications are possible without departing from the scope and spirit of the present invention.

[0114] For example, the steps in the process described herein do not necessarily have to be executed chronologically in the order shown in the sequence diagram. For example, the steps in the process may be executed in a different order than that shown in the sequence diagram, or they may be executed in parallel. Also, some of the steps in the process may be deleted, or additional steps may be added to the process.

[0115] Furthermore, methods including processing of the components of the apparatus described herein (e.g., a first output unit, a second output unit, and / or a control unit) may be provided, and programs for causing a processor to perform the processing of the above components may be provided. A non-transitory computer-readable medium on which such a program is recorded may also be provided. Naturally, such apparatuses, modules, methods, programs, and computer-readable non-transitory mediums are also included in the present invention.

[0116] Some or all of the above embodiments may also be described as follows, but are not limited to the following.

[0117] (Note 1) A video encoding or decoding device comprising a control unit that controls the portion of the image used for in-screen prediction, which spans the edges of the unit in a predetermined direction, to be less than or equal to a predetermined maximum range, based on the relationship between the position of a candidate image used for in-screen prediction for a block to be processed and the position of the unit to which the block to be processed belongs.

[0118] (Note 2) The unit is a video encoding or decoding device as described in Appendix 1, which includes a syntax structure for encoding pixel samples.

[0119] (Note 3) The video encoding or decoding device according to Appendix 1 or 2, wherein the unit is one encoding tree unit included in a slice.

[0120] (Note 4) The video encoding or decoding device according to any one of the appendices 1 to 3, wherein the predetermined maximum range is the maximum range that can be used for in-screen prediction, spanning the end of the unit in the predetermined direction.

[0121] (Note 5) The system further includes a first derivation unit that, based on the position of the unit and the predetermined maximum range, derives a first boundary position that is usable for in-screen prediction, straddling the end of the unit in the predetermined direction, and is furthest from the unit in the predetermined direction. The video encoding or decoding device according to Appendix 4, wherein the control unit controls the portion range used for in-screen prediction, spanning the end of the unit in the predetermined direction, to be less than or equal to the predetermined range, based on the relationship between the position of the candidate image used for in-screen prediction and the first boundary position.

[0122] (Note 6) The system further includes a second derivation unit that derives a second boundary position, which is the position of a candidate image used for in-screen prediction and is furthest from the processing block in the predetermined direction, based on the position of the processing block and the candidate range of the candidate image used in the in-screen prediction in the predetermined direction. The video encoding or decoding device according to Appendix 5, wherein the control unit controls the portion range used for in-screen prediction, spanning the end of the unit in the predetermined direction, to be less than or equal to the predetermined maximum range, based on the relationship between the first boundary position and the second boundary position.

[0123] (Note 7) The video encoding or decoding device according to Appendix 6, wherein the control unit controls the portion range used for in-screen prediction, spanning the end of the unit in the predetermined direction, to be less than or equal to the predetermined maximum range, when the second boundary position is further away in the predetermined direction than the first boundary position with respect to the position of the block to be processed.

[0124] (Note 8) When the second boundary position is not farther than the first boundary position in the predetermined direction with respect to the position of the processing target block, the control unit controls the usage range of the image used for the in-picture prediction to the candidate range, the video encoding or video decoding apparatus according to appended note 6 or 7.

[0125] (Appended note 9) The predetermined direction is the upward direction on the screen, The predetermined maximum range is the maximum range Kmax that can be used for the in-picture prediction across the upper end of the unit in the upward direction on the screen, The candidate range is the candidate range Kup of the candidate image used in the upward direction on the screen in the in-picture prediction, Based on a vertical coordinate axis with the upper end of the picture as the origin and taking positive values in the downward direction on the screen, the first derivation unit derives the vertical coordinate ref_max_pos_y of the first boundary position using the following formula (1), Based on the vertical coordinate axis, the second derivation unit derives the vertical coordinate cand_min_pos_y of the second boundary position using the following formula (2), When cand_min_pos_y < ref_max_pos_y, the control unit calculates the usage range K of the image used in the upward direction on the screen in the in-picture prediction according to the following formula (3), and controls the partial range used for the in-picture prediction across the end of the unit in the predetermined direction to be not more than the predetermined range Kmax, the video encoding or video decoding apparatus according to any one of appended notes 6 to 8. ref_max_pos_y = cur_by - Kup ··· (1) cur_by is the vertical coordinate of the upper end of the unit. cand_min_pos_y = cur_cuy - Kmax ··· (2) cur_cuy is the vertical coordinate of the upper end of the unit. K = cur_by - cur_cuy + Kmax ··· (3)

[0126] (Appended note 10) When cand_min_pos_y ≧ ref_max_pos_y, the control unit calculates the usage range K of the image used in the in-picture prediction in the upward direction on the screen according to the following formula (4). The video encoding or video decoding device according to Supplementary Note 9. K = Kup ··· (4)

[0127] (Supplementary Note 11) The video encoding or video decoding device according to Supplementary Note 9 or 10, further comprising an image generation unit that generates an image used in the in-picture prediction based on the usage range K of the image used in the in-picture prediction in the upward direction on the screen.

[0128] (Supplementary Note 12) When K < Kup, the image generation unit generates an image used in the in-picture prediction by copying an image whose vertical coordinate position belongs to cur_by - Kmax to cur_by - 1 and mapping the copied image to a position corresponding to cur_by - Kup to cur_by - Kmax - 1. The video encoding or video decoding device according to Supplementary Note 11.

[0129] (Supplementary Note 13) Based on the relationship between the position of a candidate image used in in-picture prediction for a processing target block and the position of a unit to which the processing target block belongs, controlling a partial range used in the in-picture prediction that straddles an end of the unit in the usage range of the image used in the in-picture prediction to be below a predetermined maximum range. This includes a video encoding or video decoding method.

[0130] (Supplementary Note 14) A program for causing a computer to execute a video encoding or video decoding process including controlling a partial range used in the in-picture prediction that straddles an end of a unit in the usage range of the image used in the in-picture prediction to be below a predetermined maximum range, based on the relationship between the position of a candidate image used in in-picture prediction for a processing target block and the position of a unit to which the processing target block belongs.

[0131] (Supplementary Note 15) A computer-readable non-temporary recording medium that records a program for causing a computer to perform video encoding or video decoding processing, which includes controlling the portion of the range of use of the image used for in-screen prediction that spans the edges of the unit in a predetermined direction, to be less than or equal to a predetermined maximum range, based on the relationship between the position of a candidate image used for in-screen prediction for a block to be processed and the position of the unit to which the block to be processed belongs.

[0132] This application claims priority based on Japanese Patent Application No. 2018-120872, filed on 26 June 2018, and incorporates all of its disclosures herein. [Industrial applicability]

[0133] In systems that encode or decode video, it becomes possible to adaptively control the range of images used for in-screen prediction. [Explanation of Symbols]

[0134] 100 In-screen prediction device 110 Area Control Processing Unit 111 First Derivation Section 113 Second Derivation Section 115, 810, 910 Control Unit 120 Reference Image Generation Unit 130 Predictive Image Generation Unit 800 Video Encoding Device 900 Video Decoder

Claims

1. It includes a control unit that controls the range of images used for in-screen prediction for the block to be processed, The control unit, The usage range is controlled to be less than or equal to the first maximum range usable for in-screen prediction, spanning the ends of the coding tree unit upwards. If the usage range is not controlled to be less than or equal to the first maximum range, the usage range is controlled to be less than or equal to a second maximum range that is greater than the first maximum range. When the usage range is controlled to be less than or equal to the second maximum range, the usage range includes a plurality of lines, and at least one of the plurality of lines is used for in-screen prediction for the processing block. A video encoding device in which all of the aforementioned multiple lines are arranged in a continuous sequence.

2. This includes controlling the range of images used for in-screen prediction for the block being processed, The aforementioned control means The usage range is controlled to be less than or equal to the first maximum range usable for in-screen prediction, spanning the ends of the coding tree unit upwards, If the usage range is not controlled to be less than or equal to the first maximum range, the usage range is controlled to be less than or equal to a second maximum range that is greater than the first maximum range. Includes, When the usage range is controlled to be less than or equal to the second maximum range, the usage range includes a plurality of lines, and at least one of the plurality of lines is used for in-screen prediction for the processing block. All of the aforementioned lines are arranged in a continuous sequence. Video encoding methods.

3. It includes a control unit that controls the range of images used for in-screen prediction for the block to be processed, The control unit, The usage range is controlled to be less than or equal to the first maximum range usable for in-screen prediction, spanning the ends of the coding tree unit upwards. If the usage range is not controlled to be less than or equal to the first maximum range, the usage range is controlled to be less than or equal to a second maximum range that is greater than the first maximum range. When the usage range is controlled to be less than or equal to the second maximum range, the usage range includes a plurality of lines, and at least one of the plurality of lines is used for in-screen prediction for the processing block. A video decoding device in which all of the aforementioned multiple lines are arranged in a continuous sequence.

4. This includes controlling the range of images used for in-screen prediction for the block being processed, The aforementioned control means The usage range is controlled to be less than or equal to the first maximum range usable for in-screen prediction, spanning the ends of the coding tree unit upwards, If the usage range is not controlled to be less than or equal to the first maximum range, the usage range is controlled to be less than or equal to a second maximum range that is greater than the first maximum range. Includes, When the usage range is controlled to be less than or equal to the second maximum range, the usage range includes a plurality of lines, and at least one of the plurality of lines is used for in-screen prediction for the processing block. All of the aforementioned lines are arranged in a continuous sequence. How to decode video.

5. The control unit controls the usage range to be less than or equal to the first maximum range when the image used for in-screen prediction is located beyond the end of the coding tree unit in the upward direction. The video encoding device according to claim 1.

6. The control unit controls the usage range to be less than or equal to the first maximum range when the image used for in-screen prediction is located beyond the end of the coding tree unit in the upward direction. The video decoding device according to claim 3.

Images