Inter-prediction device, encoding device, decoding device, and program

Abstract

An inter prediction device (172, 242) that performs inter prediction by generating a motion vector for each sub-block obtained by dividing an inter-prediction target block comprises: an identifying unit (11) that identifies a first reference region on a reference image referred to by a first motion vector assigned to a decoded block around the target block; a determining unit (12) that determines a second reference region on the reference image determined in accordance with the relative positions of the target block and the decoded block; an acquiring unit (13) that acquires a motion vector group comprising motion vectors in units of sub-blocks in the second reference region; and a correcting unit (14) that corrects each motion vector included in the motion vector group on the basis of the first motion vector and a second motion vector assigned to the first reference region.

Description

Interpretation device, coding device, decoding device, and program Interaction of related applications

[0001] This application is based on Japanese application number 2024-182015, filed on October 17, 2024, and its contents are incorporated herein by reference.

[0002] This disclosure relates to an interpretation device, an encoding device, a decoding device, and a program.

[0003] In video coding schemes such as HEVC (High Efficiency Video Coding) and / or VVC (Versatile Video Coding), the coding device predicts coding blocks (CU: Coding Unit) obtained by dividing the original image (original frame) into blocks, generates predicted blocks, and transmits the predicted residual, which is the difference between the coding blocks of the original image and the predicted blocks, after transformation, quantization, and entropy coding. Prediction methods include intra-prediction, which utilizes spatial correlation within a frame, and inter-prediction, which utilizes correlation between frames.

[0004] In VVC, one of the methods for predicting motion vectors used in interpretation is called Subblock-based Temporal Motion Vector Prediction (SbTMVP). SbTMVP is a method that derives motion vectors at the subblock level by utilizing the correlation of motion vectors in the time direction. It copies each motion vector within the subblocks obtained by dividing the target block for interpretation from the corresponding position in the reference image (reference picture).

[0005] In SbTMVP, 1) a first reference region on the reference image is identified, which is referenced by motion vectors assigned to adjacent decoded blocks adjacent to the target block; 2) a second reference region is determined, which is a region on the reference image determined according to the position of the first reference region and is the same size as the target block; and 3) a group of motion vectors consisting of motion vectors at the sub-block level in the second reference region is obtained (copied) and assigned as the motion vectors for each sub-block of the target block.

[0006] VVC Specification "Recommendation ITU-T H.266"

[0007] SbTMVP is a method that utilizes the temporal correlation of motion vectors, but because it simply copies the set of motion vectors assigned to the second reference region (motion vector reference region) and assigns them to each subblock of the target block, discontinuities may occur between the set of motion vectors assigned to the target block and the motion vectors assigned to adjacent blocks. In other words, the accuracy of the set of motion vectors assigned to the target block may be low, which may lead to a decrease in the prediction accuracy of interpretation and a decrease in coding efficiency.

[0008] Therefore, the object of this disclosure is to provide an interpreter, an encoder, a decoder, and a program that can improve the coding efficiency when performing interpreteration using the temporal correlation of motion vectors.

[0009] An interpretation device according to the first embodiment is an interpretation device that performs interpretation by generating motion vectors for each subblock obtained by dividing the target block for interpretation, and comprises: an identification unit that identifies a first reference region on a reference image referenced by a first motion vector assigned to a decoded block surrounding the target block; a determination unit that determines a second reference region on the reference image determined according to the relative position between the target block and the decoded block; an acquisition unit that acquires a group of motion vectors consisting of motion vectors in subblock units in the second reference region; and a correction unit that corrects each motion vector included in the group of motion vectors based on the first motion vector and the second motion vector assigned to the first reference region.

[0010] The encoding device according to the second embodiment includes an interpretation device according to the first embodiment.

[0011] The decoding device according to the third embodiment includes the interpretation device according to the first embodiment.

[0012] The program according to the fourth embodiment causes the computer to function as an interpretation device according to the first embodiment.

[0013] This figure shows the configuration of the encoding device according to the embodiment. This figure shows the configuration of the motion vector prediction unit of the interpretation unit on the encoding side according to the embodiment. This figure illustrates the operation of a general SbTMVP. This figure illustrates the operation of a general SbTMVP. This figure illustrates the operation of the motion vector prediction unit according to the embodiment. This figure shows the configuration of the decoding device according to the embodiment. This figure shows the configuration of the motion vector prediction unit of the interpretation unit on the decoding side according to the embodiment. This figure shows the operation flow of motion vector prediction according to the embodiment. This figure illustrates an example of a modification of the operation according to the embodiment. This figure illustrates an example of a modification of the operation according to the embodiment.

[0014] An encoding and decoding device equipped with an interpretation device according to an embodiment will be described with reference to the drawings. The encoding and decoding devices respectively perform encoding and decoding of video (i.e., moving images) represented by MPEG. In the following drawings, identical or similar parts are denoted by the same or similar reference numerals.

[0015] (1) Encoding device The encoding device according to this embodiment will be described with reference to Figures 1 to 5.

[0016] (1.1) Configuration of the Encoding Device First, the configuration of the encoding device 1 according to this embodiment will be described using Figure 1. The encoding device 1 is a device that encodes an input image to generate a bitstream and outputs a bitstream. The encoding device 1 has a block division unit 100, a subtraction unit 110, a transformation / quantization unit 120, an entropy encoding unit 130, an inverse quantization / inverse transformation unit 140, a synthesis unit 150, a memory 160, and a prediction unit 170.

[0017] The block division unit 100 divides the original image, which is an input image in units of frames (also called "pictures") that constitute a moving image, into a plurality of image blocks, and outputs the image blocks obtained by the division to the subtraction unit 110. The size of the image block is, for example, 32 x 32 pixels, 16 x 16 pixels, 8 x 8 pixels, or 4 x 4 pixels. The shape of the image block is not limited to a square; it may also be rectangular (non-square). An image block is a unit that the encoding device 1 performs encoding on, and a unit that the decoding device performs decoding on. Such an image block is also called an encoded block (CU). The image block that the encoding device 1 performs encoding on is also called an "encoded target block," and the image block that the decoding device performs decoding on is also called a "decoded target block." When these are not particularly distinguished, they are simply called "target block" or "block." A "target block" may also be called a "current block."

[0018] Each pixel in the input image is composed of a luminance component (Y) and chrominance components (Cb, Cr). The block division unit 100 outputs luminance blocks by performing block division on the luminance component and chrominance blocks by performing block division on the chrominance component. The shape of the block division may be the same for the luminance component and the chrominance component. The shape of the division may be independently controlled for the luminance component and the chrominance component.

[0019] The subtraction unit 110 calculates a predicted residual, which represents the difference (error) between the block to be encoded output by the block division unit 100 and the predicted block obtained by the prediction unit 170 predicting the block to be encoded. Specifically, the subtraction unit 110 calculates the predicted residual by subtracting each pixel value of the predicted block from each pixel value of the block to be encoded, and outputs the calculated predicted residual to the conversion / quantization unit 120.

[0020] The conversion and quantization unit 120 performs conversion and quantization processing on a block-by-block basis. The conversion and quantization unit 120 comprises a conversion unit 121 and a quantization unit 122.

[0021] The transformation unit 121 performs a transformation process on the predicted residual output by the subtraction unit 110 to calculate transformation coefficients, and outputs the calculated transformation coefficients to the quantization unit 122. Transformations include, for example, discrete cosine transform (DCT) and / or discrete sine transform (DST), Karhunen-Loeve transform (KLT), etc. Transformation processing includes transformation skipping, where no transformation processing is performed. Transformation skipping also includes transformations where only horizontal transformation processing is applied and / or transformations where only vertical transformation processing is applied. Furthermore, the transformation unit 121 may perform a secondary transformation process, applying further transformation processing to the transformation coefficients obtained through the transformation process. The secondary transformation process may be applied only to a portion of the transformation coefficients.

[0022] The quantization unit 122 quantizes the conversion coefficients output by the conversion unit 121 using quantization parameters and a quantization matrix, and outputs the quantized conversion coefficients to the entropy coding unit 130 and the inverse quantization / inverse conversion unit 140. The quantization parameters are parameters that are applied in common to each conversion coefficient within a block and determine the coarseness of the quantization. The quantization matrix is ​​a matrix whose elements are the quantization values ​​when each conversion coefficient is quantized, and is also called a ScaleList. The ScaleList information may be included in the bitstream by APS (Adaptation Parameter Set) and / or other means. This information may be predefined as a quantization matrix with different values ​​(default matrix). Selection control and / or application control of the quantization matrix may be performed according to the block size and / or prediction mode, conversion processing, etc.

[0023] The entropy coding unit 130 performs entropy coding on the quantization conversion coefficients output by the quantization unit 122, compresses the data to generate a bitstream, and outputs the bitstream to the outside of the coding device 1. For entropy coding, Huffman coding and / or CABAC (Context-based Adaptive Binary Arithmetic Coding) can be used. The entropy coding unit 130 also receives information related to predictions (flags and / or syntax such as indices) from the prediction unit 170, and performs coding of the input information and outputting it as a bitstream.

[0024] The inverse quantization / inverse transformation unit 140 performs inverse quantization and inverse transformation processing on a unit-by-unit basis. The inverse quantization / inverse transformation unit 140 comprises an inverse quantization unit 141 and an inverse transformation unit 142.

[0025] The inverse quantization unit 141 performs inverse quantization processing corresponding to the quantization processing performed by the quantization unit 122. Specifically, the inverse quantization unit 141 reconstructs the conversion coefficients by inverse quantization of the quantization conversion coefficients output by the quantization unit 122 using quantization parameters and a quantization matrix (ScalingList), and outputs the reconstructed conversion coefficients to the inverse conversion unit 142. As described above, the information of the ScalingList may be included in the bitstream by APS (Adaptation Parameter Set) and / or other means. This information may pre-define quantization matrices with different values ​​(default matrices). Selection control and / or application control of the quantization matrix may be performed according to the block size and / or prediction mode, conversion processing, etc.

[0026] The inverse transform unit 142 performs an inverse transform process corresponding to the transform process performed by the transform unit 121. For example, if the transform unit 121 performs a discrete cosine transform, the inverse transform unit 142 performs an inverse discrete cosine transform. The inverse transform unit 142 performs an inverse transform process on the transform coefficients output by the inverse quantization unit 141 to reconstruct the predicted residuals, and outputs the reconstructed predicted residuals to the synthesis unit 150.

[0027] The synthesis unit 150 synthesizes the reconstructed predicted residuals output by the inverse transform unit 142 by adding them pixel by pixel to the predicted blocks output by the prediction unit 170. The synthesis unit 150 decodes (reconstructs) the blocks by adding each pixel value of the reconstructed predicted residuals to each pixel value of the predicted blocks, and outputs the reconstructed blocks to the memory 160. Hereinafter, the reconstructed blocks will also be referred to as decoded blocks.

[0028] The memory 160 stores the reconstruction blocks output by the synthesis unit 150 and accumulates the reconstruction blocks as decoded images on a frame-by-frame basis. The memory 160 outputs the stored reconstruction blocks or decoded images to the prediction unit 170. A loop filter may be provided between the synthesis unit 150 and the memory 160. The loop filter includes luminance mapping processing for the reconstruction blocks and / or scaling processing for the residual color difference signals, similar to LMCS (Luma mapping and chroma residual scaling) introduced in VVC.

[0029] The prediction unit 170 performs predictions on a unit-by-unit basis for encoding. The prediction unit 170 includes an intra-prediction unit 171, an inter-prediction unit 172, and a switching unit 173. In this embodiment, the inter-prediction unit 172 constitutes an inter-prediction device.

[0030] The intra-prediction unit 171 generates an intra-prediction block by referring to the decoded pixels surrounding the block in the decoded image stored in the memory 160, and outputs the generated intra-prediction block to the switching unit 173. Generally, the intra-prediction unit 171 selects an intra-prediction mode from among several intra-prediction modes to apply to the block to be coded for intra-prediction, and uses the selected intra-prediction mode to predict the block to be coded for intra-prediction. The intra-prediction unit 171 outputs information about the selected intra-prediction mode to the entropy coding unit 130.

[0031] The interpretation unit 172 uses the decoded image stored in the memory 160 as a reference image to calculate motion vectors using methods such as block matching, predicts the blocks to be encoded using the motion vectors to generate interpretation blocks, and outputs the generated interpretation blocks to the switching unit 173. Here, the interpretation unit 172 selects the optimal interpretation method from interpretation using multiple reference images (typically biprediction) and / or interpretation using one reference image (unidirectional prediction), and performs interpretation using the selected interpretation method. The interpretation unit 172 outputs information related to interpretation (motion vectors, etc.) to the entropy coding unit 130.

[0032] The interpretation unit 172 according to this embodiment includes a motion vector prediction unit 10a that performs subblock-based temporal motion vector prediction, specifically an improved SbTMVP. Details of the motion vector prediction unit 10a will be described later.

[0033] The switching unit 173 switches between the intra-prediction block output by the intra-prediction unit 171 and the inter-prediction block output by the inter-prediction unit 172, and outputs one of the prediction blocks to the subtraction unit 110 and the synthesis unit 150.

[0034] (1.2) Configuration of the Motion Vector Prediction Unit Next, the configuration of the motion vector prediction unit 10a of the interpretation unit 172 according to this embodiment will be described with reference to Figure 2. The motion vector prediction unit 10a includes a first reference region identification unit 11a, a second reference region determination unit 12a, a motion vector group acquisition unit 13a, a motion vector correction unit 14a, and a motion vector assignment unit 17a.

[0035] First, an overview of the motion vector prediction unit 10a according to this embodiment will be described. The motion vector prediction unit 10a according to this embodiment realizes an improved SbTMVP.

[0036] The interpretation unit 172 includes a motion vector prediction unit 10a, which generates motion vectors for each subblock obtained by dividing the target block for interpretation and performs interpretation. In the motion vector prediction unit 10a according to this embodiment, the first reference region identification unit 11a identifies a first reference region on the reference image that is referenced by a first motion vector assigned to a decoded block (e.g., an adjacent block) surrounding the target block. The second reference region determination unit 12a determines a second reference region that is an area on the reference image determined according to the relative position between the target block and the decoded block, and is the same size as the target block. The motion vector group acquisition unit 13a acquires a group of motion vectors consisting of motion vectors in subblock units in the second reference region. The motion vector correction unit 14a corrects each motion vector included in the motion vector group based on the first motion vector and the second motion vector assigned to the first reference region. However, the second reference region determination unit 12a may increase or decrease the size of the motion vector reference region (second reference region) considering that scaling is performed between the reference image and the original image. For example, if the prediction mode assigned to the decoded block and / or the first reference region is a prediction mode that uses reduction and / or enlargement processing in predicting the original image using the reference image, the second reference region determination unit 12a may determine the size and / or position of the second reference region taking that prediction mode and / or enlargement / reduction processing into consideration.

[0037] As described above, the motion vector prediction unit 10a in this embodiment is configured not to simply copy the motion vector group assigned to the second reference region (motion vector reference region) and assign it to each subblock of the target block, but to correct the motion vector group using the motion vector correction unit 14a. The motion vector correction unit 14a corrects each motion vector included in the motion vector group based on the first motion vector assigned to the decoded blocks surrounding the target block and the second motion vector assigned to the first reference region on the reference image referenced by the first motion vector. In this embodiment, the motion vector correction unit 14a performs the correction process based on the second motion vector assigned to the first reference region, but the correction process may also be performed based on other prediction information of the first reference region. For example, it may depend on the prediction mode and / or conversion process assigned to the first reference region. Specifically, the motion vector correction unit 14a may control the magnitude of the influence of the vector correction (correction intensity control) when the prediction mode and / or conversion process assigned to the first reference region is of a predetermined specific type. The motion vector correction unit 14a may also apply control to determine whether or not to perform the vector correction process. Furthermore, the above-mentioned correction intensity control and / or application control may be performed based on the block size of the first reference region and / or the target block.

[0038] This suppresses discontinuities between the motion vectors assigned to the target block and the motion vectors assigned to the decoded blocks surrounding the target block. As a result, the accuracy of the motion vectors assigned to the target block is improved, which in turn improves the prediction accuracy of interpretation and thus improves coding efficiency. In this embodiment, motion vector correction for a target block divided into subblocks has been described, but this disclosure is also applicable when the target block is not divided into subblocks. That is, it is similarly applicable even if the subblocks and the target block are the same size.

[0039] In this embodiment, the motion vector correction unit 14a corrects each motion vector in the motion vector group using a correction vector calculated according to the first motion vector and the second motion vector. Specifically, the motion vector correction unit 14a includes a correction vector calculation unit 15a that calculates the difference between the first motion vector and the second motion vector as the correction vector, and a correction vector addition unit 16a that corrects each motion vector in the motion vector group by adding the correction vector. This allows each motion vector in the motion vector group to be corrected appropriately.

[0040] The motion vector assignment unit 17a then assigns the corrected motion vector group as the motion vector for each subblock of the target block. This makes it possible to appropriately assign the motion vector for each subblock of the target block.

[0041] Secondly, specific examples of the operation of the motion vector prediction unit 10a according to this embodiment will be described using Figures 2 to 5. Figures 3 and 4 are diagrams for explaining the operation of a general SbTMVP, and Figure 5 is a diagram for explaining the operation of the motion vector prediction unit 10a according to this embodiment.

[0042] As shown in Figure 2, the memory 160 shown in Figure 1 includes a reference image storage unit 161 for storing a reference image and a motion vector storage unit 162 for storing motion vectors assigned to each block within the reference image. The reference image is also referred to as a reference picture.

[0043] In SbTMVP, when predicting the image to be encoded (i.e., the image to be interpreted) by interpretation, the motion vector used in a previously decoded reference image is referenced as the temporally predicted motion vector, and the image to be encoded is divided into 8x8 pixel subblocks, and a motion vector is predicted for each subblock. In this embodiment, the size of the subblock is not limited to 8x8 pixels, but may be other sizes.

[0044] In the typical SbTMVP shown in Figure 3, the first reference region identification unit 11a identifies the first motion vector V assigned to the adjacent block A of the target block in the image to be encoded shown in Figure 3(a).０ｙ Identify the first reference region (also referred to as the "adjacent block reference region") A' on the reference image shown in FIG. 3(b), which is referred to by . Here, the adjacent block A shown in FIG. 3(a) is an example of a decoded block around the target block. In the illustrated example, the target block is divided into a total of four sub-blocks, namely sub-block Sb ０，０ , sub-block Sb １，０ , sub-block Sb ０，１ , and sub-block Sb １，１ .

[0045] Next, as shown in FIG. 3(b), the second reference region determination unit 12a determines a region on the reference image that is determined according to the relative position between the target block and the adjacent block A, and determines a second reference region (also referred to as the "motion vector reference region") having the same size as the target block. In the illustrated example, since the adjacent block A is a decoded block adjacent to the left of the target block, a region having the same size as the target block and adjacent to the right of the adjacent block reference region A' is determined as the motion vector reference region.

[0046] For example, if the upper left pixel position of the target block is (x ０ , x １ ), the upper left pixel position of the decoded block (in the illustrated example, the adjacent block A) is (x １ , y １ ), and the motion vector assigned to the decoded block is (V ０ｘ , V ０ｙ ), then the upper left pixel position of the adjacent block reference region is (x １ + V ０ｘ , y １ + V ０ｙ ). If the above relative position relationship does not vary temporally, the upper left pixel position of the motion vector reference region is (x ０ + V ０ｘ , y ０ + V ０ｙ ). As in the modification example described later, if the relative position relationship varies temporally, it is (x ０ + k[x １ - x ０ + V ０ｘ , y ０ + k[y １-y ０ ] + V ０ｙ Alternatively, the position of the motion vector reference region may be determined using a function k that takes the distance between the target block and the decoded block as an argument, as shown above.

[0047] Next, the motion vector group acquisition unit 13a acquires a group of motion vectors consisting of motion vectors in the sub-block units within the motion vector reference region shown in Figure 3(b). In the illustrated example, the motion vector group acquisition unit 13a acquires the motion vectors for each of the four sub-blocks when the motion vector reference region is divided into four sub-blocks.

[0048] Finally, in a typical SbTMVP, the motion vector correction unit 14a does not exist, and the motion vector assignment unit 17a assigns (copies) the motion vector group acquired by the motion vector group acquisition unit 13a as the motion vector for each subblock of the target block.

[0049] Figure 4 is a diagram illustrating the problems with typical SbTMVP.

[0050] The first motion vector V assigned to the adjacent block A shown in Figure 4(a) ０ And the second motion vector V assigned to the adjacent block reference region A' shown in Figure 4(b) １ This is significantly different. Specifically, the first motion vector V ０ The first vector refers to the upper left direction, but the second motion vector V １ This refers to the direction to the lower left.

[0051] Here, as shown in Figure 4(b), the adjacent block reference region A' is adjacent to the motion vector reference region, and there is a high probability that there is a high correlation between the motion of the adjacent block reference region A' and the motion of the motion vector reference region. In the illustrated example, the motion vector at the subblock level of the motion vector reference region is the second motion vector V of the adjacent block reference region A'. １ Similarly, both refer to the lower left direction.

[0052] In a typical SbTMVP, the motion vectors at the subblock level of such a motion vector reference region are simply copied as the motion vectors at the subblock level of the target block shown in Figure 4(a). Therefore, as shown in Figure 4(a), the first motion vector V assigned to the adjacent block A ０ The motion vectors assigned to each subblock of the target block (i.e., the motion vectors copied from the motion vector reference region) are significantly different, resulting in a discontinuity in motion vectors between the adjacent block A and the target block.

[0053] Here, even though there is a high probability of a strong correlation between the movement of adjacent block A and the movement of the target block, the discontinuity in the movement vectors between adjacent block A and the target block suggests that the movement vectors assigned to each subblock of the target block may be inappropriate. Therefore, there is a problem in that the prediction accuracy of the inter-prediction for the target block may be low.

[0054] Therefore, in this embodiment, the motion vector correction unit 14a corrects the first motion vector V as shown in Figure 5(c). ０ and the second motion vector V １ The correction vector V calculated accordingly ｍ Each motion vector V of the motion vector group acquired by the motion vector group acquisition unit 13a is used ｘ，ｙ Correct it.

[0055] Specifically, the correction vector calculation unit 15a of the motion vector correction unit 14a calculates the first motion vector V ０ and the second motion vector V １ The difference (V) ０ -V １ ) Correction vector V ｍ It is calculated as follows. Then, the correction vector addition unit 16a of the motion vector correction unit 14a adds each motion vector V of the motion vector group acquired by the motion vector group acquisition unit 13a. ｘ，ｙ Correction vector V ｍ The correction is applied by adding the correction to each motion vector V'. ｘ，ｙ is, V ｘ，ｙ +V ｍ It will become.

[0056] For example, the first motion vector V assigned to the adjacent block A. ０ = (V ０ｘ , V ０ｙ ), the second motion vector V assigned to the adjacent block reference region １ = (V １ｘ , V １ｙ If this is the case, the correction vector V ｍ is, V ｍ = (V ０ｘ -V １ｘ , V ０ｙ -V １ｙ It can be calculated by ( ).

[0057] Then, the motion vector of the (i,j)th subblock in the motion vector reference region is V ｉ，ｊ In this case, the motion vector V' to be assigned to the (i,j)th subblock of the target block. ｉ，ｊ is V' ｉ，ｊ = V ｉ，ｊ +V ｍ It is calculated by the first motion vector V. However, the motion vector correction unit 14a is the first motion vector V ０ and the second motion vector V １ Considering the relationship, the motion vector V of the target block ｘ，ｙ It is sufficient to correct for the first motion vector V, and it is not necessarily required to correct for it. ０ and the second motion vector V １ The difference between each motion vector V ｘ，ｙ There is no need to correct it. For example, the motion vector correction unit 14a corrects the first motion vector V ０ and the second motion vector V １ Each motion vector V depends on the ratio with ｘ，ｙ Correction may be performed. The motion vector correction unit 14a corrects the first motion vector V ０ and the second motion vector V １ Each motion vector V corresponds to the average of the others. ｘ，ｙ You may correct this.

[0058] (2) Decoding device Next, the decoding device 2 according to this embodiment will be described with reference to Figure 6. The decoding device 2 is a device that derives and outputs a decoded image from the input bitstream.

[0059] The decoding device 2 includes an entropy decoding unit 200, an inverse quantization / inverse transformation unit 210, a synthesis unit 220, a memory 230, and a prediction unit 240.

[0060] The entropy decoding unit 200 decodes the bitstream generated by the encoding device 1 and outputs the quantized conversion coefficients to the inverse quantization / inverse conversion unit 210. The entropy decoding unit 200 also acquires information regarding predictions (intra-prediction and inter-prediction) and outputs the acquired information to the prediction unit 240.

[0061] In this embodiment, the entropy decoding unit 200 may acquire a flag indicating that the improved SbTMVP is to be applied and output the flag to the prediction unit 240.

[0062] The inverse quantization / inverse transformation unit 210 performs inverse quantization and inverse transformation processing on a unit-by-unit basis. The inverse quantization / inverse transformation unit 210 comprises an inverse quantization unit 211 and an inverse transformation unit 212.

[0063] The inverse quantization unit 211 performs inverse quantization processing corresponding to the quantization processing performed by the quantization unit 122 of the encoding device 1. The inverse quantization unit 211 reconstructs the conversion coefficients of the decoded block by inverse quantization of the quantization conversion coefficients output by the entropy decoding unit 200 using quantization parameters and a quantization matrix (ScalingList), and outputs the reconstructed conversion coefficients to the inverse conversion unit 212. As described above, the information in the ScalingList may be included in the bitstream by APS (Adaptation Parameter Set) and / or other means. This information may be predefined as a quantization matrix with different values ​​(default matrix). Selection control and / or application control of the quantization matrix may be performed according to the block size and / or prediction mode, conversion processing, etc.

[0064] The inverse transform unit 212 performs an inverse transform process corresponding to the transform process performed by the transform unit 121 of the encoding device 1. The inverse transform unit 212 performs an inverse transform process on the transform coefficients output by the inverse quantization unit 211 to reconstruct the predicted residuals, and outputs the reconstructed predicted residuals, which are the reconstructed predicted residuals, to the synthesis unit 220. The inverse transform process includes a transform skip, in which no inverse transform process is performed. The inverse transform unit 212 may also perform an inverse quadratic transform process, which applies an inverse transform process to the components obtained by the inverse transform process.

[0065] The synthesis unit 220 synthesizes the predicted residual output by the inverse transform unit 212 and the predicted block output by the prediction unit 240 on a pixel-by-pixel basis, decodes (reconstructs) the original block, and outputs the reconstructed block to the memory 230.

[0066] The memory 230 stores the reconstruction blocks output by the synthesis unit 220 and accumulates the reconstruction blocks as decoded images on a frame-by-frame basis. The memory 230 outputs the reconstruction blocks or decoded images to the prediction unit 240. The memory 230 also outputs the decoded images on a frame-by-frame basis to the outside of the decoding device 2. A loop filter may be provided between the synthesis unit 220 and the memory 230.

[0067] The prediction unit 240 performs predictions on a per-block basis for decoding. The prediction unit 240 includes an intra-prediction unit 241, an inter-prediction unit 242, and a switching unit 243. In this embodiment, the inter-prediction unit 242 constitutes an inter-prediction device.

[0068] The intra-prediction unit 241 generates an intra-prediction block by referring to the decoded pixels surrounding the block to be predicted (encoded block) from the decoded image stored in the memory 230, and outputs the generated intra-prediction block to the switching unit 243.

[0069] The interpretation unit 242 uses the decoded image stored in the memory 230 as a reference image to predict the encoded block by interpretation. The interpretation unit 242 generates interpretation blocks by performing interpretation according to motion vector information output by the entropy decoding unit 200, and outputs the generated interpretation blocks to the switching unit 243. The interpretation unit 242 in this embodiment has a motion vector prediction unit 10b that performs subblock-based temporal motion vector prediction, specifically an improved SbTMVP.

[0070] The switching unit 243 switches between the intra-prediction block output by the intra-prediction unit 241 and the inter-prediction block output by the inter-prediction unit 242, and outputs one of the prediction blocks to the synthesis unit 220.

[0071] Figure 7 shows the configuration of the motion vector prediction unit 10b of the interpretation unit 242 according to this embodiment.

[0072] The motion vector prediction unit 10b includes a first reference region identification unit 11b, a second reference region determination unit 12b, a motion vector group acquisition unit 13b, a motion vector correction unit 14b, and a motion vector assignment unit 17b. The motion vector correction unit 14b includes a correction vector calculation unit 15b and a correction vector addition unit 16b.

[0073] The first reference region identification unit 11b, the second reference region determination unit 12b, the motion vector group acquisition unit 13b, the motion vector correction unit 14b (correction vector calculation unit 15b, correction vector addition unit 16b), and the motion vector assignment unit 17b perform the same processing as the first reference region identification unit 11a, the second reference region determination unit 12a, the motion vector group acquisition unit 13a, the motion vector correction unit 14a (correction vector calculation unit 15a, correction vector addition unit 16a), and the motion vector assignment unit 17a of the encoding-side motion vector prediction unit 10a.

[0074] (3) Operation Flow of Motion Vector Prediction Next, the operation flow of motion vector prediction according to this embodiment will be described using Figures 5 and 8.

[0075] Here, the operation of the motion vector prediction unit 10b on the decoding side will be described by way of example. Therefore, the encoding target image in FIG. 5 is read as the decoding target image. Note that the motion vector prediction unit 10a on the encoding side also performs the same operation as the motion vector prediction unit 10b on the decoding side.

[0076] The motion vector prediction unit 10b on the decoding side may perform this operation flow when a flag indicating that the improved SbTMVP is to be applied to the decoding target block is signaled from the encoding side. The improved SbTMVP may be applicable to the images of P pictures and B pictures.

[0077] As shown in FIG. 8, in step S1, the first reference region specifying unit 11b specifies the adjacent block reference region (first reference region) A' on the reference image shown in FIG. 5(b), which is referred to by the first motion vector V assigned to the adjacent block A of the decoding target block in the decoding target image shown in FIG. 5(a). For example, if the upper left pixel position of the decoding target block is (x ０ , y ０ , the upper left pixel position of the adjacent block A is (x １ , y １ , and the motion vector assigned to the adjacent block A is (V １ , V ０ｘ , V ０ｙ ), then the upper left pixel position of the adjacent block reference region A' is (x １ + V ０ｘ , y １ + V ０ｙ ).

[0078] In step S2, as shown in FIG. 5(b), the second reference region determination unit 12b determines a motion vector reference region (second reference region) on the reference image that is determined according to the relative position between the decoding target block and the adjacent block A and has the same size as the decoding target block. The upper left pixel position of the motion vector reference region is (x ０ + V ０ｘ , y ０ + V ０ｙ). However, the second reference region determination unit 12b may increase or decrease the size of the motion vector reference region (second reference region) in consideration of the fact that enlargement / reduction is performed between the reference image and the original image. For example, when the prediction mode assigned to the decoded block and / or the first reference region is a prediction mode that uses reduction processing or enlargement processing in the prediction of the original image using the reference image, the second reference region determination unit 12b may determine the size and / or position of the second reference region in consideration of the prediction mode and / or the enlargement / reduction processing.

[0079] In step S3, the motion vector group acquisition unit 13b acquires a group of motion vectors composed of motion vectors in sub-block units in the motion vector reference region. In the example of FIG. 5(b), the motion vector group acquisition unit 13b acquires the motion vectors V ｘ，ｙ (V ０，０ , V １，０ , V ０，１ , V １，１ ) when the motion vector reference region is divided into four sub-blocks.

[0080] In step S4, the motion vector correction unit 14b corrects each motion vector V ０ of the motion vector group V １ shown in FIG. 5(a) and the second motion vector V ｘ，ｙ assigned to the adjacent block reference region A' shown in FIG. 5(b) based on the first motion vector V ０，０ , V １，０ , V ０，１ , V １，１ . For example, the correction vector calculation unit 15b calculates the difference (V ０ - V １ ) between the first motion vector V ０ and the second motion vector V １ as the correction vector V ｍ . Then, the correction vector addition unit 16b adds and corrects the correction vector V ｍ to each motion vector V ｘ，ｙ acquired by the motion vector group acquisition unit 13b. That is, each corrected motion vector V' ｘ，ｙ is V ｘ，ｙ + V ｍIn step 2, if the second reference region determination unit 12b determines a second reference region of a different size from the target block, the motion vector correction unit 14b may control the correction process according to the size ratio between the target block and the second reference region. Specifically, the motion vector correction unit 14b may correct the magnitude and / or position of the motion vector assigned to the second reference region according to the size ratio.

[0081] For example, the first motion vector V ０ = (V ０ｘ , V ０ｙ ), second motion vector V １ = (V １ｘ , V １ｙ If this is the case, the correction vector V ｍ is, V ｍ = (V ０ｘ -V １ｘ , V ０ｙ -V １ｙ It can be calculated by ). Then, the motion vector of the (i,j)th subblock in the motion vector reference region is V ｉ，ｊ In this case, the corrected motion vector V' is assigned to the (i,j)th subblock of the block to be decoded. ｉ，ｊ is V' ｉ，ｊ = V ｉ，ｊ +V ｍ It is calculated by [this method].

[0082] In step S5, the motion vector assignment unit 17b assigns the corrected motion vector group V' ｘ，ｙ This is assigned as the motion vector for each subblock of the block to be decoded.

[0083] Thus, according to this embodiment, the correction vector calculation unit 15b calculates the motion vector group V ｘ，ｙ By correcting this, the corrected motion vector V' assigned to each subblock of the block to be decoded is obtained. ｘ，ｙ However, the first motion vector V assigned to the adjacent block A ０ It is corrected to approximate the value.

[0084] Therefore, discontinuities between adjacent block A and the block to be decoded can be suppressed. Since there is considered to be a high correlation between the movement of adjacent block A and the movement of the block to be decoded, the correction vector calculation unit 15b calculates the movement vector group V ｘ，ｙ By correcting this, the motion vectors assigned to each subblock of the decryption target block become more appropriate. Therefore, the prediction accuracy of interpretation for the decryption target block is improved, and as a result, coding efficiency is improved.

[0085] (4) Examples of changes in operation Figures 9 and 10 are diagrams illustrating examples of changes in operation according to the above-described embodiment.

[0086] As shown in Figure 9(a), in this modified example, instead of the adjacent block A of the target block, a non-adjacent block B that is not adjacent to the target block is used as the decoded block for determining the motion vector reference region. Specifically, as shown in Figure 9(b), the first reference region identification unit 11 (11a, 11b) determines the first motion vector V assigned to the non-adjacent block B. ０ The region referenced by is identified as the non-adjacent block reference region (first reference region) B'. Alternatively, the second reference region determination unit 12 (12a, 12b) may determine the motion vector reference region (second reference region) based on the position of the non-adjacent block reference region B', according to the relative position of the non-adjacent block B and the target block.

[0087] The second reference region determination unit 12 (12a, 12b) can determine the motion vector reference region corresponding to the position of the target block by making the distance N between the non-adjacent block B and the target block equal to the distance N between the non-adjacent block reference region and the motion vector reference region.

[0088] However, the distance N between the non-adjacent block B and the target block and the distance N between the non-adjacent block reference region and the motion vector reference region do not necessarily have to be equal. For example, the second reference region determination unit 12 (12a, 12b) determines the first motion vector V assigned to the non-adjacent block B. ０ The size of and the second motion vector V assigned to the non-adjacent block reference region B'. １The position of the motion vector reference region may be determined according to its size.

[0089] Similarly, if the prediction mode for non-adjacent block B, non-adjacent block reference region B', and / or motion vector reference region is a prediction process that includes rotation and / or scaling, then the relative positional relationship between non-adjacent block B and the target block is likely to differ from the relative positional relationship between the non-adjacent block reference region and the motion vector reference region. Therefore, the second reference region determination unit 12 (12a, 12b) may determine the position of the motion vector reference region based on any of the prediction modes for non-adjacent block B, non-adjacent block reference region, and motion vector reference region.

[0090] Using Figure 10, a method for determining the position of the motion vector reference region determined by the second reference region determination unit 12 (12a, 12b) according to the prediction process of the first reference region (in the illustrated example, the adjacent block reference region A') will be explained. Assume that the prediction process of the adjacent block reference region A' shown in Figure 10(b), which is referenced by the adjacent block A shown in Figure 10(a), is a prediction process that generates a predicted image by rotation and / or scaling, as in the Affine prediction adopted in VVC. For example, as shown in Figures 10(b) and 10(c), if the prediction process assigned to the adjacent block reference region A' is rotation prediction, the second reference region determination unit 12 (12a, 12b) may determine the position of the subblock units of the motion vector reference region according to the amount of rotation (or whether or not rotation occurs) of the prediction process.

[0091] (5) Other Embodiments In the embodiments described above (Figures 2 to 5), the case of unidirectional prediction using one reference image for interpretation was given as an example, but the invention is not limited to unidirectional prediction. The operation according to the embodiments described above may also be applied to bidirectional prediction using multiple reference images for interpretation.

[0092] The functions realized by the above-described apparatus may be implemented in a circuit or processing circuit, including a general-purpose processor, application-specific processor, integrated circuit, ASIC, CPU, conventional circuit, and / or combination thereof, programmed to realize the described functions. The processor includes transistors and / or other circuits and is considered a circuit or processing circuit. The processor may be a programmed processor that executes a program stored in memory. In this specification, circuit, unit, and means are hardware programmed to realize or perform the described functions. Such hardware may be any hardware disclosed herein, or any hardware known to be programmed to realize or perform the described functions. If the hardware is a processor that is considered to be of the type of circuit, then the circuit, means, or unit is a combination of hardware and software used to constitute the hardware and / or processor.

[0093] A program may be provided that causes a computer to perform each of the processes described above by the device. The program may be recorded on a computer-readable medium. Using a computer-readable medium, it is possible to install the program on a computer. Here, the computer-readable medium on which the program is recorded may be a non-transient storage medium. The non-transient storage medium is not particularly limited, but may be a storage medium such as a CD-ROM and / or DVD-ROM. The circuits that perform each of the processes described above by the device may be integrated, and the device may be configured as a semiconductor integrated circuit (chipset, SoC).

[0094] The phrases “based on” and “depending on / in response to” as used in this disclosure do not mean “based solely on” or “in response solely” unless otherwise specified. “Based on” means both “based solely on” and “at least partially on.” Similarly, “depending” means both “at least partially on” and “in at least partially on.” The terms “include,” “comprise,” and variations thereof do not mean that only the listed items are included, but that they may include only the listed items or may include additional items in addition to the listed items. Furthermore, the term “or” as used in this disclosure is not intended to mean exclusive OR. Moreover, any reference to elements using designations such as “first,” “second,” etc., as used in this disclosure does not generally limit the quantity or order of those elements. These designations may be used herein as a convenient way to distinguish between two or more elements. Therefore, references to the first and second elements do not imply that only two elements may be adopted therein, or that the first element must precede the second element in any way. In this disclosure, where articles are added by translation, such as a, an, and the in English, these articles shall be plural unless it is clearly indicated from the context that they are not.

[0095] Although the embodiments have been described in detail above with reference to the drawings, the specific configuration is not limited to those described above, and various design changes can be made without departing from the gist of the invention.

[0096] (6) Additional notes: Features of the above-described embodiments are noted below.

[0097] - Appendix 1 Interpretation device (172, 242) that performs interpretation by generating motion vectors for each subblock obtained by dividing the target block of interpretation, comprising: an identification unit (11) that identifies a first reference region on a reference image referenced by a first motion vector assigned to a decoded block surrounding the target block; a determination unit (12) that determines a second reference region on the reference image determined according to the relative position between the target block and the decoded block; an acquisition unit (13) that acquires a group of motion vectors consisting of motion vectors in subblock units in the second reference region; and a correction unit (14) that corrects each motion vector included in the group of motion vectors based on the first motion vector and the second motion vector assigned to the first reference region.

[0098] - Appendix 2 The inter-prediction device according to Appendix 1, wherein the correction unit corrects each motion vector in the motion vector group using a correction vector calculated according to the first motion vector and the second motion vector.

[0099] - Appendix 3 The inter-prediction device according to Appendix 2, wherein the correction unit comprises a calculation unit (15) that calculates the difference between the first motion vector and the second motion vector as the correction vector, and an addition unit (16) that corrects each motion vector in the motion vector group by adding the correction vector.

[0100] - Appendix 4 The inter-prediction device according to any one of Appendix 1 to 3, further comprising an assignment unit (17) that assigns the corrected motion vector group as motion vectors for each sub-block of the target block.

[0101] - Appendix 5: An encoding device (1) equipped with an interpretation device as described in any of Appendix 1 to 3.

[0102] - Appendix 6 A decoding device (2) equipped with an interpretation device as described in any of Appendix 1 to 3.

[0103] - Appendix 7: A program that causes a computer to function as an interpretation device as described in any of the appendices 1 to 3.

[0104] 1: Encoding unit 2: Decoding unit 10: Motion vector prediction unit 11: First reference region identification unit 12: Second reference region determination unit 13: Motion vector group acquisition unit 14: Motion vector correction unit 15: Correction vector calculation unit 16: Correction vector addition unit 17: Motion vector assignment unit 100: Block division unit 110: Subtraction unit 120: Transformation / quantization unit 121: Transformation unit 122: Quantization unit 130: Entropy coding unit 140: Inverse quantization / inverse transformation unit 141: Inverse quantization unit 142: Inverse transformation unit 150: Synthesis unit 160: Memory 161: Reference image storage unit 162: Motion vector storage unit 170: Prediction unit 171: Intra prediction unit 172: Inter prediction unit 173 : Switching unit 200 : Entropy decoding unit 210 : Inverse quantization / inverse transformation unit 211 : Inverse quantization unit 212 : Inverse transformation unit 220 : Synthesis unit 230 : Memory 240 : Prediction unit 241 : Intra prediction unit 242 : Inter prediction unit 243 : Switching unit

Claims

1. An interpretation device that performs interpretation by generating motion vectors for each subblock obtained by dividing a target block for interpretation, comprising: an identification unit that identifies a first reference region on a reference image referenced by a first motion vector assigned to a decoded block surrounding the target block; a determination unit that determines a second reference region on the reference image determined according to the relative position between the target block and the decoded block; an acquisition unit that acquires a group of motion vectors consisting of motion vectors in the subblock units in the second reference region; and a correction unit that corrects each motion vector included in the group of motion vectors based on the first motion vector and the second motion vector assigned to the first reference region.

2. The inter-prediction device according to claim 1, wherein the correction unit corrects each motion vector in the motion vector group using a correction vector calculated according to the first motion vector and the second motion vector.

3. The inter-prediction device according to claim 2, wherein the correction unit comprises a calculation unit that calculates the difference between the first motion vector and the second motion vector as the correction vector, and an addition unit that corrects each motion vector in the motion vector group by adding the correction vector.

4. The inter-prediction device according to any one of claims 1 to 3, further comprising an assignment unit for assigning the corrected motion vector group as motion vectors for each sub-block of the target block.

5. An encoding device comprising an interpretation device according to any one of claims 1 to 3.

6. A decoding device comprising an interpretation device according to any one of claims 1 to 3.

7. A program that causes a computer to function as an interpretation device according to any one of claims 1 to 3.

Images