Dynamic picture image decoder and dynamic picture image coder

Abstract

To provide a dynamic picture image coder and a dynamic picture image decoder which can increase the efficiency of coding.SOLUTION: The dynamic picture image decoder includes: an OOB determination unit for comparing the coordinate of a reference block with the coordinate of a picture and determining whether the reference block is a target of OOB processing; and an OOB mask deriving unit for comparing the coordinate of pixels in the reference block with the coordinate of the picture and deriving mask data showing whether each pixel is usable. The OOB determination unit determines whether the reference block is a target of OOB processing by using the presence or absence of processing which is applied to the reference picture including the reference block.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] An embodiment of the present invention relates to a video decoding device and a video encoding device. [Background technology]

[0002] In order to efficiently transmit or record moving images, a moving image encoding device is used that generates encoded data by encoding moving images, and a moving image decoding device is used that generates a decoded image by decoding the encoded data.

[0003] Specific examples of video coding methods include H.264 / AVC, High-Efficiency Video Coding (HEVC), and Versatile Video Coding (VVC).

[0004] In such a video coding method, images (pictures) constituting a video are managed in a hierarchical structure consisting of slices obtained by dividing images, coding tree units (CTUs) obtained by dividing slices, coding units (sometimes called coding units: CUs) obtained by dividing coding tree units, and transform units (TUs) obtained by dividing coding units, and are coded / decoded for each CU.

[0005] In such a video coding method, a predicted image is usually generated based on a locally decoded image obtained by coding / decoding an input image, and a prediction error (sometimes called a "difference image" or "residual image") obtained by subtracting the predicted image from the input image (original image) is coded. Methods for generating predicted images include inter-frame prediction (inter prediction) and intra-frame prediction (intra prediction). In VVC, as shown in FIG. 8(a), a wrap-around process is performed on the coordinates of a motion vector to make the left and right ends of a picture continuous in the horizontal coordinate system, thereby making it possible to perform motion compensation. Therefore, by applying the wrap-around process to a video in which the left and right ends of a picture are spatially continuous, such as a 360-degree panoramic image or a 360-degree image, it is possible to improve the coding efficiency. In addition, in VVC, as shown in FIG. 8(b), a reference picture resampling (RPR) process is possible, which changes the resolution on a picture-by-picture basis to perform motion compensation. By applying the RPR process to a service with a variable transmission rate, such as video distribution on the Internet, it is possible to improve the image quality.

[0006] In addition, non-patent document 1 discloses an OOB (Out-Of-Boundary) processing technology that replaces part or all of an area of ​​one bi-predictive reference block, including an area outside the picture of the reference image, with part or all of an area of ​​the other reference block. [Prior art documents] [Non-patent literature]

[0007] [Non-Patent Document 1] “AHG12: Enhanced bi-directional motion compensation”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 25th Meeting, by teleconference, JVET-Y0125 Summary of the Invention [Problem to be solved by the invention]

[0008] In the method described in Non-Patent Document 1, when a reference block includes an area outside a picture, a bi-predictive image is derived while deriving a mask indicating the area outside the picture. Therefore, there is a problem that the amount of calculation is large. [Means for solving the problem]

[0009] In order to solve the above problem, a video decoding device according to one aspect of the present invention includes:

[0010] The image coding apparatus includes an OOB determination unit that determines whether a reference block is subject to OOB processing by comparing the coordinates of the reference block with the coordinates of a picture, and an OOB mask derivation unit that derives mask data indicating whether each pixel can be used or not by comparing the coordinates of a pixel included in the reference block with the coordinates of a picture boundary or a region boundary, and the OOB determination unit determines whether the reference block is subject to OOB processing based on the presence or absence of processing applied to a reference picture in which the reference block is included.

[0011] In a video decoding device according to one embodiment of the present invention, the OOB judgment unit compares the valid area of ​​an L0 reference area with the valid area of ​​an L1 reference area, and invalidates OOB judgment for a reference picture in which the valid area of ​​the reference area is larger. Effect of the Invention

[0012] According to an aspect of the present invention, it is possible to reduce the amount of calculation required for OOB processing in video encoding / decoding processing. [Brief description of the drawings]

[0013] [Figure 1] 1 is a schematic diagram showing a configuration of an image transmission system according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a diagram showing a hierarchical structure of data in an encoded stream. [Diagram 3]FIG. 1 is a schematic diagram showing a configuration of a video decoding device. [Figure 4] FIG. 13 is a schematic diagram showing a configuration of an inter-prediction image generating unit. [Diagram 5] FIG. 13 is a schematic diagram showing a configuration of an inter-prediction parameter derivation unit. [Figure 6] 11 is a flowchart illustrating a schematic operation of the video decoding device. [Figure 7] FIG. 1 is a block diagram showing a configuration of a video encoding device. [Figure 8] FIG. 1 is a diagram illustrating an example of wraparound processing and reference picture resampling processing of the prior art. [Figure 9] 11 is a flowchart illustrating a schematic operation of OOB processing. [Figure 10] FIG. 1 is a schematic diagram showing an example of OOB determination. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] (First embodiment)

[0015] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0016] FIG. 1 is a schematic diagram showing the configuration of an image transmission system 1 according to this embodiment.

[0017] The image transmission system 1 is a system that transmits an encoded stream obtained by encoding a target image, decodes the transmitted encoded stream, and displays an image. The image transmission system 1 includes a video encoding device (image encoding device) 11, a network 21, a video decoding device (image decoding device) 31, and a video display device (image display device) 41.

[0018] An image T is input to the video encoding device 11 .

[0019] The network 21 transmits the encoded stream Te generated by the video encoding device 11 to the video decoding device 31. The network 21 is the Internet, a wide area network (WAN), a local area network (LAN), or a combination of these. The network 21 is not necessarily limited to a bidirectional communication network, and may be a unidirectional communication network that transmits broadcast waves such as terrestrial digital broadcasting and satellite broadcasting. The network 21 may also be replaced by a storage medium on which the encoded stream Te is recorded, such as a DVD (Digital Versatile Disc: registered trademark) or a BD (Blue-ray Disc: registered trademark).

[0020] The video decoding device 31 decodes each of the coded streams Te transmitted by the network 21, and generates one or more decoded images Td.

[0021] The video display device 41 displays all or part of one or more decoded images Td generated by the video decoding device 31. The video display device 41 includes a display device such as a liquid crystal display or an organic EL (Electro-luminescence) display. The display may be in the form of a stationary display, a mobile display, an HMD, or the like. When the video decoding device 31 has high processing power, it displays high quality images, and when it has only low processing power, it displays images that do not require high processing power or display power.

[0022] <operator>

[0023] The operators used in this specification are listed below.

[0024] >> is a right bit shift, << is a left bit shift, & is a bitwise AND, | is a bitwise OR, |= is the OR assignment operator, and || indicates logical OR.

[0025] x?y:z is a ternary operator that takes y when x is true (non-zero) and z when x is false (0).

[0026] Clip3(a, b, c) is a function that clips c to a value between a and b (inclusive), returning a if c < a, b if c > b, and c otherwise (where a <= b).

[0027] ClipH(o, W, x) is a function that returns x if x < 0, x - o if x > W - 1, and x otherwise.

[0028] sign(a) is a function that returns 1 if a > 0, 1 if a == 0, and -1 if a < 0.

[0029] abs(a) is a function that returns the absolute value of a.

[0030] Int(a) is a function that returns the integer value of a.

[0031] floor(a) is a function that returns the largest integer less than or equal to a.

[0032] ceil(a) is a function that returns the smallest integer greater than or equal to a.

[0033] a / d represents the division of a by d, rounded down to the nearest integer.

[0034] <Structure of the Encoded Stream Te>[[]]END]]

[0035] Prior to the detailed description of the moving image encoding device 11 and the moving image decoding device 31 according to this embodiment, the data structure of the encoded stream Te generated by the moving image encoding device 11 and decoded by the moving image decoding device 31 will be described.

[0036] Fig. 2 is a diagram showing a hierarchical structure of data in an encoded stream Te. The encoded stream Te illustratively includes a sequence and a plurality of pictures constituting the sequence. (a) to (f) of Fig. 2 show an encoded video sequence that defines a sequence SEQ, an encoded picture that defines a picture PICT, an encoded slice that defines a slice S, encoded slice data that defines slice data, an encoding tree unit included in the encoded slice data, and an encoding unit included in the encoding tree unit, respectively.

[0037] (Coded Video Sequence)

[0038] The coded video sequence defines a set of data that the video decoding device 31 refers to in order to decode the sequence SEQ to be processed. As shown in Fig. 2, the sequence SEQ includes a video parameter set, a sequence parameter set SPS (Sequence Parameter Set), a picture parameter set PPS (Picture Parameter Set), an adaptation parameter set (APS), a picture PICT, and supplemental enhancement information SEI (Supplemental Enhancement Information).

[0039] The video parameter set VPS specifies a set of coding parameters common to multiple videos composed of multiple layers, as well as a set of coding parameters related to multiple layers and each individual layer included in the video.

[0040] The sequence parameter set SPS specifies a set of coding parameters that the video decoding device 31 refers to in order to decode the target sequence. For example, the width and height of a picture are specified. Note that there may be multiple SPSs. In that case, one of the multiple SPSs is selected from the PPS.

[0041] The picture parameter set PPS specifies a set of coding parameters that the video decoding device 31 refers to in order to decode each picture in the target sequence. For example, the picture parameter set PPS includes a reference value of the quantization width (pic_init_qp_minus26) used in decoding the picture and a flag (weighted_pred_flag) indicating the application of weighted prediction. Note that there may be multiple PPSs. In that case, one of the multiple PPSs is selected for each picture in the target sequence.

[0042] (Encoded Picture)

[0043] A coded picture defines a set of data to be referenced by the video decoding device 31 in order to decode a picture PICT to be processed. As shown in FIG. 2, the picture PICT includes slices 0 to NS-1 (NS is the total number of slices included in the picture PICT).

[0044] In the following description, when there is no need to distinguish between slices 0 to NS-1, the subscripts of the symbols may be omitted. The same applies to other data that are included in the coded stream Te and that are to be described below and that are to be given subscripts.

[0045] (Coded Slice)

[0046] A coded slice defines a set of data to be referenced by the video decoding device 31 in order to decode a target slice S. As shown in Fig. 2, a slice includes a slice header and slice data.

[0047] The slice header includes a group of coding parameters to be referred to by the video decoding device 31 in order to determine a decoding method for the current slice. Slice type designation information (slice_type) that designates the slice type is an example of a coding parameter included in the slice header.

[0048] Slice types that can be specified by the slice type specification information include (1) an I slice that uses only intra prediction when encoding, (2) a P slice that uses unidirectional prediction or intra prediction when encoding, and (3) a B slice that uses unidirectional prediction, bidirectional prediction, or intra prediction when encoding. Note that inter prediction is not limited to uni-prediction or bi-prediction, and a predicted image may be generated using more reference pictures. Hereinafter, when referring to P or B slice, it refers to a slice including a block that can use inter prediction.

[0049] In addition, the slice header may include a reference to a picture parameter set PPS (pic_parameter_set_id).

[0050] (Encoded slice data)

[0051] The coded slice data specifies a set of data to be referenced by the video decoding device 31 in order to decode the slice data to be processed. As shown in Fig. 2(d), the slice data includes a CTU. A CTU is a block of a fixed size (e.g., 64x64) that constitutes a slice, and is also called a Largest Coding Unit (LCU).

[0052] (coding tree unit)

[0053] 2 specifies a set of data that the video decoding device 31 refers to in order to decode a CTU to be processed. The CTU is divided into coding units CU, which are basic units of coding processing, by recursive quad tree division (QT (Quad Tree) division), binary tree division (BT (Binary Tree) division), or ternary tree division (TT (Ternary Tree) division). BT division and TT division are collectively called multi tree division (MT (Multi Tree) division). A node of a tree structure obtained by recursive quad tree division is called a coding node. Intermediate nodes of the quad tree, binary tree, and ternary tree are coding nodes, and the CTU itself is specified as the top coding node. The lowest coding node is specified as a coding unit.

[0054] (Encoding Unit)

[0055] 2 specifies a set of data to be referenced by the video decoding device 31 in order to decode a coding unit to be processed. Specifically, a CU is composed of a CU header CUH, prediction parameters, transformation parameters, quantization transformation coefficients, etc. The CU header specifies a prediction mode, etc.

[0056] The prediction process may be performed on a CU basis, or on a sub-CU basis by further dividing a CU. If the size of a CU and a sub-CU are the same, there is one sub-CU in the CU. If the size of a CU is larger than that of a sub-CU, the CU is divided into sub-CUs. For example, if the CU is 8x8 and the sub-CU is 4x4, the CU is divided into 2 parts horizontally and 2 parts vertically, into 4 sub-CUs.

[0057] Prediction types (prediction modes) include intra prediction (MODE_INTRA), inter prediction (MODE_INTER), and intra block copy (MODE_IBC). Intra prediction is a prediction within the same picture, and inter prediction refers to a prediction process performed between different pictures (for example, between display times or between layer images).

[0058] The transform and quantization processes are performed in units of CUs, but the quantized transform coefficients may be entropy coded in units of sub-blocks such as 4x4.

[0059] (Prediction parameters)

[0060] The predicted image is derived from prediction parameters associated with the block, which include intra-prediction and inter-prediction parameters.

[0061] (Inter prediction parameters)

[0062] The prediction parameters of inter prediction will be described. The inter prediction parameters are composed of prediction list use flags predFlagL0 and predFlagL1, reference picture indexes refIdxL0 and refIdxL1, and motion vectors mvL0 and mvL1. predFlagL0 and predFlagL1 are flags indicating whether or not a reference picture list (L0 list, L1 list) is used, and when the value is 1, the corresponding reference picture list is used. Note that in this specification, when the term "flag indicating whether or not XX" is used, a flag other than 0 (for example, 1) is XX, and 0 is not XX, and in logical negation, logical product, etc., 1 is treated as true and 0 is treated as false (similar below). However, in an actual device or method, other values ​​can also be used as true and false values.

[0063] Syntax elements for deriving inter prediction parameters include, for example, a merge flag merge_flag (general_merge_flag), a merge index merge_idx, merge_subblock_flag, regulare_merge_flag, ciip_flag, merge_gpm_partition_idx, merge_gpm_idx0, merge_gpm_idx1, inter_pred_idc, a reference picture index refIdxLX, mvp_LX_idx, a difference vector mvdLX, and a motion vector precision mode amvr_mode. merge_subblock_flag is a flag indicating whether to use inter prediction in subblock units. regulare_merge_flag is a flag indicating whether to use a normal merge mode or MMVD. ciip_flag is a flag indicating whether to use a CIIP (combined inter-picture merge and intra-picture prediction) mode. merge_gpm_partition_idx is an index indicating a partition shape of the GPM mode. merge_gpm_idx0 and merge_gpm_idx1 are indices that indicate merge indices in the GPM mode. inter_pred_idc is an inter prediction identifier for selecting a reference picture to be used in the AMVP mode. mvp_LX_idx is a predicted vector index for deriving a motion vector.

[0064] (See picture list)

[0065] The reference picture list is a list of reference pictures stored in the reference picture memory 306. In each CU, refIdxLX is used to specify which picture in the reference picture list RefPicListX (X=0 or 1) is actually referenced. Note that LX is a description method used when there is no distinction between L0 prediction and L1 prediction, and hereinafter, parameters for the L0 list and parameters for the L1 list are distinguished by replacing LX with L0 or L1.

[0066] (Merge prediction and AMVP prediction)

[0067] There are two methods of decoding (encoding) prediction parameters: merge prediction mode (merge mode) and AMVP (Advanced Motion Vector Prediction, adaptive motion vector prediction) mode, and general_merge_flag is a flag for identifying these. The merge mode is a prediction mode in which some or all of the motion vector difference is omitted, and the prediction list usage flag predFlagLX, the reference picture index refIdxLX, and the motion vector mvLX are not included in the encoded data, but are derived from the prediction parameters of the neighboring blocks that have already been processed. The AMVP mode is a mode in which inter_pred_idc, refIdxLX, and mvLX are included in the encoded data. Note that mvLX is encoded as mvp_LX_idx, which identifies the prediction vector mvpLX, and the difference vector mvdLX. The general name for prediction modes in which the motion vector difference is omitted or simplified is called the general merge mode, and the general merge mode and the AMVP prediction may be selected by the general_merge_flag.

[0068] If general_merge_flag is 1, regular_merge_flag may be transmitted separately. If regular_merge_flag is 1, normal merge mode or MMVD may be selected, otherwise CIIP mode or GPM mode may be selected. In CIIP mode, a predicted image is generated by a weighted sum of an inter predicted image and an intra predicted image. In GPM mode, a predicted image is generated by dividing a target CU into two non-rectangular prediction units by a line segment.

[0069] inter_pred_idc is a value indicating the type and number of reference pictures, and takes one of the values ​​PRED_L0, PRED_L1, and PRED_BI. PRED_L0 and PRED_L1 indicate uni-prediction using one reference picture managed in the L0 list and L1 list, respectively. PRED_BI indicates bi-prediction using two reference pictures managed in the L0 list and L1 list.

[0070] The merge_idx is an index indicating which prediction parameter is to be used as the prediction parameter of the current block among prediction parameter candidates (merge candidates) derived from blocks for which processing has been completed.

[0071] (Motion Vector)

[0072] mvLX indicates the amount of shift between blocks on two different pictures. A prediction vector and a difference vector related to mvLX are called mvpLX and mvdLX, respectively.

[0073] (Inter prediction identifier inter_pred_idc and prediction list usage flag predFlagLX)

[0074] The relationship between inter_pred_idc, predFlagL0, and predFlagL1 is as follows, and they are mutually convertible.

[0075] inter_pred_idc = (predFlagL1<<1)+predFlagL0

[0076] predFlagL0 = inter_pred_idc & 1

[0077] predFlagL1 = inter_pred_idc >> 1

[0078] In addition, the inter prediction parameter may use a prediction list usage flag or an inter prediction identifier. Furthermore, the determination using the prediction list usage flag may be replaced with a determination using the inter prediction identifier. Conversely, the determination using the inter prediction identifier may be replaced with a determination using the prediction list usage flag.

[0079] (Bi-predictive biPred decision)

[0080] The flag biPred indicating whether or not the prediction is performed can be derived based on whether or not two prediction list usage flags are both 1. For example, the flag can be derived by the following formula.

[0081] biPred = (predFlagL0==1 && predFlagL1==1)

[0082] Alternatively, biPred can be derived based on whether the inter prediction identifier is a value indicating the use of two prediction lists (reference pictures). For example, biPred can be derived using the following formula:

[0083] biPred = (inter_pred_idc==PRED_BI) ? 1 : 0

[0084] (Configuration of a video decoding device)

[0085] The configuration of a video decoding device 31 (FIG. 3) according to this embodiment will be described.

[0086] The video decoding device 31 includes an entropy decoding unit 301, a parameter decoding unit (prediction image decoding device) 302, a loop filter 305, a reference picture memory 306, a prediction parameter memory 307, a prediction image generating unit (prediction image generating device) 308, an inverse quantization and inverse transform unit 311, an adder 312, and a prediction parameter derivation unit 320. Note that, in accordance with the video encoding device 11 described below, the video decoding device 31 may also be configured not to include the loop filter 305.

[0087] The parameter decoding unit 302 further includes a header decoding unit 3020, a CT information decoding unit 3021, and a CU decoding unit 3022 (prediction mode decoding unit), and the CU decoding unit 3022 further includes a TU decoding unit 3024. These may be collectively referred to as a decoding module. The header decoding unit 3020 decodes parameter set information such as VPS, SPS, PPS, and APS, and slice header (slice information) from the encoded data. The CT information decoding unit 3021 decodes the CT from the encoded data. The CU decoding unit 3022 decodes the CU from the encoded data. The TU decoding unit 3024 decodes the CU from the encoded data.

[0088] When a prediction error is included in a TU, the TU decoding unit 3024 decodes the QP update information and the quantized transform coefficient from the encoded data. The QP update information is a difference value from the quantization parameter predicted value qPpred, which is a predicted value of the quantization parameter QP.

[0089] The predicted image generation unit 308 includes an inter predicted image generation unit 309 (FIG. 4) and an intra predicted image generation unit 310.

[0090] The prediction parameter derivation unit 320 includes the inter prediction parameter derivation unit 303 (FIG. 5) and an intra prediction parameter derivation unit.

[0091] In the following, an example will be described in which CTU and CU are used as processing units, but the present invention is not limited to this example and processing may be performed in sub-CU units. Alternatively, CTU and CU may be read as blocks and sub-CU as sub-blocks, and processing may be performed in block or sub-block units.

[0092] The entropy decoding unit 301 performs entropy decoding on the externally input coded stream Te to decode individual codes (syntax elements). Entropy coding includes a method of variable-length coding the syntax elements using a context (probability model) adaptively selected according to the type of syntax element or surrounding circumstances, and a method of variable-length coding the syntax elements using a predetermined table or formula.

[0093] The entropy decoding unit 301 outputs the decoded code to the parameter decoding unit 302. The decoded code is, for example, a prediction mode predMode, general_merge_flag, merge_idx, inter_pred_idc, refIdxLX, mvp_LX_idx, mvdLX, amvr_mode, etc. Control of which code to decode is performed based on an instruction from the parameter decoding unit 302.

[0094] (Basic flow)

[0095] FIG. 6 is a flowchart illustrating a schematic operation of the video decoding device 31.

[0096] (S1100: Decode Parameter Set Information) The header decoding unit 3020 decodes parameter set information such as the VPS, SPS, and PPS from the encoded data.

[0097] (S1200: Decode slice information) The header decoding unit 3020 decodes the slice header (slice information) from the encoded data.

[0098] Thereafter, the video decoding device 31 repeats the processes from S1300 to S5000 for each CTU included in the target picture to derive a decoded image of each CTU.

[0099] (S1300: Decode CTU Information) The CT information decoding unit 3021 decodes the CTU from the encoded data.

[0100] (S1400: Decode CT Information) The CT information decoding unit 3021 decodes the CT from the encoded data.

[0101] (S1500: CU Decoding) The CU decoding unit 3022 performs S1510 and S1520 to decode the CU from the encoded data.

[0102] (S1510: Decode CU Information) The CU decoding unit 3022 decodes the CU information, prediction information, TU division flag, CU residual flag, and the like from the encoded data.

[0103] (S1520: Decode TU information) When a prediction error is included in a TU, the TU decoding unit 3024 decodes the quantized prediction error and the like from the encoded data.

[0104] (S2000: Generation of predicted image) The predicted image generation unit 308 generates a predicted image for each block included in the current CU based on the prediction information.

[0105] (S3000: Inverse Quantization and Inverse Transformation) The inverse quantization and inverse transform unit 311 executes inverse quantization and inverse transform processing for each TU included in the target CU.

[0106] (S4000: Generate decoded image) The adder 312 adds the predicted image supplied from the predicted image generation unit 308 and the prediction error supplied from the inverse quantization and inverse transform unit 311 to generate a decoded image of the current CU.

[0107] (S5000: Loop Filter) The loop filter 305 applies a loop filter such as a deblocking filter, SAO, or ALF to the decoded image to generate a decoded image.

[0108] The loop filter 305 is a filter provided in the encoding loop, which removes block distortion and ringing distortion to improve image quality. The loop filter 305 applies a filter such as a deblocking filter, a sample adaptive offset (SAO), and an adaptive loop filter (ALF) to the decoded image of the CU generated by the adder 312.

[0109] The reference picture memory 306 stores the decoded image of the CU generated by the adder 312 in a location that is determined in advance for each current picture and current CU.

[0110] The prediction parameter memory 307 stores prediction parameters at a predetermined position for each CTU or CU to be decoded. Specifically, the prediction parameter memory 307 stores the parameters decoded by the parameter decoding unit 302 and the prediction mode predMode separated by the entropy decoding unit 301.

[0111] The predicted image generating unit 308 receives input of predMode, prediction parameters, etc. The predicted image generating unit 308 also reads a reference picture from the reference picture memory 306. The predicted image generating unit 308 generates a predicted image of a block or sub-block using the prediction parameters and the read reference picture (reference block) in the prediction mode indicated by predMode. Here, the reference block is a set of pixels on the reference picture (usually rectangular, so called a block), and is an area to be referenced for generating a predicted image.

[0112] (Configuration of inter-prediction parameter derivation unit)

[0113] As shown in Fig. 5, the inter prediction parameter derivation unit 303 derives inter prediction parameters by referring to prediction parameters stored in the prediction parameter memory 307 based on the syntax elements input from the parameter decoding unit 302. The inter prediction parameter derivation unit 303 also outputs the inter prediction parameters to the inter prediction image generation unit 309 and the prediction parameter memory 307. The inter prediction parameter derivation unit 303 and its internal elements, the AMVP prediction parameter derivation unit 3032, the merge prediction parameter derivation unit 3036, the MMVD prediction unit 30376, and the MV addition unit 3038, are means common to the video encoding device and the video decoding device, and therefore may be collectively referred to as a motion vector derivation unit (motion vector derivation device).

[0114] When general_merge_flag is 1, that is, when it indicates the merge prediction mode, it derives merge_idx and outputs it to the merge prediction parameter derivation unit 3036 .

[0115] When general_merge_flag is 0, that is, when it indicates the AMVP prediction mode, the AMVP prediction parameter derivation unit 3032 derives mvpLX from inter_pred_idc, refIdxLX, or mvp_LX_idx.

[0116] (MV addition section)

[0117] The MV adder 3038 adds the derived mvpLX and mvdLX to derive mvLX.

[0118] (Merge prediction)

[0119] The merge prediction parameter derivation unit 3036 derives merge candidates including prediction parameters (predFlagLX, mvLX, refIdxLX) and constructs a merge candidate list. Furthermore, the merge prediction parameter derivation unit 3036 selects motion information (mvLXN[0], mvLXN[1]), predFlagLXN, and refIdxLXN of a merge candidate N indicated by merge_idx from among the merge candidates included in the merge candidate list as inter prediction parameters of the current block. The merge prediction parameter derivation unit 3036 stores the inter prediction parameters of the selected merge candidate in the prediction parameter memory 307 and outputs them to the inter prediction image generation unit 309.

[0120] (Inter-prediction image generation unit 309)

[0121] When predMode indicates inter prediction, the inter prediction image generation unit 309 generates a prediction image of a block or sub-block by inter prediction using the inter prediction parameters and reference picture input from the inter prediction parameter derivation unit 303.

[0122] 4 is a schematic diagram showing a configuration of an inter-prediction image generation unit 309 included in the prediction image generation unit 308 according to this embodiment. The inter-prediction image generation unit 309 includes a motion compensation unit (prediction image generation device) 3091, an OOB processing unit 3092, and a synthesis unit 3095. The synthesis unit 3095 includes a weighted prediction unit 3094.

[0123] (Motion Compensation)

[0124] The motion compensation unit 3091 (interpolated image generation unit 3091) generates an interpolated image (motion compensated image) by reading a reference block from the reference picture memory 306 based on the inter prediction parameters (predFlagLX, refIdxLX, mvLX) input from the inter prediction parameter derivation unit 303. The reference block is a block at a position shifted by mvLX from the position of the target block on the reference picture RefPicLX specified by refIdxLX. Here, if mvLX is not integer precision, a filter for generating pixels at decimal positions called a motion compensation filter is applied to generate an interpolated image.

[0125] The motion compensation unit 3091 derives the integer position (xInt, yInt) and phase (xFrac, yFrac) corresponding to the top left coordinates (xPb, yPb) of a block of size bW*bH, the coordinates within the prediction block (xL, yL), and the motion vector (mvLX[0], mvLX[1]) using the following formula (MC-P1).

[0126] xInt = xPb+(mvLX[0]>>(log2(MVPREC)))+xL

[0127] xFrac = mvLX[0]&(MVPREC-1)

[0128] yInt = yPb+(mvLX[1]>>(log2(MVPREC)))+yL

[0129] yFrac = mvLX[1]&(MVPREC-1)

[0130] Here, MVPREC indicates the accuracy of mvLX (1 / MVPREC pixel accuracy), log2MVPREC=(log2(MVPREC)), x=0...bW-1, y=0...bH-1. For example, MVPREC=16.

[0131] In order to perform RPR, the values ​​may be derived as in (MC-P2) described later. Furthermore, the motion compensation unit 3091 may correct the positions of (xInt, yInt) derived in (MC-P1) and (MC-P2) for wraparound purposes.

[0132] If the flag to treat subpicture boundaries as picture boundaries is enabled (sps_subpic_treated_as_pic_flag == 1) and the number of subpictures for the reference picture is greater than 1 (sps_num_subpics_minus1 > 0 for reference picture refPicLX)

[0133] xInt = Clip3( SubpicLeftBoundaryPos, SubpicRightBoundaryPos, refWraparoundEnabledFlag ? ClipH( ( PpsRefWraparoundOffset ) * MinCbSizeY, picW, xInt ) : xInt )

[0134] yInt = Clip3( SubpicTopBoundaryPos, SubpicBotBoundaryPos, yInt )

[0135] Here, SubpicLeftBoundaryPos, SubpicRightBoundaryPos, SubpicTopBoundaryPos, and SubpicBottomBoundaryPos are the left, right, top, and bottom boundary positions of the subpicture, respectively.

[0136] In all other cases (when the flag to treat subpicture boundaries as picture boundaries is disabled (sps_subpic_treated_as_pic_flag == 0) or when the number of subpictures in the reference picture is 1 (sps_num_subpics_minus1 for reference picture refPicLX == 0)

[0137] xInt = Clip3( 0, picW - 1, refWraparoundEnabledFlag ? ClipH( ( PpsRefWraparoundOffset ) * MinCbSizeY, picW, xInt ) : xInt )

[0138] yInt = Clip3( 0, picH - 1, yInt )

[0139] Here, refWraparoundEnabledFlag = pps_ref_wraparound_enabled_flag && !refPicIsScaled, PpsRefWraparoundOffset = pps_pic_width_in_luma_samples / MinCbSizeY - pps_pic_width_minus_wraparound_offset, where MinCbSizeY is a predetermined constant or variable (e.g., 4), and pps_pic_width_minus_wraparound_offset is an offset decoded from the encoded data indicating the position of the wraparound.

[0140] The motion compensation unit 3091 derives a temporary image temp[][] by performing horizontal interpolation processing on the reference picture refImg using an interpolation filter. In the following, Σ is the sum for k=0..NTAP-1, mcFilter[Frac][k] is the k-th interpolation filter coefficient in phase Frac, shift1 is a normalization parameter that adjusts the value range, and offset1=1<<(shift1-1).

[0141] temp[x][y] = (ΣmcFilter[xFrac][k]*refImg[xInt+k-NTAP / 2+1][yInt]+offset1)>>shift1

[0142] Next, the motion compensation unit 3091 derives the interpolated image Pred[][] by vertically interpolating the temporary image temp[][]. In the following, Σ is the sum for k=0..NTAP-1, shift2 is a normalization parameter that adjusts the value range, and offset2=1<<(shift2-1).

[0143] Pred[x][y] = (ΣmcFilter[yFrac][k]*temp[x][y+k-NTAP / 2+1]+offset2)>>shift2 (formula MC-1)

[0144] (Out of Picture Range (OOB) Processing)

[0145] The OOB processing unit 3092 includes an OOB determination unit 30921 and an OOB mask derivation unit 30922. The OOB determination unit 30921 determines, based on the inter prediction parameters (predFlagLX, refIdxLX, mvLX, etc.) input from the inter prediction parameter derivation unit 303, whether or not each reference block in the bi-prediction mode includes an area outside the valid range of the reference picture (OOB: Out-Of-Boundary).

[0146] The above determination may be made based on whether or not an area outside the valid range of a picture or subpicture is included, or based on whether or not an area outside a slice is included.

[0147] Description will be given with reference to FIG.

[0148] When the OOB determination unit 30921 determines that some pixels of the reference block are outside the picture (outside the valid range), that is, the reference block is OOB (isOOB==true, S1002), the OOB mask derivation unit 30922 derives mask data (mask value, OOB mask) indicating which area of ​​the reference block is outside the valid range (S1004). The inter predicted image generation unit 309 applies the OOB mask to the reference block that is OOB to generate a predicted image (S1006). The inter predicted image generation unit 309 generates a normal predicted image for the reference block that is not OOB (S1008). Details will be described below.

[0149] (OOB judgment)

[0150] In bi-prediction, the OOB determination unit 30921 determines whether or not a part of each of two (first and second) reference blocks is outside the picture (OOB). The area of ​​the LX reference block (X=0 or 1) corresponding to the reference image refLX is represented by the block upper left coordinate (xRefLX, yRefLX) and the block size (width and height: bW, bH). The width and height of the picture are picW and picH, respectively. In the following, the value obtained by converting mvLX into an integer according to the accuracy MVPREC of mvLX is used as the coordinate system. Using these values, the OOB determination unit 30921 derives a truth value isOOB[X] indicating whether or not the LX reference block includes outside the picture (outside the valid range) according to the following formula. The area within the valid range is referred to as the valid area.

[0151] An example is shown in Figure 10. Figure 10 is a diagram in integer units, and the motion vector mvLX' and the reference pixel position (xRefLX', yRefLX') are parameters that represent the motion vector mvLX and the reference pixel position (xRefLX, yRefLX) as integers. The example in the diagram is X=0, but the same applies when X=1.

[0152] xRefLX = (xPb< <log2MVPREC) + mvLX[0] (DecideOOB1)

[0153] yRefLX = (yPb << log2MVPREC) + mvLX[1]

[0154] picMinX = 0 - MVPREC / 2

[0155] picMinY = 0 - MVPREC / 2

[0156] picMaxX = ((picW - 1) << log2MVPREC) + MVPREC / 2

[0157] picMaxY = ((picH - 1) << log2MVPREC) + MVPREC / 2

[0158] isOOB[X] = (xRefLX < picMinX) || (yRefLX < picMinY) || (xRefLX + (bW << log2MVPREC) - 1) >= picMaxX) || (yRefLX + (bH << log2MVPREC) - 1) >= picMaxY)

[0159] The left shift of log2MVPREC bits ("<<log2MVPREC") can be processed as multiplication with MVPREC ("*MVPREC"), or the subsequent "*MVPREC" can be processed as "<<log2MVPREC".

[0160] Alternatively, the derivation formula of isOOB[X] in (DecideOOB1) may be as follows.

[0161] isOOB[X] = (xRefLX <= picMinX) || (yRefLX <= picMinY) || (xRefLX + (bW << log2MVPREC) - 1) >= picMaxX) || (yRefLX + (bH << log2MVPREC) - 1) >= picMaxY)

[0162] Alternatively, the OOB determination unit 30921 may omit the term for rounding and make a simplified determination as follows.

[0163] picMaxX = (picW-1)< <log2MVPREC) (DecideOOB2)

[0164] picMaxY = (picH-1)< <log2MVPREC)

[0165] isOOB[X] = (xRefLX < 0) || (yRefLX < 0) || (xRefLX + (bW<<log2MVPREC) - 1) > picMaxX) || (yRefLX + (bH<<log2MVPREC) - 1) > pixMaxY)

[0166] Also, the OOB determination unit 30921 may perform the determination using integer coordinates instead of decimal boundary coordinates.

[0167] xRefLX = xPb + (mvLX[0]>>log2MVPREC) (DecideOOB3)

[0168] yRefLX = yPb + (mvLX[1]>>log2MVPREC)

[0169] isOOB[X] = (xRefLX < 0) || (yRefLX < 0) || (xRefLX + bW - 1 >= picW-1) || (yRefLX + bH - 1 >= picH-1)

[0170] Alternatively, the derivation formula for isOOB[X] in (DecideOOB3) may be the following formula.

[0171] isOOB[X] = (xRefLX <= 0) || (yRefLX <= 0) || (xRefLX + bW - 1 >= picW-1) || (yRefLX + bH - 1 >= picH-1)

[0172] According to the above, since integer coordinates obtained by right-shifting the motion vector mvLX[] with MV precision are used, an effect is achieved in that the amount of processing can be reduced compared to determination with decimal precision.

[0173] The OOB determination unit 30921 may switch whether to apply OOB processing depending on the block size. For example, the OOB determination unit 30921 may use the following formula to derive isOOB[X], and set isOOB=true only to blocks whose width and height are both BLKTHW1, BLKTHH1 pixels or more, and apply OOB processing.

[0174] isOOB[X] = (bW >= BLKTHW1 && bH >= BLKTHH1) && ((xRefLX < picMinX) || (yRefLX < picMinY) || (xRefLX + bW*MVPREC - 1 >= picMaxX) || (yRefLX + bH*MVPREC - 1 >= picMaxY))

[0175] Here, applying the OOB process means that isOOB[X]=true. This means that the motion compensation unit 3091 (interpolated image generation unit 3091) uses any of the reference images preferentially without weighting to derive the predicted image Pred[][].

[0176] For example, BLKTHW1=4, 8, 16, 32, BLKTHH1=4, 8, 16, 32, etc. The block size threshold for applying OOB may be 8x8, 4x4, 4x8, 8x4, 16x16, etc.

[0177] Alternatively, OOB processing may be prohibited in large blocks.

[0178] isOOB[X] = (bW <= BLKTHW2 && bH <= BLKTHW2) && ((xRefLX < picMinX) || (yRefLX < picMinY) || (xRefLX + bW*MVPREC - 1 >= picMaxX) || (yRefLX + bH*MVPREC - 1 >= picMaxY))

[0179] For example, BLKTHW2=16, 32, 64, BLKTHH2=16, 32, 64, etc.

[0180] Alternatively, OOB processing may be applied only to blocks within a particular range of sizes.

[0181] isOOB[X] = (bW > BLKTHW3 && bW <= BLKTHW4 && bH > BLKTHH3 && bH <= BLKTHH4) && ((xRefLX < picMinX) || (yRefLX < picMinY) || (xRefLX + bW*MVPREC - 1 >= picMaxX) || (yRefLX + bH*MVPREC - 1 >= picMaxY))

[0182] For example, (BLKTHW3, BLKTHW4) = {(4,16),(8,32),(16,32),(32,64)}, (BLKTHH3,BLKHH4) = {(4,16),(8,32),(16,32),(32,64)}, etc.

[0183] Alternatively, the OOB processing may be applied based on the size of the area.

[0184] isOOB[X] = (bW*bH >= BLKTHW5*BLKTHH5) && ((xRefLX < picMinX) || (yRefLX < picMinY) || (xRefLX + bW*MVPREC - 1 >= picMaxX) || (yRefLX + bH*MVPREC - 1 >= picMaxY))

[0185] For example, BLKTHW5 = 4, 8, 16, 32, BLKTHH5 = 4, 8, 16, 32, etc. may be used. Alternatively, BLKTHS5 may be used instead of BLKTHW5 * BLKTHH5. For example, BLKTHS5 = 32, 64, 128, etc. may be used.

[0186] In the above example, conditions for restricting the block size are added based on the OOB determination formula of (DecideOOB1), but conditions for restricting the block size may be added based on the OOB determination formula of (DecideOOB2) or (DecideOOB3).

[0187] In the above example, (xRefLX < picMinX) || (yRefLX < picMinY) is used as part of the condition of the derivation formula of isOOB[X], but (xRefLX <= picMinX) || (yRefLX <= picMinY) may be used.

[0188] As described above, it is possible to limit the block size to which the OOB process is applied and reduce the computational complexity.

[0189] The OOB determination unit 30921 may be configured to avoid both isOOB[X] and isOOB[1 - X] (X = 0, 1) becoming 1. That is, in both of the reference blocks, the inter-prediction image generation unit 309 is prohibited from generating a prediction image using the OOB mask. As a result, for one of the reference blocks, isOOB[X] will always be false, that is, outside the scope of the OOB process, and the creation and application of the OOB mask can be omitted in the subsequent stage. Therefore, it is possible to reduce the computational complexity.

[0190] <Example 1 of OOB determination omission configuration>

[0191] The OOB determination unit 30921 may derive isOOB[X] (X=0,1) by a determination according to the size of the effective range of the L0 reference area and the L1 reference area. The LX reference block with the larger effective area InsideAreaX included in the picture is set to X=K (K=0,1). The OOB determination of the reference picture with the larger effective area is set to invalid (OOB determination presence / absence flag isOOB[K]=false), and the OOB determination of the other with the smaller effective area (isOOB[!K] or isOOB[1-K]) is set to valid (isOOB[1-K]=true). The OOB determination unit 30921 obtains InsideAreaX, for example, by the following formula. Note that (0,1) may be used as the value of (false, true), or another value (1,0) may be used. The effective range of the LX reference area may be derived from the common part of the LX reference block and the picture.

[0192] xRefLX = (xPb< <log2MVPREC) + mvLX[0]

[0193] yRefLX = (yPb< <log2MVPREC) + mvLX[1]

[0194] InsideWidthX = (min(picMaxX, xRefLX + bW*MVPREC-1) - max(picMinX, xRefLX))

[0195] InsideHeightX = (min(picMaxY, yRefLX + bH*MVPREC-1) - max(picMinY, yRefLX))

[0196] InsideAreaX = (InsideWidthX>0 && InsideHeightX>0) ? InsideWidthX * InsideHeightX : 0

[0197] If the value of InsideWidthX or InsideHeightX is negative, the reference block does not overlap with the picture (the reference block is not included in the picture), so InsideAreaX=0 may be used.

[0198] The OOB determination unit 30921 derives InsideArea0 and InsideArea1 for the L0 reference block and the L1 reference block, respectively, and derives isOOB[X] as follows.

[0199] if (InsideArea0 >= InsideArea1) { isOOB[0] = false, isOOB[1] = true } else { isOOB[0] = true, isOOB[1] = false }

[0200] Alternatively, the determination value of one, isOOB[X], may be derived first, as shown below, and then the determination value of the other, isOOB[Y] (Y=1-X), may be derived from the determination value of isOOB[X].

[0201] isOOB[0] = (InsideArea0 >= InsideArea1) ? 0 : 1

[0202] isOOB[1] = 1 - isOOB[0]

[0203] The above conditional expression may be "InsideArea0>InsideArea1".

[0204] According to the above configuration, the OOB judgment in the reference picture with the larger effective area can be disabled (flag isOOB[K]=false) and the OOB judgment can be omitted (OOB processing is not applied), so that the amount of calculation in generating a predicted image can be reduced. In particular, when the effective area is large, there is almost no degradation in image quality even if the OOB judgment is disabled.

[0205] <Example 2 of OOB determination omission configuration>

[0206] If the sizes of the valid areas InsideArea for the two reference images are equal, the OOB determination unit 30921 may invalidate the OOB determination for both. Specifically, isOOB[X] may be derived to be a value (false) indicating omission of the OOB determination.

[0207] if (InsideArea0 > InsideArea1) { isOOB[0] = false, isOOB[1] = true } else if (InsideArea0 < InsideArea1) { isOOB[0] = true, isOOB[1] = false } else { isOOB[0] = false, isOOB[1] = false }

[0208] According to the above configuration, it is possible to reduce the amount of calculation in the prediction image generation.

[0209] <Example 3 of OOB determination omission configuration>

[0210] The OOB determination unit 30921 may derive the OOB determination presence / absence flag isOOB[X] using the magnitude relationship of InsideWidthX and InsideHeightX representing the vertical and horizontal sizes of the valid range of the LX reference area (for example, the common part of the LX reference block and the picture). For example, when both the width and height of the valid area of a certain LX reference block are larger than the width and height of the other LY reference block (Y = 1 - X), the OOB determination for the LX reference block may be invalidated (isOOB = false).

[0211] if (InsideWidth0 > InsideWidth1 && InsideHeight0 > InsideHeight1) { isOOB[0] = false, isOOB[1] = true } else { isOOB[0] = true, isOOB[1] = false }

[0212] Or the following:

[0213] if (InsideWidth0 >= InsideWidth1 && InsideHeight0 > InsideHeight1) { isOOB[0] = false, isOOB[1] = true } else { isOOB[0] = true, isOOB[1] = false }

[0214] Or the following would be fine:

[0215] if (InsideWidth0 > InsideWidth1 && InsideHeight0 >= InsideHeight1) { isOOB[0] = false, isOOB[1] = true } else { isOOB[0] = true, isOOB[1] = false }

[0216] Alternatively, if both the width and height of the effective area of ​​an LX reference block are larger than the width and height of the other LY reference block (Y=1-X), the OOB judgment of the LX reference block may be invalidated (isOOB=false), and if the width or height of the two effective areas are equal, both OOB judgments may be invalidated.

[0217] if (InsideWidth0 > InsideWidth1 && InsideHeight0 > InsideHeight1) { isOOB[0] = false, isOOB[1] = true } else if (InsideWidth0 < InsideWidth1 && InsideHeight0 < InsideHeight1) { isOOB[0] = true, isOOB[1] = false } else { isOOB[0] = false, isOOB[1] = false }

[0218] Alternatively, the OOB determination may be performed using only the width or height.

[0219] if (InsideWidth0 > InsideWidth1) { isOOB[0] = false, isOOB[1] = true } else { isOOB[0] = true, isOOB[1] = false

[0220] Or, the following would also work:

[0221] if (InsideHeight0 > InsideHeight1) { isOOB[0] = false, isOOB[1] = true } else { isOOB[0] = true, isOOB[1] = false

[0222] According to the above configuration, since the OOB determination for the reference block with a large effective area (or both reference blocks) can be omitted, the amount of calculations required for generating a predicted image can be reduced. Furthermore, it is no longer necessary to derive the area, and the amount of calculations required for multiplication can be reduced.

[0223] <Summary>

[0224] In summary, the effective area of ​​the L0 reference area may be compared with the effective area of ​​the L1 reference area, and the OOB determination of the reference picture with the larger reference area may be invalidated. Also, the determination of whether the reference area is large may be made by comparing the areas of the effective areas of L0 and L1. Also, the determination of whether the reference area is large may be made by comparing the width or height of the effective areas of L0 and L1.

[0225] (OOB mask derivation process)

[0226] The OOB mask derivation unit 30922 derives an OOB mask (OOBMask[X]) for the reference block. OOBMask[X] is a binary image (mask) of bWxbH pixels, and the value of OOBMask[X][px][py] (px=0..bW-1, py=0..bH-1) indicates that the pixel (px, py) is inside the picture if it is 0 (false), and outside the picture if it is 1 (true). Note that false and true are not limited to 0 and 1, and may be other values, such as bit mask values ​​"0000000000b" and "1111111111b" consisting of a binary string of 0 or 1 with a length equal to or greater than BitDepth. Hexadecimal values ​​"000", "3FF", "000", "FFF", "0000", and "FFFF" may also be used. The OOB mask derivation unit 30922 sets all values ​​of OOBMask[X] to false for blocks that the OOB determination unit 30921 determines are not targets for OOB processing (isOOB[X] == false). In addition, for blocks that the OOB determination unit 30921 determines are targets for OOB processing (isOOB[X] == true), the OOB mask derivation unit 30922 uses the following formula to derive a mask. Note that the inequality sign (">=" or "<=") does not have to include the equality sign (">" or "<").

[0227] OOBMask[X][px][py] = ((xRefLX + px < picMinX) || (yRefLX + py < picMinY) || (xRefLX + px >= picMaxX) || (yRefLX + py >= picMaxY)) ? 1 : 0

[0228] where px = 0..bW-1, py = 0..bH-1. In the above, the OOBMask is derived based on the isOOB[X] formula in (DecideOOB1). In addition, the OOBMask may be derived based on the isOOB[X] formula in (DecideOOB2) and (DecideOOB3). Furthermore, the OOBMask may be derived based on the isOOB[X] formula with valid range restrictions added to (DecideOOB1), (DecideOOB2), and (DecideOOB3).

[0229] The OOB mask derivation unit 30922 may derive OOBMask[X] as a mask in units of NxM pixels. That is, one mask value corresponds to the OOB determination of NxM pixels. In this case, the size of OOBMask[X] is (bW / N)*(bH / M). Each mask value OOBMask[X][x][y] (x=0..bW / N-1, y=0..bH / M-1) is calculated, for example, as follows:

[0230] OOBMask[X][px][py] = ((xRefLX + px*N + N / 2 <= picMinX) || (yRefLX + py*M + M / 2 <= picMinY) || (xRefLX + px*N + N / 2 >= picMaxX) || (yRefLX + py*M + M / 2 >= picMaxY)) ? 1 : 0

[0231] Here, px = 0..bW / N-1, py = 0..bH / M-1. The terms N / 2 and M / 2 may be omitted. In the above, OOBMask is derived based on the equation for isOOB[X] in (DecideOOB1). In addition, OOBMask may be derived based on the equations for isOOB[X] in (DecideOOB2) and (DecideOOB3). Furthermore, OOBMask may be derived based on the equations for isOOB[X] with valid range restrictions added to (DecideOOB1), (DecideOOB2), and (DecideOOB3).

[0232] This allows the mask to have the same value for every NxM pixels, reducing the memory required for the mask as well as the amount of calculation.

[0233] (OOB predicted image generation)

[0234] In the case of bi-prediction, the motion compensation unit 3091 (interpolated image generation unit 3091) sets Pred[][] derived by the above (Equation MC-1) as the interpolated images PredL0[][] and PredL1[][] for each of the L0 list and the L1 list. Then, an interpolated image Pred[][] is generated from PredL0[][] and PredL1[][]. When a bi-prediction reference block is a target of OOB processing, an OOBMask[X] corresponding to the reference block is used to generate a predicted image. In this case, Pred[][] is derived as follows.

[0235] if (OOBMask[0][px][py] && !OOBMask[1][px][py]) { (formula WP-1) Pred[px][py] = (PredL1[px][py] + offset1)>> shift1 } else if (!OOBMask[0][px][py] && OOBMask[1][px][py]) { Pred[px][py] = (PredL0[px][py] + offset1)>> shift1 } else { Pred[px][py] = (PredL0[px][py] + PredL1[px][py] + offset2)>> shift2 }

[0236] Here, px = 0..bW-1, py = 0..bH-1. Also, shift1 = Max(2,14-bitDepth), shift2 = Max(3,15-bitDepth), offset1 = 1<<(shift1-1), offset2 = 1<<(shift2-1). Note that shift1 is a shift (e.g. 14-bitDepth) to return the interpolated image bit depth (e.g. 14-bit) to the original bit depth. shift2 is the right shift amount that returns to the original bit and also serves as the average, and offset1 and offset2 are rounding values ​​when shifting right by shift1 and shift2.

[0237] The above processing may be switched depending on whether CU-based explicit weighted bi-prediction (BCW) or combined inter-picture merge and intra-picture prediction (CIIP) is used. bcwIndex is an index indicating the weight of BCW. ciip_flag is a flag indicating whether CIIP is on or off. When bcwIdx==0 or the CIIP flag (ciip_flag) is 1, the above processing (Equation WP-1) is performed. In other cases, the following processing is performed.

[0238] w1 = bcwWLut[bcwIdx]

[0239] w0 = (8 - w1)

[0240] where bcwWLut[k] = {4, 5, 3, 10, -2}

[0241] if (OOBMask[0][px][py] && !OOBMask[1][px][py]) { Pred[px][py] = (PredL1[px][py] + offset1)>> shift1 } else if (!OOBMask[0][px][py] && OOBMask[1][px][py]) { Pred[px][py] = (PredL0[px][py] + offset1)>> shift1 } else {

[0242] Pred[px][py] = ( w0 * PredL0[px][py] + w1 * PredL1[px][py] + offset3 )>>(shift1+3)) }

[0243] Here, offset3=1<<(shift1+2).

[0244] The amount of processing can be reduced by combining the flag predFlagLX, which indicates whether or not to use the LX reference picture on a block-by-block basis, and the processing by OOBMask into one branch and processing as follows.

[0245] if ((predFlagL0 == 1 && predFlagL1 ==0) || (!OOBMask[0][px][py] && OOBMask[1][px][py])) { (formula WP-2) Pred[px][py] = (PredL0[px][py]*we0 + offset1)>> shift1 } else if ((predFlagL0 == 0 && predFlagL1 ==1) || OOBMask[0][px][py] && !OOBMask[1][px][py]) { Pred[px][py] = (PredL1[px][py]*we1 + offset1)>> shift1 } else if ((bcwIdx ==0 || ciip_flag == 1) { Pred[px][py] = (w0 * PredL0[px][py] + w1 * PredL1[px][py] + offset2)>>shift2 } else { / / bcwIdx !=0 and ciip_flag == 0) Pred[px][py] = (we0 * PredL0[px][py] + we1 * PredL1[px][py] + offset3)>>(shift1+3)) }

[0246] In weighted prediction that generates a predicted image by the weighted sum of reference blocks on two reference pictures, the OOB determination unit 30921 does not need to apply OOB processing to the block. That is, for a block to which weighted prediction is applied, the OOB determination flag isOOB[X] is set to false. This makes it possible to prevent distortion or a decrease in coding efficiency caused by not using one of the reference blocks to which bidirectional weighted prediction is applied. Such weighted prediction includes BCW (Bi-prediction with CU-level Weight) prediction and WP (Weighted prediction) prediction.

[0247] In BCW prediction, weighting coefficients (gbw0, gbw1) are selected for each block, and a predicted image is generated by multiplying the L0 interpolated image and the L1 interpolated image in bi-prediction by the weighting coefficient. The parameter decoding unit 302 decodes a parameter bcw_idx indicating a weighting coefficient in a coding unit CU from the coded data. The parameter coding unit 111 codes bcw_idx. In the case of merge prediction, the bcwIdx of the reference block (adjacent block) is set to the bcwIdx of the current block. In other cases, the bcwIdx of the current block is set to bcw_idx. If bcwIdx!=0, the BCW prediction is applied to the CU. If the OOB determination unit 30921 applies BCW prediction (bcwIdx!=0), it sets isOOB[X] to false.

[0248] In the WP prediction, the weighting factor and offset for each reference picture are switched in slice units, a weighting factor and an offset are selected based on the reference picture index in CU units, and a predicted image is generated by multiplying the L0 interpolated image and the L1 interpolated image by the weight. The parameter decoding unit 302 decodes pps_weighted_bipred_flag, which indicates whether or not to apply WP prediction to a B picture, from the encoded data of the PPS. When pps_weighted_bipred_flag==1, the WP prediction is applied to the B picture of the PPS. When applying explicit WP prediction (pps_weighted_bipred_flag==1), the OOB determination unit 30921 sets isOOB[X] to false. Note that when two reference pictures are not used (when any of predFlagLX(X=0,1) is 0), the OOB process may be applied.

[0249] <Omitted configuration>

[0250] When using the above-described omitted configuration, both OOBMask[0] and OOBMask[1] will never be 1, so the motion compensation unit 3091 (interpolated image generation unit 3091) can derive an interpolated image by the following branching process.

[0251] if (OOBMask[0][px][py]) { Pred[px][py] = (PredL1[px][py] + offset1)>> shift1 } else if (OOBMask[1][px][py]) { Pred[px][py] = (PredL0[px][py] + offset1)>> shift1 } else { Pred[px][py] = (PredL0[px][py] + PredL1[px][py] + offset2)>> shift2 }

[0252] It is also appropriate to calculate PredL0, PredL1, and PredBI in advance and then derive the final Pred by the following mask processing.

[0253] First, PredL0, PredL1, and PredBI are calculated.

[0254] PredL0[px][py] = (PredL0[px][py] + offset1) >> shift1

[0255] PredL1[px][py] = (PredL1[px][py] + offset1) >> shift1

[0256] PredBI[px][py] = (PredL0[px][py] + PredL1[px][py] + offset2) >> shift2

[0257] Next, if bit masks of binary "1111111111b" and "0000000000b" with a BitDepth length are used as the values ​​of OOBMask[0][px][py] and OOBMask[1][px][py], the final Pred is derived using the following process.

[0258] Pred[px][py]= (PredBI[px][py] & (!OOBMask[0][px][py] & !OOBMask[1][px][py])) + (PredL0[px][py] & OOBMask[0][px][py]) + (PredL1[px][py] & OOBMask[1][px][py])

[0259] In addition, if the values ​​are true=1 and false=0, the bit mask values ​​"0000000000b" and "1111111111b" can be generated from 0 and 1 and the following processing can be performed. Here, "+" can be replaced with "|" (the same applies to the following mask calculation processing).

[0260] Pred[px][py]= (formula excl-1) (PredBI[px][py] & (0-((!OOBMask[0][px][py] & !OOBMask[1][px][py]) ? 1 : 0)))+ (PredL0[px][py] & (0-(OOBMask[0][px][py] ? 1 : 0))) + (PredL1[px][py] & (0-(OOBMask[1][px][py] ? 1 : 0)))

[0261] The above formula includes a calculation to convert a binary (0 or 1) mask to 0 (all bits are 0, pixel value is invalid) or -1 (all bits are 1, pixel value is valid) in order to create a mask with a bit width equal to or greater than the bit depth of the pixel value. The derivation of the predicted image is not limited to this, and any calculation can be used in which the term that is invalid due to the OOB mask among the three terms corresponding to PredBI, PredL0, and PredL1 becomes 0. For example, when the bit depth of the pixel value is a maximum of 16 bits, the following formula may be used.

[0262] Pred[px][py]= (PredBI[px][py] & ((!OOBMask[0][px][py] & !OOBMask[1][px][py])? 0xFFFF : 0))+ (PredL0[px][py] & (OOBMask[0][px][py] ? 0xFFFF : 0)) + (PredL1[px][py] & (OOBMask[1][px][py] ? 0xFFFF : 0))

[0263] Instead of using addition to combine the three terms, we can use bitwise OR as shown below.

[0264] Pred[px][py]= (PredBI[px][py] & ((!OOBMask[0][px][py] & !OOBMask[1][px][py])? 0xFFFF : 0)) | (PredL0[px][py] & (OOBMask[0][px][py] ? 0xFFFF : 0)) | (PredL1[px][py] & (OOBMask[1][px][py] ? 0xFFFF : 0))

[0265] Alternatively, multiplication may be used.

[0266] Pred[px][py]= (PredBI[px][py] * ((!OOBMask[0][px][py] & !OOBMask[1][px][py])? 1 : 0)) + (PredL0[px][py] * (OOBMask[0][px][py] ? 1 : 0)) + (PredL1[px][py] * (OOBMask[1][px][py] ? 1 : 0))

[0267] According to the above, the exclusive processing can be simplified. The exclusive processing can be simplified more than the calculation of (expression excl-1).

[0268] The motion compensation unit 3091 (interpolated image generation unit 3091) may refer to the value of isOOB and divide the processing in advance. For example, it may be as follows. In this way, when isOOB[X] is false, there is no need to refer to OOBMask[X][px][py] for each pixel, and the amount of calculation can be reduced.

[0269] if (isOOB[0] && !isOOB[1]) { if (OOBMask[0][px][py]) { Pred[px][py] = (PredL1[px][py] + offset1)>> shift2 } else { Pred[px][py] = (PredL0[px][py] + PredL1[px][py] + offset2)>> shift }} else if (!isOOB[0] && isOOB[1]) { if (OOBMask[1][px][py]) { Pred[px][py] = (PredL0[px][py] + offset1)>> shift2 } else { Pred[px][py] = (PredL0[px][py] + PredL1[px][py] + offset2)>> shift } } else if (isOOB[0] && isOOB[1]) { if (OOBMask[0][px][py] && !OOBMask[1][px][py]) { Pred[px][py] = (PredL1[px][py] + offset1)>> shift2 } else if (!OOBMask[0][px][py] && OOBMask[1][px][py]) { Pred[px][py] = (PredL0[px][py] + offset1)>> shift2 } else { Pred[px][py] = (PredL0[px][py] + PredL1[px][py] + offset2)>> shift } } else { Pred[px][py] = (PredL0[px][py] + PredL1[px][py] + offset2)>> shift }

[0270] (Weighted prediction)

[0271] The weighted prediction unit 3094 generates a predicted image of the block by multiplying the interpolated image PredLX by a weighting coefficient.

[0272] The inter-prediction image generation unit 309 outputs the generated prediction image of the block to the addition unit 312 .

[0273] (Configuration of a video encoding device)

[0274] Next, the configuration of the video encoding device 11 according to this embodiment will be described. Fig. 7 is a block diagram showing the configuration of the video encoding device 11 according to this embodiment. The video encoding device 11 includes a prediction image generating unit 101, a subtraction unit 102, a transformation and quantization unit 103, an inverse quantization and inverse transformation unit 105, an addition unit 106, a loop filter 107, a prediction parameter memory (prediction parameter storage unit, frame memory) 108, a reference picture memory (reference image storage unit, frame memory) 109, an encoding parameter determining unit 110, a parameter encoding unit 111, a prediction parameter derivation unit 120, and an entropy encoding unit 104.

[0275] The predicted image generation unit 101 generates a predicted image for each CU. The predicted image generation unit 101 includes the inter predicted image generation unit 309 and the intra predicted image generation unit already described, and therefore a description thereof will be omitted.

[0276] The subtraction unit 102 generates a prediction error by subtracting the pixel values ​​of the predicted image of the block input from the predicted image generation unit 101 from the pixel values ​​of the image T. The subtraction unit 102 outputs the prediction error to the transformation and quantization unit 103.

[0277] The transform / quantization unit 103 calculates transform coefficients by frequency transforming the prediction errors input from the subtraction unit 102, and derives quantized transform coefficients by quantizing the prediction errors. The transform / quantization unit 103 outputs the quantized transform coefficients to the parameter coding unit 111 and the inverse quantization / inverse transform unit 105.

[0278] The transform / quantization unit 103 includes a separation transform unit (first transform unit), a non-separation transform unit (second transform unit), and a scaling unit.

[0279] The separate transform unit applies a separate transform to the prediction error, and the scaling unit scales the transform coefficients with a quantization matrix.

[0280] The inverse quantization and inverse transform unit 105 is the same as the inverse quantization and inverse transform unit 311 in the video decoding device 31, and a description thereof will be omitted.

[0281] The parameter coding unit 111 includes a header coding unit 1110, a CT information coding unit 1111, and a CU coding unit 1112 (prediction mode coding unit). The CU coding unit 1112 further includes a TU coding unit 1114. The following describes an outline of the operation of each module.

[0282] The header encoding unit 1110 performs encoding processing of parameters such as header information, division information, prediction information, and quantized transform coefficients.

[0283] The CT information encoding unit 1111 encodes the QT, MT (BT, TT) division information and the like.

[0284] The CU encoding unit 1112 encodes the CU information, prediction information, division information, and so on.

[0285] When a prediction error is included in a TU, the TU encoding unit 1114 encodes the QP update information and the quantized prediction error.

[0286] The CT information encoding unit 1111 and the CU encoding unit 1112 supply syntax elements such as inter prediction parameters (predMode, general_merge_flag, merge_idx, inter_pred_idc, refIdxLX, mvp_LX_idx, mvdLX), intra prediction parameters, and quantized transform coefficients to the parameter encoding unit 111.

[0287] The entropy coding unit 104 receives the quantized transform coefficients and coding parameters (division information, prediction parameters) from the parameter coding unit 111. The entropy coding unit 104 entropy codes these to generate and output a coded stream Te.

[0288] The prediction parameter derivation unit 120 is a means including the inter prediction parameter encoding unit 112 and an intra prediction parameter encoding unit, and derives intra prediction parameters and inter prediction parameters from the parameters input from the encoding parameter determination unit 110. The derived intra prediction parameters and inter prediction parameters are output to the parameter encoding unit 111.

[0289] (Configuration of the inter-prediction parameter encoding unit)

[0290] The inter prediction parameter encoding unit 112 includes a parameter encoding control unit 1121 and an inter prediction parameter derivation unit 303. The inter prediction parameter derivation unit 303 has the same configuration as the video decoding device. The parameter encoding control unit 1121 includes a merge index derivation unit 11211 and a vector candidate index derivation unit 11212.

[0291] The merge index derivation unit 11211 derives merge candidates and the like, and outputs them to the inter prediction parameter derivation unit 303. The vector candidate index derivation unit 11212 derives predictive vector candidates and the like, and outputs them to the inter prediction parameter derivation unit 303 and the parameter coding unit 111.

[0292] (Configuration of intra-prediction parameter encoding unit)

[0293] The intra-prediction parameter coding unit includes a parameter coding control unit and an intra-prediction parameter derivation unit. The intra-prediction parameter derivation unit has a common configuration with the video decoding device.

[0294] However, unlike the video decoding device, the inputs to the inter prediction parameter derivation unit 303 and the intra prediction parameter derivation unit are the coding parameter determination unit 110 and the prediction parameter memory 108 , and they are output to the parameter coding unit 111 .

[0295] The adder 106 generates a decoded image by adding, for each pixel, the pixel value of the predicted block input from the predicted image generation unit 101 and the prediction error input from the inverse quantization and inverse transform unit 105. The adder 106 stores the generated decoded image in a reference picture memory 109.

[0296] The loop filter 107 performs deblocking filtering, SAO, and ALF on the decoded image generated by the adder 106. Note that the loop filter 107 does not necessarily have to include the above three types of filters, and may be configured, for example, as only a deblocking filter.

[0297] The prediction parameter memory 108 stores the prediction parameters generated by the coding parameter determination unit 110 in a predetermined location for each current picture and CU.

[0298] The reference picture memory 109 stores the decoded image generated by the loop filter 107 at a predetermined position for each current picture and CU.

[0299] The coding parameter determination unit 110 selects one set from among a plurality of sets of coding parameters. The coding parameters are the above-mentioned QT, BT or TT division information, prediction parameters, or parameters to be coded that are generated in relation to these. The predicted image generation unit 101 generates a predicted image using these coding parameters.

[0300] The coding parameter determination unit 110 calculates an RD cost value indicating the magnitude of the information amount and the coding error for each of the multiple sets. The RD cost value is, for example, the sum of the code amount and the value obtained by multiplying the squared error by a coefficient λ. The code amount is the information amount of the coding stream Te obtained by entropy coding the quantization error and the coding parameters. The squared error is the sum of the squares of the prediction errors calculated in the subtraction unit 102. The coefficient λ is a real number greater than zero that is set in advance. The coding parameter determination unit 110 selects a set of coding parameters that minimizes the calculated cost value. The coding parameter determination unit 110 outputs the determined coding parameters to the parameter coding unit 111 and the prediction parameter derivation unit 120.

[0301] In addition, a part of the video encoding device 11 and the video decoding device 31 in the above-mentioned embodiment, for example, the entropy decoding unit 301, the parameter decoding unit 302, the loop filter 305, the predicted image generating unit 308, the inverse quantization and inverse transform unit 311, the addition unit 312, the prediction parameter derivation unit 320, the predicted image generating unit 101, the subtraction unit 102, the transform and quantization unit 103, the entropy coding unit 104, the inverse quantization and inverse transform unit 105, the loop filter 107, the coding parameter determination unit 110, the parameter coding unit 111, and the prediction parameter derivation unit 120 may be realized by a computer. In this case, a program for realizing this control function may be recorded in a computer-readable recording medium, and the program recorded in the recording medium may be read into and executed by a computer system. In addition, the "computer system" referred to here is a computer system built into either the video encoding device 11 or the video decoding device 31, and includes hardware such as an OS and peripheral devices. In addition, "computer-readable recording medium" refers to portable media such as flexible disks, optical magnetic disks, ROMs, and CD-ROMs, and storage devices such as hard disks built into computer systems. Furthermore, "computer-readable recording medium" may also include devices that dynamically hold a program for a short period of time, such as a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line, and devices that hold a program for a certain period of time, such as volatile memory inside a computer system that serves as a server or client in such cases. Furthermore, the above program may be one that realizes part of the above-mentioned functions, or may be one that can realize the above-mentioned functions in combination with a program already recorded in the computer system.

[0302] In addition, a part or the whole of the video encoding device 11 and the video decoding device 31 in the above-mentioned embodiment may be realized as an integrated circuit such as an LSI (Large Scale Integration). Each functional block of the video encoding device 11 and the video decoding device 31 may be individually made into a processor, or a part or the whole may be integrated into a processor. The integrated circuit method is not limited to LSI, and may be realized by a dedicated circuit or a general-purpose processor. Furthermore, when an integrated circuit technology that replaces LSI appears due to the progress of semiconductor technology, an integrated circuit based on that technology may be used.

[0303] Although one embodiment of the present invention has been described in detail above with reference to the drawings, the specific configuration is not limited to the above, and various design changes, etc. are possible within the scope that does not deviate from the gist of the present invention.

[0304] The present invention is not limited to the above-described embodiment, and various modifications are possible within the scope of the claims. In other words, the technical scope of the present invention also includes embodiments obtained by combining technical means that are appropriately modified within the scope of the claims. [Industrial Applicability]

[0305] The embodiments of the present invention can be suitably applied to a video decoding device that decodes coded data in which image data is coded, and a video coding device that generates coded data in which image data is coded, and can also be suitably applied to the data structure of coded data that is generated by a video coding device and referenced by the video decoding device. [Explanation of symbols]

[0306] 31 Video Decoding Device 301 Entropy Decoding Unit 302 Parameter Decoding Unit 3022 CU Decoding Unit 3024 TU Decoding Unit 303 Inter-prediction parameter derivation unit 305, 107 Loop Filter 306, 109 Reference Picture Memory 307, 108 Prediction parameter memory 308, 101 Prediction image generation unit 309 Inter-prediction image generation unit 3092 OOB Processing Unit 30921 OOB judgment section 30922 OOB mask lead-out part 311, 105 Inverse quantization and inverse transformation unit 312, 106 Addition section 320 Prediction Parameter Derivation Unit 11 Video Encoding Device 102 Subtraction section 103 Transformation and Quantization Section 104 Entropy coding unit 110 Encoding parameter determination unit 111 Parameter Encoding Unit 112 Inter-prediction parameter coding unit 120 Prediction parameter derivation part

Claims

1. A video decoding device comprising: an OOB determination unit that, in bi-prediction, determines whether a reference block is subject to OOB processing by comparing coordinates of a reference block boundary with coordinates of a picture boundary; and an OOB mask derivation unit that derives mask data indicating whether each pixel is available for use by comparing coordinates of pixels included in the reference block with coordinates of the picture boundary or region boundary, the OOB determination unit determines whether the reference block includes an area outside a valid range based on the upper left coordinate of the reference block derived from the motion vector of the target block and the block size; the OOB mask derivation unit derives mask data indicating whether a pixel at each position of the reference block is outside the valid range based on the determination, In the case of bi-prediction, a video decoding device generates a predicted image based on the mask data.

2. 2. The video decoding device according to claim 1, wherein the OOB determination unit compares the valid area of ​​the L0 reference area with the valid area of ​​the L1 reference area, and disables the OOB determination for a reference picture whose valid area of ​​the reference area is larger.

3. 2. The video decoding device according to claim 1, wherein the OOB determination unit determines the size of the effective area by comparing the areas of the effective areas of the L0 reference area and the L1 reference area.

4. 2. The video decoding device according to claim 1, wherein the OOB determination unit determines the size of the effective area by comparing the width or height of the effective area of ​​the L0 reference area with that of the L1 reference area.

5. The video decoding device according to claim 1, characterized in that the OOB determination unit disables OOB determination for blocks to which block-based weighted bi-prediction (BCW: Bi-prediction with CU-level Weight) is applied.

6. 2. The video decoding device according to claim 1, wherein the OOB mask derivation unit derives the mask in units of NxM pixels (N and M are positive integers).

7. 2. The video decoding device according to claim 1, wherein the OOB determination unit invalidates the OOB determination when the target block is equal to or smaller than a predetermined size.

8. A video encoding device comprising: an OOB determination unit that determines whether a reference block is subject to OOB processing by comparing coordinates of a reference block boundary with coordinates of a picture boundary in bi-prediction; and an OOB mask derivation unit that derives mask data indicating whether each pixel is available for use by comparing coordinates of pixels included in the reference block with coordinates of the picture boundary, the OOB determination unit determines whether the reference block includes an area outside a valid range based on the upper left coordinate of the reference block derived from the motion vector of the target block and the block size; the OOB mask derivation unit derives mask data indicating whether a pixel at each position of the reference block is outside the valid range based on the determination, In the case of bi-prediction, a video encoding device generates a predicted image based on the mask data.

Images