Moving image decoding device and moving image encoding device

Abstract

Provided is a moving image encoding / decoding device capable of improving encoding efficiency. A moving picture decoding device is provided with: an OOB determination unit that determines whether or not a reference block is an object of OOB processing by comparing coordinates of the reference block with coordinates of a picture; and an OOB mask derivation unit that derives mask data indicating whether or not each pixel is available by comparing coordinates of pixels included in a reference block with coordinates of a picture, the OOB determination unit being characterized in that whether or not the reference block is an OOB processing target is determined by detecting whether or not surround processing is applied to a reference picture including the reference block, and the OOB mask derivation unit derives mask data indicating whether or not each pixel is available by comparing the coordinates of the pixels included in the reference block with the coordinates of the picture. The OOB determination unit is characterized by detecting whether or not a reference picture resampling process is applied to a reference picture including the reference block, and determining whether or not the reference block is an OOB processing target.

Description

Technical Field

[0001] Embodiments of the present invention relate to a moving picture decoding device and a moving picture encoding device. Background Art

[0002] In order to efficiently transmit or record moving images, a moving image encoding device that generates encoded data by encoding the moving image and a moving image decoding device that generates a decoded image by decoding the encoded data are used.

[0003] Specific moving picture coding methods include, for example, H.264 / AVC, HEVC (High-Efficiency Video Coding), and Versatile Video Coding (VVC).

[0004] In such a motion image coding method, images (pictures) constituting the motion image are managed through a hierarchical structure and encoded / decoded for each CU. The hierarchical structure includes slices obtained by dividing the image, coding tree units (CTU: Coding Tree Unit) obtained by dividing the slices, coding units (sometimes also called coding units (Coding Unit: CU)) obtained by dividing the coding tree units, and transform units (TU: Transform Unit) obtained by dividing the coding units.

[0005] In such a motion picture coding method, a prediction image is usually generated based on a local decoded image obtained by encoding / decoding an input image, and a prediction error (sometimes also referred to as a "difference image" or "residual image") obtained by subtracting the prediction image from the input image (original image) is encoded. As a method for generating a prediction image, inter-picture prediction (inter-frame prediction) and intra-picture prediction (intra-frame prediction) can be listed. In VVC, as Figure 8 As shown in (a), the coordinates of the motion vector can be wrapped around so that the horizontal coordinate system can be continuously motion compensated at the left and right ends of the picture. Therefore, when wrapping is applied to images such as 360-degree panoramic images and 360° images where the left and right ends of the picture are spatially continuous, the coding efficiency can be improved. In addition, in VVC, Figure 8 As shown in (b), a reference picture resampling (RPR) process can be performed to change the resolution in units of pictures to perform motion compensation. When the RPR process is applied to a service with a variable transmission rate such as video distribution on the Internet, the image quality can be improved.

[0006] In addition, non-patent document 1 discloses an OOB (Out-Of-Boundary) processing technology that replaces part or all of the area in one reference block of a bidirectional prediction that exceeds the picture of the reference image with part or all of the area in the other reference block.

[0007] Prior art literature

[0008] Non-patent literature

[0009] Non-patent document 1: “AHG12: Enhanced bi-directional motion compensation”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 25th Meeting, by teleconference, JVET-Y0125 Summary of the invention

[0010] Problem that the invention aims to solve

[0011] In non-patent document 1, there is no OOB determination process corresponding to the wraparound process, and there is a technical problem that the wraparound process and the OOB process cannot be applied at the same time. Figure 8 When the reference block is outside the left end of the picture, the area on the right side of the reference picture is used. Therefore, the coordinate system in the horizontal direction will not be outside the picture, and there is a problem that the coding efficiency may be reduced by applying the OOB process. In addition, in non-patent document 1, there is no OOB determination process corresponding to the RPR process, and there is a technical problem that the RPR process and the OOB process cannot be applied at the same time. In the RPR process, the reference picture whose size is different from that of the target picture is reduced to a prescribed scale and maintained in the reference picture buffer. As a result, there is the following problem: the coordinate system of the reference picture is different from the coordinate system of the target picture, and when the RPR process and the OOB process are used together, the correct OOB determination process cannot be performed.

[0012] Technical Solution

[0013] In order to solve the above technical problems, a moving picture decoding device according to one embodiment of the present invention comprises: an OOB determination unit for determining whether a reference block is an object of OOB processing by comparing the coordinates of the reference block with the coordinates of a picture; and an OOB mask derivation unit for deriving mask data indicating whether each pixel can be used by comparing the coordinates of pixels included in the reference block with the coordinates of the picture.

[0014] The OOB determination unit is characterized by detecting whether a surround process is applied to a reference picture including the reference block to determine whether the reference block is an OOB process target.

[0015] The OOB determination unit is characterized by detecting whether a reference picture resampling process is applied to a reference picture including the reference block, to determine whether the reference block is an OOB processing target.

[0016] Beneficial Effects

[0017] According to the aspect of the present invention, the amount of calculation of OOB processing can be reduced in moving picture encoding / decoding processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 This is a schematic diagram showing the configuration of the image transmission system according to the present embodiment.

[0019] Figure 2 This is a diagram showing the hierarchical structure of the data of the coded stream.

[0020] Figure 3 This is a schematic diagram showing the structure of a moving picture decoding device.

[0021] Figure 4 This is a schematic diagram showing the structure of an inter-frame prediction image generation unit.

[0022] Figure 5 This is a schematic diagram showing the structure of an inter-frame prediction parameter derivation unit.

[0023] Figure 6 This is a flowchart for explaining the schematic operation of the moving picture decoding device.

[0024] Figure 7 This is a block diagram showing the structure of a moving picture encoding device.

[0025] Figure 8 FIG. 1 is a diagram showing an example of conventional wrap-around processing and reference picture resampling processing.

[0026] Fig. 9 This is a flowchart for explaining the schematic operation of OOB processing.

[0027] Fig.10 This is a schematic diagram showing an example of OOB determination processing. DETAILED DESCRIPTION

[0028] (First Embodiment)

[0029] Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[0030] Figure 1It is a schematic diagram showing the configuration of the image transmission system 1 according to the present embodiment.

[0031] The image transmission system 1 is a system for transmitting a coded stream obtained by encoding a target image, decoding the transmitted coded stream and displaying the image. The image transmission system 1 is configured to include: a moving image encoding device (image encoding device) 11, a network 21, a moving image decoding device (image decoding device) 31, and a moving image display device (image display device) 41.

[0032] The video image encoding device 11 receives the image T as input.

[0033] The network 21 transmits the coded stream Te generated by the motion image coding device 11 to the motion image decoding device 31. The network 21 is the Internet, a wide area network (WAN), a small network (LAN) or a combination thereof. The network 21 is not necessarily limited to a two-way communication network, but may also be a one-way communication network that transmits broadcast waves such as terrestrial digital broadcasting and satellite broadcasting. In addition, the network 21 may also be replaced by a storage medium such as a DVD (Digital Versatile Disc: Digital Versatile Disc, registered trademark) or a BD (Blue-ray Disc: Blue-ray Disc, registered trademark) that records the coded stream Te.

[0034] The moving picture decoding device 31 decodes each of the coded streams Te transmitted through the network 21 , and generates one or more decoded pictures Td after decoding.

[0035] The moving image display device 41 displays all or part of one or more decoded images Td generated by the moving image decoding device 31. The moving image display device 41 includes, for example, a liquid crystal display, an organic EL (Electro-luminescence) display, and other display devices. Examples of the display type include a fixed type, a mobile type, and an HMD (Head Mounted Display). In addition, when the moving image decoding device 31 has high processing power, it displays images with high image quality, and when it has only low processing power, it displays images that do not require high processing power or high display power.

[0036] <operator>

[0037] The following describes operators used in this specification.

[0038] >> is a right shift, << is a left shift, & is a bitwise AND, | is a bitwise OR, |= is an OR substitution operator, and || represents a logical AND.

[0039] x? y : z is a ternary operator that takes y when x is true (other than 0) and takes z when x is false (0).

[0040] Clip3(a, b, c) is a function that clips c to a value between a and b (inclusive), returning a if c < a, b if c > b, and c otherwise (where a <= b).

[0041] ClipH(o, W, x) is a function that returns x if x < 0, x - o if x > W - 1, and x otherwise.

[0042] sign(a) is a function that returns 1 if a > 0, 1 if a == 0, and -1 if a < 0.

[0043] abs(a) is a function that returns the absolute value of a.

[0044] Int(a) is a function that returns the integer value of a.

[0045] floor(a) is a function that returns the largest integer less than or equal to a.

[0046] ceil(a) is a function that returns the smallest integer greater than or equal to a.

[0047] a / d means a divided by d (truncating the fractional part).

[0048] ~ represents bitwise negation (not). It is a function that sets 0 to 1 and 1 to 0 for each bit and returns the result.

[0049] <Structure of the encoded stream Te>

[0050] Before explaining the moving image encoding device 11 and the moving image decoding device 31 of the present embodiment in detail, the data structure of the encoded stream Te generated by the moving image encoding device 11 and decoded by the moving image decoding device 31 will be described.

[0051] Figure 2 It is a diagram showing the hierarchical structure of the data in the encoded stream Te. The encoded stream Te exemplarily includes a sequence and multiple pictures constituting the sequence. Figure 2 The (a) - (f) are diagrams showing the encoded video sequence of a given sequence SEQ, the encoded picture of a specified picture PICT, the encoded slice of a specified slice S, the encoded slice data of the specified slice data, the encoded tree unit included in the encoded slice data, and the encoded unit included in the encoded tree unit, respectively.

[0052] (Encoded video sequence)

[0053] In the coded video sequence, a set of data is defined for reference by the moving image decoding device 31 in order to decode the sequence SEQ to be processed. Figure 2 As shown, the sequence SEQ includes a video parameter set (Video Parameter Set), a sequence parameter set SPS (Sequence Parameter Set), a picture parameter set PPS (Picture Parameter Set), an adaptation parameter set (Adaptation Parameter Set: APS), a picture PICT and supplemental enhancement information SEI (Supplemental Enhancement Information).

[0054] The video parameter set VPS defines a set of coding parameters common to a plurality of moving images, and a set of coding parameters associated with a plurality of layers included in the moving image and each layer, in the moving image.

[0055] In the sequence parameter set SPS, a set of coding parameters is specified for reference by the motion image decoding device 31 to decode the object sequence. For example, the width and height of the picture are specified. It should be noted that there can be multiple SPSs. In this case, any one of the multiple SPSs is selected from the PPS.

[0056] In the picture parameter set PPS, a set of coding parameters is specified for reference by the motion image decoding device 31 to decode each picture in the target sequence. For example, it includes a reference value (pic_init_qp_minus26) of the quantization width used for decoding the picture and a flag (weighted_pred_flag) indicating the application of weighted prediction. It should be noted that there can be multiple PPSs. In this case, any one of the multiple PPSs is selected from each picture in the target sequence.

[0057] (Encoded image)

[0058] In the coded picture, a set of data is defined for reference by the video decoding device 31 in order to decode the picture PICT to be processed. Figure 2 As shown, it includes slice 0 to slice NS-1 (NS is the total number of slices included in the picture PICT).

[0059] It should be noted that, in the following, when there is no need to distinguish between slices 0 to NS-1, the code subscript may be omitted for description. The same is true for data included in the coded stream Te described below, that is, other data with a suffix.

[0060] (Encoding slice)

[0061] In the coded slice, a set of data is defined for reference by the video decoding device 31 to decode the slice S to be processed. Figure 2 The shown includes a slice header and slice data.

[0062] The slice header includes a coding parameter group that the moving picture decoding device 31 refers to in order to determine a decoding method of a target slice. Slice type designation information (slice_type) that designates a slice type is one example of a coding parameter included in the slice header.

[0063] Examples of slice types that can be specified by the slice type specification information include: (1) I slices that use only intra-frame prediction when encoding, (2) P slices that use unidirectional prediction or intra-frame prediction when encoding, and (3) B slices that use unidirectional prediction, bidirectional prediction, or intra-frame prediction when encoding. It should be noted that inter-frame prediction is not limited to unidirectional prediction and bidirectional prediction, and more reference pictures can be used to generate predicted images. Hereinafter, the cases referred to as P and B slices refer to slices that include blocks that can use inter-frame prediction.

[0064] It should be noted that the slice header may also include a reference to the picture parameter set PPS (pic_parameter_set_id).

[0065] (Encoded slice data)

[0066] The coded slice data specifies a set of data that the video decoding device 31 refers to in order to decode the slice data to be processed. Figure 2 (d) includes CTU. CTU is a block of a fixed size (for example, 64×64) constituting a slice, and is also called the largest coding unit (LCU).

[0067] (Coding Tree Unit)

[0068] exist Figure 2In the specification, there is a set of data for the motion picture decoding device 31 to refer to in order to decode the CTU of the processing object. The CTU is divided into coding units CU which are the basic units of the encoding process by recursive quadtree partitioning (QT (Quad Tree) partitioning), binary tree partitioning (BT (Binary Tree) partitioning) or ternary tree partitioning (TT (Ternary Tree) partitioning). BT partitioning and TT partitioning are collectively referred to as multi-tree partitioning (MT (Multi Tree) partitioning). The nodes of the tree structure obtained by recursive quadtree partitioning are called coding nodes (Coding Node). The middle nodes of the quadtree, binary tree and ternary tree are coding nodes, and the CTU itself is also specified as the top-level coding node. In addition, the bottom-level coding node is specified as a coding unit.

[0069] (Coding unit)

[0070] exist Figure 2 In the CU header, a set of data is specified for reference by the video decoding device 31 to decode the coding unit to be processed. Specifically, the CU is composed of a CU header CUH, prediction parameters, transformation parameters, quantized transformation coefficients, etc. The prediction mode, etc. are specified in the CU header.

[0071] The prediction process may be performed in units of CU or in units of sub-CUs obtained by further dividing the CU. When the size of the CU and the sub-CU are equal, there is one sub-CU in the CU. When the size of the CU is larger than the size of the sub-CU, the CU is divided into sub-CUs. For example, when the CU is 8×8 and the sub-CU is 4×4, the CU is divided into 4 sub-CUs, including two parts divided horizontally and two parts divided vertically.

[0072] The prediction types (prediction modes) include intra prediction (MODE_INTRA), inter prediction (MODE_INTER), and intra block copy (MODE_IBC). Intra prediction is prediction within the same picture, while inter prediction is prediction between different pictures (e.g., between display times, between layer images).

[0073] The transform / quantization process is performed in units of CUs, but the quantized transform coefficients may be entropy-coded in units of 4×4 or other sub-blocks.

[0074] (Prediction parameters)

[0075] The predicted image is derived from prediction parameters attached to the block. The prediction parameters include prediction parameters for intra-frame prediction and inter-frame prediction.

[0076] (Prediction parameters for inter-frame prediction)

[0077] The prediction parameters of inter-frame prediction are explained. The inter-frame prediction parameters are composed of the prediction list utilization flags predFlagL0 and predFlagL1, the reference picture indexes refIdxL0 and refIdxL1, and the motion vectors mvL0 and mvL1. predFlagL0 and predFlagL1 are flags indicating whether to use the reference picture list (L0 list, L1 list). When the value is 1, the corresponding reference picture list is used. It should be noted that in the case of "flag indicating whether it is ××" in this specification, the flag is set to be ×× when it is other than 0 (for example, 1), and the flag is set to be not ×× when it is 0. In logical NOT, logical product, etc., 1 is regarded as true and 0 is regarded as false (the same applies below). However, other values ​​can also be used as true values ​​and false values ​​in actual devices and methods.

[0078] The syntax elements used to derive inter-frame prediction parameters include, for example, merge flag merge_flag (general_merge_flag), merge index merge_idx, merge_subblock_flag, regulare_merge_flag, ciip_flag, merge_gpm_partition_idx, merge_gpm_idx0, merge_gpm_idx1, inter_pred_idc, reference picture index refldxLX, mvp_LX_idx, differential vector mvdLX, and motion vector accuracy mode amvr_mode. merge_subblock_flag is a flag indicating whether inter-frame prediction in sub-block units is used. regulare_merge_flag is a flag indicating whether the normal merge mode or MMVD is used. ciip_flag is a flag indicating whether the CIIP (combined inter-picture merge and intra-picture prediction) mode is used. merge_gpm_partition_idx is an index indicating the partition shape of the GPM mode. merge_gpm_idx0 and merge_gpm_idx1 are indexes indicating the merge index of the GPM mode. inter_pred_idc is an inter-frame prediction identifier for selecting a reference picture used in the AMVP mode. mvp_LX_idx is a prediction vector index for deriving a motion vector.

[0079] (See picture list)

[0080] The reference picture list is a list of reference pictures stored in the reference picture memory 306. In each CU, refIdxLX specifies which picture in the reference picture list RefPicListX (X = 0 or 1) is actually referenced. It should be noted that LX is a description method used when L0 prediction and L1 prediction are not distinguished. In the following, LX is replaced by L0 and L1 to distinguish the parameters for the L0 list and the parameters for the L1 list.

[0081] (Combined forecast and AMVP forecast)

[0082] The decoding (encoding) methods of prediction parameters include merge prediction mode (merge mode) and AMVP (Advanced Motion Vector Prediction: Advanced Motion Vector Prediction, Adaptive Motion Vector Prediction) mode, and general_merge_flag is a flag used to identify them. Merge mode is a prediction mode that omits part or all of the motion vector difference. It is a mode that does not include the prediction list using the flag predFlagLX, the reference picture index refIdxLX, and the motion vector mvLX in the encoded data, but derives it from the prediction parameters of the processed nearby blocks. AMVP mode is a mode that includes inter_pred_idc, refIdxLX, and mvLX in the encoded data. It should be noted that mvLX is encoded as mvp_LX_idx that identifies the prediction vector mvpLX and the differential vector mvdLX. In addition, the general term for the prediction mode that omits / simplifies the motion vector difference can also be called general merge mode, and the general merge mode and AMVP prediction are selected according to general_merge_flag.

[0083] When general_merge_flag is 1, regular_merge_flag may be transmitted separately. Alternatively, when regular_merge_flag is 1, the normal merge mode or MMVD is selected, and in other cases, the CIIP mode or the GPM mode is selected. The CIIP mode generates a predicted image by weighted sum of an inter-frame predicted image and an intra-frame predicted image. The GPM mode generates a predicted image by treating the target CU as two non-rectangular prediction units divided by a line segment.

[0084] inter_pred_idc is a value indicating the type and number of reference pictures, and takes any value of PRED_L0, PRED_L1, or PRED_BI. PRED_L0 and PRED_L1 respectively indicate unidirectional prediction using a reference picture managed in the L0 list and the L1 list. PRED_BI indicates bidirectional prediction using two reference pictures managed in the L0 list and the L1 list.

[0085] merge_idx is an index indicating whether any prediction parameter among prediction parameter candidates derived from a block for which processing has been completed (merge candidates) is used as a prediction parameter of the target block.

[0086] (Motion Vector)

[0087] mvLX represents the amount of shift between blocks in two different pictures. The prediction vector and difference vector related to mvLX are referred to as mvpLX and mvdLX, respectively.

[0088] (Inter prediction identifier inter_predvidc and prediction list use flag predFlagLX)

[0089] The relationship between inter_red_idc and predFlagL0 and predFlagL1 can be converted to each other as follows.

[0090]

[0091] predFlagLl=inter_pred_idc>>1

[0092] It should be noted that the inter-frame prediction parameter may use a prediction list utilization flag or an inter-frame prediction identifier. In addition, the determination using the prediction list utilization flag may be replaced by the determination using the inter-frame prediction identifier. Conversely, the determination using the inter-frame prediction identifier may also be replaced by the determination using the prediction list utilization flag.

[0093] (Bidirectional prediction biPred determination)

[0094] The flag biPred indicating whether bidirectional prediction is used can be derived from whether both prediction list utilization flags are 1. For example, it can be derived by the following formula.

[0095]

[0096] Alternatively, biPred can also be derived based on whether the inter-frame prediction identifier is a value indicating that two prediction lists (reference pictures) are used. For example, it can be derived using the following formula.

[0097]

[0098] (Configuration of Moving Image Decoding Device)

[0099] The video decoding device 31 ( Figure 3 ) is described below.

[0100] The moving picture decoding device 31 is configured to include an entropy decoding unit 301, a parameter decoding unit (prediction image decoding device) 302, a loop filter 305, a reference picture memory 306, a prediction parameter memory 307, a prediction image generation unit (prediction image generation device) 308, an inverse quantization / inverse transformation unit 311 and an addition unit 312, and a prediction parameter derivation unit 320. It should be noted that, according to the moving picture encoding device 11 described later, there is also a configuration in which the loop filter 305 is not included in the moving picture decoding device 31.

[0101] The parameter decoding unit 302 also includes a header decoding unit 3020, a CT information decoding unit 3021, and a CU decoding unit 3022 (prediction mode decoding unit). The CU decoding unit 3022 also includes a TU decoding unit 3024. They can also be collectively referred to as decoding modules. The header decoding unit 3020 decodes parameter set information such as VPS, SPS, PPS, APS, and slice header (slice information) from the encoded data. The CT information decoding unit 3021 decodes CT from the encoded data. The CU decoding unit 3022 decodes CU from the encoded data. The TU decoding unit 3024 decodes CU from the encoded data.

[0102] When a prediction error is included in a TU, the TU decoding unit 3024 decodes QP update information and quantized transform coefficients from the coded data. The QP update information is a difference value from a quantization parameter prediction value qPpred which is a prediction value of a quantization parameter QP.

[0103] The prediction image generation unit 308 is configured to include an inter-frame prediction image generation unit 309 ( Figure 4 ) and the intra-frame prediction image generation unit 310.

[0104] The prediction parameter derivation unit 320 is configured to include an inter-frame prediction parameter derivation unit 303 ( Figure 5 ) and intra-frame prediction parameter derivation unit.

[0105] In addition, the following describes an example of using CTU and CU as processing units, but the example is not limited thereto, and processing may be performed in sub-CU units. Alternatively, CTU and CU may be replaced with blocks, and sub-CU may be replaced with sub-blocks, and processing may be performed in blocks or sub-blocks.

[0106] The entropy decoding unit 301 performs entropy decoding on the coded stream Te input from the outside, and decodes each code (syntax element). There are the following methods in entropy coding: a method of variable-length coding the syntax element using a context (probability model) appropriately selected according to the type of syntax element and the surrounding conditions; and a method of variable-length coding the syntax element using a predetermined table or calculation formula.

[0107] The entropy decoding unit 301 outputs the decoded code to the parameter decoding unit 302. The decoded code is, for example, prediction mode predMode, general_merge_flag, merge_idx, inter_pred_idc, refIdxLX, mvp_LX_idx, mvdLX, amvr_mode, etc. Based on the instruction of the parameter decoding unit 302, control is performed on which code is decoded.

[0108] (Basic process)

[0109] Figure 6 This is a flowchart for explaining the schematic operation of the moving picture decoding device 31 .

[0110] (S1100: Parameter set information decoding) The header decoding unit 3020 decodes parameter set information such as VPS, SPS, and PPS from the encoded data.

[0111] (S1200: Slice Information Decoding) The header decoding unit 3020 decodes a slice header (slice information) from the encoded data.

[0112] Thereafter, the video decoding device 31 repeatedly performs the processes from S1300 to S5000 for each CTU included in the target picture, thereby deriving a decoded image of each CTU.

[0113] (S1300: CTU Information Decoding) The CT information decoding unit 3021 decodes CTU from the encoded data.

[0114] (S1400: CT Information Decoding) The CT information decoding unit 3021 decodes CT from the encoded data.

[0115] (S1500: CU Decoding) The CU decoding unit 3022 implements S1510 and S1520 to decode the CU from the encoded data.

[0116] (S1510: CU Information Decoding) The CU decoding unit 3022 decodes CU information, prediction information, TU partition flag, CU residual flag, etc. from the encoded data.

[0117] (S1520: TU information decoding) When a prediction error is included in a TU, the TU decoding unit 3024 decodes the quantized prediction error and the like from the encoded data.

[0118] (S2000: Prediction Image Generation) The prediction image generation unit 308 generates a prediction image for each block included in the target CU based on the prediction information.

[0119] (S3000: Inverse Quantization / Inverse Transformation) The inverse quantization / inverse transformation unit 311 performs an inverse quantization / inverse transformation process on each TU included in the target CU.

[0120] (S4000: Decoded Image Generation) The addition unit 312 generates a decoded image of the target CU by adding the predicted image supplied from the predicted image generation unit 308 and the prediction error supplied from the inverse quantization / inverse transformation unit 311 .

[0121] (S5000: Loop Filtering) The loop filter 305 applies loop filtering such as deblocking filtering, SAO (Sample Adaptive Offset), and ALF (Adaptive Loop Filter) to the decoded image to generate a decoded image.

[0122] The loop filter 305 is a filter provided in the encoding loop, and is a filter for improving image quality by removing block distortion and ringing distortion. The loop filter 305 performs filtering such as deblocking filtering, sample adaptive offset (SAO), and adaptive loop filtering (ALF) on the decoded image of the CU generated by the adding unit 312.

[0123] The reference picture memory 306 stores the decoded image of the CU generated by the adding unit 312 at a predetermined position for each target picture and target CU.

[0124] The prediction parameter memory 307 stores the prediction parameter at a predetermined position for each CTU or CU to be decoded. Specifically, the prediction parameter memory 307 stores the parameters decoded by the parameter decoding unit 302 and the prediction mode predMode separated by the entropy decoding unit 301 .

[0125] The prediction image generation unit 308 receives inputs of predMode, prediction parameters, and the like. In addition, the prediction image generation unit 308 reads a reference picture from the reference picture memory 306. The prediction image generation unit 308 generates a prediction image of a block or a sub-block using the prediction parameters and the read reference picture (reference block) in the prediction mode indicated by predMode. Here, the reference block refers to a collection of pixels on the reference picture (usually rectangular, so it is called a block), which is a region that is referenced to generate a prediction image. (Configuration of the inter-frame prediction parameter derivation unit)

[0126] like Figure 5As shown, the inter-frame prediction parameter derivation unit 303 derivates the inter-frame prediction parameters based on the syntax elements input from the parameter decoding unit 302, referring to the prediction parameters stored in the prediction parameter memory 307. In addition, the inter-frame prediction parameters are output to the inter-frame prediction image generation unit 309 and the prediction parameter memory 307. The inter-frame prediction parameter derivation unit 303 and its internal elements, such as the AMVP prediction parameter derivation unit 3032, the merged prediction parameter derivation unit 3036, the MMVD prediction unit 30376, and the MV addition unit 3038, are common units in the motion picture encoding device and the motion picture decoding device, and therefore, they may also be collectively referred to as a motion vector derivation unit (motion vector derivation device).

[0127] When general_merge_flag is 1, indicating the merge prediction mode, merge_idx is derived and output to the merge prediction parameter derivation unit 3036 .

[0128] When general_merge_flag is 0, indicating the AMVP prediction mode, the AMVP prediction parameter derivation unit 3032 derives mvpLX from inter_pred_idc, refIdxLX or mvp_LX_idx.

[0129] (MV addition)

[0130] The derived mvpLX and mvdLX are added in the MV adding unit 3038 to derive mvLX.

[0131] (Combined Forecast)

[0132] The merge prediction parameter derivation unit 3036 derives a merge candidate composed of prediction parameters (predFlagLX, mvLX, refIdxLX) to form a merge candidate list. Furthermore, the merge prediction parameter derivation unit 3036 selects the motion information (mvLXN[0], mvLXN[1]), predFlagLXN, and refIdxLXN of the merge candidate N indicated by merge_idx in the merge candidate list as the inter-frame prediction parameters of the target block. The merge prediction parameter derivation unit 3036 stores the inter-frame prediction parameters of the selected merge candidate in the prediction parameter memory 307, and outputs them to the inter-frame prediction image generation unit 309.

[0133] (Inter-frame prediction image generation unit 309)

[0134] When predMode indicates inter prediction, the inter prediction image generation unit 309 generates a prediction image of a block or a subblock by inter prediction using the inter prediction parameters input from the inter prediction parameter derivation unit 303 and a reference picture.

[0135] Figure 4 : is a schematic diagram showing the structure of the inter-frame prediction image generation unit 309 included in the prediction image generation unit 308 of this embodiment. The inter-frame prediction image generation unit 309 is configured to include:

[0136] A motion compensation unit (prediction image generation device) 3091 , an OOB processing unit 3092 , and a synthesis unit 3095 . The synthesis unit 3095 includes a weighted prediction unit 3094 .

[0137] (Motion Compensation)

[0138] The motion compensation unit 3091 (interpolation image generation unit 3091) generates an interpolation image (motion compensation image) by reading a reference block from the reference picture memory 306 based on the inter prediction parameters (predFlagLX, refIdxLX, mvLX) input from the inter prediction parameter derivation unit 303. The reference block is a block that is shifted from the position of the target block by mvLX on the reference picture RefPicLX specified by refIdxLX. Here, when mvLX is not an integer precision, filtering for generating pixels at decimal positions, called motion compensation filtering, is performed to generate an interpolation image.

[0139] The motion compensation unit 3091 derives the upper left coordinates (xPb, yPb) of the block of size bW*bH, the coordinates within the prediction block (xL, yL), the integer position (xInt, yInt) corresponding to the motion vector (mvLX[0], mvLX[1]), and the phase (xFrac, yFrac) (MC-P1) through the following formula.

[0140]

[0141] yInt=yPb+(mvLX[1]>>(log2(MVPREC)))+yL

[0142] yFrac=mvLX[1]&(MVPREC-1)

[0143] Here, MVPREC represents the precision of mvLX (1 / MVPREC pixel precision), log2MVPREC=(log2(MVPREC)), x=0...bW-1, y=0...bH-1. For example, MVPREC=16 may be used.

[0144] It should be noted that the RPR may be performed by deriving as in (MC-P2) described later. Furthermore, the motion compensation unit 3091 may correct the position of (xInt, yInt) derived from (MC-P1) and (MC-P2) to perform wrapping.

[0145] When the flag that the sub-picture boundary is treated as the picture boundary is valid (sps_subpic_treated_as_pic_flag==1) and the number of sub-pictures of the reference picture is greater than 1 (sps_num_subpics_minus1>0 for the reference picture refPicLX) xInt=CLip3(SubpicLeftBoundaryPos, SubpicRightBoundaryPos, refWraparoundEnabledFlag?CLipH((PpsRefWraparoundOffset)*MinCbSizeY, picW, xInt):xInt)

[0146] yInt=Clip3(SubpicTopBoundaryPos, SubpicBotBoundaryPos, yInt)

[0147] Here, SubpicLefiBoundaryPos, SubpicRightBoundaryPos, SubpicTopBoundaryPos, and SubpicBottomBoundaryPos are the left, right, top, and bottom boundary positions of each sub-picture.

[0148] In cases other than the above (when the flag for treating the sub-picture boundary as the picture boundary is invalid (sps_subpic_treated_as3) ic_flag==0) or when the number of sub-pictures of the reference picture is 1 (sps_Bum_subpics_minus1 for the reference picture refPicLX==0)

[0149]

[0150] Here, refWraparoundEnabledFlag=pps_ref_wraparound_enabled_flag&&!refPiclsScaled, PpsRefWraparoundOffset=ppsjoic_width_in_luma_samples / MinCbSizeY?pps_pic_width_minus_wraparound_offset. MinCbSizeY is a predetermined constant or variable (eg, 4), and pps_pic_width_minus_wraparound_offset is an offset decoded from the coded data indicating the position of the wraparound.

[0151] The motion compensation unit 3091 uses an interpolation filter to perform horizontal interpolation processing on the reference image reflmg, thereby deriving a temporary image temp[][]. The following ∑ is a sum related to k with k=0..NTAP-1, mcFilter[Frac][k] is the kth interpolation filter coefficient in phase Frac, shifil is a normalization parameter of the interval of the adjustment value, and offsetl=l<<(shiftl-1).

[0152] temp[x][y]=(∑mcFilter[xFrac][k]*refIm9[xInt+k-NTAP / 2+1][yint]+offsetl)>>shift1

[0153] Next, the motion compensation unit 3091 performs vertical interpolation processing on the temporary image temp[][] to derive an interpolated image Pred[][]. The following ∑ is a sum related to k of k=0..NTAP-1, shift2 is a normalization parameter of the interval of the adjustment value, and offset2=1<<(shift2-1).

[0154] Pred[x][y]=(rmcFilter[yFrac][k]*temp[x][y+k-NTAP / 2+1]+offset2)>>shift2(Formula MC-1)

[0155] (Out-of-range (OOB) processing)

[0156] The OOB processing unit 3092 includes an OOB determination unit 30921 and an OOB mask derivation unit 30922. The OOB determination unit 30921 determines whether each reference block of the bidirectional prediction mode includes an area outside the range of the reference picture (OOB: Out-Of-Bouondary) based on the inter-frame prediction parameters (predFlagLX, refIdxLX, mvLX) input from the inter-frame prediction parameter derivation unit 303.

[0157] Reference Fig. 9 Provide explanation.

[0158] When the OOB determination unit 30921 determines that the pixels of a part of the reference block are outside the picture, that is, when the reference block is OOB (isOOB == true, S1002), the OOB mask derivation unit 30922 derives mask data (mask value, OOB mask) indicating which area of ​​the reference block is out of range (S1004). The inter-frame prediction image generation unit 309 applies the OOB mask to the reference block that is OOB to generate a prediction image (S1006). The inter-frame prediction image generation unit 309 generates a normal prediction image for the reference block that is not OOB (S1008). The details are described below.

[0159] (OOB determination processing)

[0160] The OOB determination unit 30921 determines whether at least a portion of the pixels of each reference block is outside the picture (OOB) for two (first and second) reference blocks in bidirectional prediction. The area of ​​the LX reference block (X=0 or 1) corresponding to the reference image refLX is represented by the block upper left coordinates (xRefLX, yRefLX) and the block size (width and height: bW, bH). The width and height of the picture are set to picW and picH, respectively. Hereinafter, the integerized value of mvLX is used for the coordinate system according to the precision MVPREC of mvLX. The OOB determination unit 30921 uses these values ​​to derive the truth value isOOB[X] indicating whether the LX reference block is outside the picture through the following formula.

[0161] Fig.10 Indicates an example. Fig.10 This is a diagram in which integers are used as units. The motion vector mvLX' and the reference pixel position (xRefLX', yRefLX') are parameters expressing the motion vector mvLX and the reference pixel position (xRefLX, yRefLX) as integers.

[0162]

[0163] picMinX, picMinY, picMaxX, picMaxY are the minimum and maximum coordinates of the image expressed with the precision of mvLX. The log2MVPREC bit shift ("<<log2MVPREC") may be processed by multiplying it with MVPREC ("*MVPREC"), or the "*MVPREC" described later may be processed by "<<log2MVPREC".

[0164] <Configuration Example 1>

[0165] The OOB determination unit 30921 of this configuration example includes a determination as to whether wraparound processing is applied to the target picture, and determines whether the target block is the target of the OOB processing. Similarly, the determination as to whether the reference block is the target of the OOB processing can be performed using the presence or absence of wraparound processing applied to the reference picture including the reference block. The syntax element pps_ref_wraparound_enabled_flag notified by the PPS is a flag indicating whether wraparound motion compensation in the horizontal direction is available. If pps_ref_wraparound_enabled_flag is true (1), it indicates that wraparound processing of the reference picture is applied to the target picture (valid). When wraparound processing is valid, the OOB determination unit 30921 is set to isOOB[X] = false.

[0166]

[0167] Alternatively, determination may be performed by a logical expression of the OOB determination process instead of by conditional branching.

[0168]

[0169] In the above example, pps_ref_wraparound_enabled_flag is used to determine whether wraparound is enabled, but the present invention is not limited thereto. Other variables or function values ​​derived from the presence or absence of wraparound of the target picture, such as refWraparoundEnabledFlag, may also be used (the same applies to the following configuration).

[0170] According to the above configuration, it is possible to prevent a decrease in encoding efficiency due to a wrap-around process that is not appropriate for OOB processing, and to suppress an increase in the amount of calculation for generating a predicted image.

[0171] <Configuration Example 2>

[0172] The OOB determination unit 30921 can change the OOB determination method based on whether surround processing is applied in the object picture. Similarly, whether the reference block is an OOB processing object can also be determined by whether surround processing is applied to the reference picture including the reference block. For example, the surround processing in VVC is limited to the horizontal direction. Therefore, when the surround processing is valid, isOOB[X] is derived by comparing only the vertical coordinates outside the object of the surround processing, for example, in the following manner. Otherwise, when the surround processing is invalid, the coordinates in both the vertical and horizontal directions are compared to derive isOOB[X] (X=0, 1).

[0173]

[0174] Alternatively, determination may be performed by a logical expression of the OOB determination process instead of by conditional branching.

[0175]

[0176] According to the above configuration, it is possible to achieve an effect that when the surround processing is effective, the OOB processing is also used, thereby improving the encoding efficiency.

[0177] <Configuration Example 3>

[0178] The OOB determination unit 30921 in this configuration example performs OOB determination processing after correcting the coordinates at a predetermined scale for the reference picture to which the reference picture resampling (RPR) processing is applied. Specifically, the coordinates are corrected as follows.

[0179] The motion compensation section 3091 derives the luminance coordinates (refxSbL, refySbL) and (refxL, refyL) (MC-P2) indicated by the motion vector refMvLX in the following manner, instead of the above-mentioned (MC-P1).

[0180]

[0181] Here, offsetP1=(1<<(shiftP1-1), offsetP2=(1<<(shift2-1), offsetP3=(1<<(shiftP3-1)). For example, shiftP1=8, shiftP2=4, shiftP3=6, offsetP1, offsetP2, offsetP3=128, 8, 6.

[0182]

[0183] Here, SubWidthC and SubHeightC are values ​​determined by the sampling method of the color difference format, and are the ratio of the width and height of the color difference pixel to the width and height of the luma pixel. The syntax element sps_chromaformat_idc in the encoded data can be decoded and derived as follows. sps_chroma_format_idc is a parameter indicating the color difference format.

[0184] When sps_chroma_format_idc=0 (Monochrome), SubWidthC=1, SubHeightC=1

[0185] When sps_chromaformat_idc=1(4:2:0), SubWidthC=2,SubHeightC=2

[0186] When sps_chromaformat_idc=2(4:2:2), SubWidthC=2,SubHeightC=1

[0187] When sps_chromaformat_idc=3(4:4:4), SubWidthC=1,SubHeightC=1

[0188] pps_scaling_win_left_offset, pps_scaling_win_top_offset, pps_scaling_win_right_offset, pps_scaling_win_bottom_offset are the offset values ​​applied during scaling. refMvLX (X=0, 1) is the motion vector on the reference picture, and the precision is MVPREC (1 / MVPREC). scalingRatio[0] and scalingRatio[1] are the scaling ratios in the horizontal and vertical directions of the reference picture relative to the target picture, respectively.

[0189]

[0190] fRefWidth is CurrPicScalWinWidthL of the j-th reference picture in RefPicListX, and fRefHeight is CurrPicScalWinHeightL of the j-th reference picture in RefPicListX. pps_pic_width_in_luma_samples and pps_pic_height_in_luma_samples are the width and height of the picture.

[0191] fRefLeftOffset and fRefTopOffset are values ​​set in the following manner.

[0192]

[0193] xL and yL are relative coordinates with the upper left coordinate of the reference block being (0, 0).

[0194] As the upper left coordinate of the reference block in the reference picture when scaling based on the RPR process is not performed, the OOB determination unit 30921 derives (xRefLX, yRefLX) as (refxL, refyL) derived using (xL, yL)=(0, 0).

[0195]

[0196] Furthermore, the width bW and the height bH of the reference block obtained by correcting the scaling based on the RPR process are derived from the following equations.

[0197]

[0198] bH=sbHeight*((scalingRatio[1]+8)>>4)

[0199] It should be noted that the correction method is not limited to the above-mentioned formula, and any other method may be used as long as the coordinate system is scaled using at least the scaling ratio scalingRatio of the RPR process.

[0200] The OOB determination unit 30921 uses the coordinates (xRefLX, xRefLY) corrected by scaling in this way to determine whether the reference block is OOB.

[0201]

[0202] According to the above configuration, the RPR process and the OOB process can be applied simultaneously. That is, the OOB determination process is performed based on the scaled coordinates for the reference picture to which the RPR process is applied, thereby achieving an effect of improving the encoding efficiency based on the OOB process.

[0203] <Configuration Example 4>

[0204] In Configuration Example 4, an example in which OOB processing is not performed on a reference picture to which RPR processing is applied is described. The OOB determination unit 30921 in this configuration example includes determination of whether to apply reference picture resampling (RPR) processing, and determines whether a target block is a target of OOB processing.

[0205]

[0206] Alternatively, determination may be performed by a logical expression of the OOB determination process instead of by conditional branching.

[0207]

[0208] Here, refPiclsScaled is a variable that becomes true when the RPR process is valid in the reference picture and the size of the target picture is different from that of the reference picture. This can be derived as follows.

[0209] refPiclsScaled[i][]=(

[0210] ppsjoic_width_in_luma_samples! =refPicWidth||

[0211] pps_joic_height_in_luma_samples! =refPicHeight||

[0212] pps_scaling_wln_left_offset! =refScalingWinLeftOffset||

[0213] pps_scaling_wm_right_offset! =refScalingWinRightOffset||

[0214] pps_scaling_wln_top_offset! =refScalingWinTopOffset||

[0215] pps_scaling_wm_bottom_offset! =refScalingWinBottomOffset||

[0216] sps_num_subpics_minus1! =fRefNumSubpics)

[0217] sps_num_subpics_minus1+1 and fRefNumSubpics indicate the number of sub-pictures of the object picture and the number of sub-pictures of the reference picture, refPicWidth and refPicHeight indicate the width and height of the reference picture, and refScalingWinLeftOffset, refScalingWinRightOffset, refScalingWinTopOffset and refScalingWinBottomOffset indicate the offset of the reference picture.

[0218] According to the above configuration, the OOB process is not applied to the reference picture to which the RPR process is applied, thereby achieving an effect of reducing the amount of calculation in generating a predicted image.

[0219] <Configuration Example 5>

[0220] In this configuration example, the case where both the surround processing and the reference picture resampling processing can be used is described. The OOB determination unit 30921 of this configuration example includes determining whether the surround processing is applied to the target picture and determining whether the reference picture resampling (RPR) processing is applied, and determining whether the target block is the target of the OOB processing.

[0221]

[0222] Alternatively, determination may be performed by a logical expression of the OOB determination process instead of by conditional branching.

[0223]

[0224] According to the above configuration, in the case where both the surround processing and the reference picture resampling processing can be used, if either of the processing is applied, the OOB processing does not need to be performed. Therefore, the increase in the amount of calculation in the prediction image generation is suppressed. It should be noted that the OOB determination unit 30921 is not limited to this configuration example, and can also be composed of other combinations of configuration examples 1 or 2 and configuration examples 3 or 4.

[0225] (OOB mask derivation processing)

[0226] The OOB mask derivation unit 30922 derives an OOB mask (OOBMask[X]) for the reference block. OOBMask[X] is a binary image (mask) of bWxbH pixels. If the value of OOBMask[X][px][py] (px=0..bW-1, py=0..bH-1) is 0 (false), it means that the pixel (px, py) is in the image. If the value is 1 (true), it means that the pixel (px, py) is outside the image. It should be noted that false and true are not limited to 0 and 1, and can also be other values, such as a bit mask value "0000000000b", "1111111111b", etc., which is a binary number and consists of a string of 0s or 1s with a length greater than BitDepth. In addition, hexadecimal numbers "000", "3FF", "000", "FFF", "0000", "FFFF" can also be used. The OOB mask derivation unit 30922 sets all values ​​of OOBMask[X] to false for blocks that the OOB determination unit 30921 determines are not OOB processing targets (isOOB[X]==false). In addition, for blocks that the OOB determination unit 30921 determines are OOB processing targets (isOOB[X]==true), the mask is derived using the following formula. It should be noted that the inequality sign (">=" or "<=") may not include the equal sign (">" or "<").

[0227]

[0228] Among them, px=0..bW-1, py=0..bH-1.

[0229] (OOB prediction image generation)

[0230] In the case of bidirectional prediction, the motion compensation unit 3091 (interpolated image generation unit 3091) sets Pred[][] derived by the above-mentioned (Formula MC-1) as interpolated images PredL0[][] and PredL1[][] for each L0 list and L1 list. Then, the interpolated image Pred[][] is generated from PredL0[][] and PredL1[][]. When the reference block of the bidirectional prediction is the object of OOB processing, OOBMask[X] corresponding to the reference block is used in the generation of the predicted image. In this case, Pred[][] is derived as shown below.

[0231]

[0232]

[0233] Here, px=0..bW-1, py=0..bH-1. In addition, shift1=Max(2, 14-bitDepth), shift2=Max(3, 15-bitDepth), offset1=1<<(shift1-1), offset2=1<<(shift2-1). It should be noted that shift1 is a shift used to return the interpolated image bit depth (e.g. 14 bits) to the original bit depth (e.g. 14-bitDepth). shift2 is the right shift amount that returns the original bit and also serves as an average. offset1 and offset2 are rounding values ​​when right-shifting shift1 and shift2.

[0234] It should be noted that the above processing can be switched depending on whether explicit weighted bidirectional prediction (BCW) or combined inter-picture merge and intra-picture prediction CIIP (combined inter-picture merge and intra-picture prediction) in units of CU is used. bcwIndex is an index indicating the weight of BCW. ciip_flag is a flag indicating whether CIIP is on or off. The above processing (Formula WP-1) is performed when bcwIndex==0 or the CIIP flag (ciip_flag) is 1. In other cases, the following processing is performed. w1=bcwWLut[bcwIdx]

[0235]

[0236] Here, bcwWLut[k]={4, 5, 3, 10, -2}

[0237]

[0238] Here, offset3=1<<(shift1+2).

[0239] When the flag predFlagLX indicating whether to use the LX reference picture in units of blocks and the processing based on OOBMask are integrated into one branch and processed as shown below, the amount of processing can be reduced.

[0240]

[0241]

[0242] Here, we0=we1=1 / 2 may also be satisfied.

[0243] <Exclusive composition>

[0244] When using the above-mentioned exclusive configuration, OOBMask[0] and OOBMask[1] are not both 1, and therefore the motion compensation unit 3091 (interpolated image generation unit 3091) can derive the interpolated image through the following branch processing.

[0245]

[0246] In addition, it is appropriate to calculate PredL0, PredL1, and PredBI in advance and derive the final Pred through the following mask processing.

[0247] First, PredL0, PredL1, and PredBI are calculated.

[0248]

[0249] Next, when using a bit mask of a binary number and a BitDepth length of "1111111111b" and "0000000000b" as the value of OOBMask[0][px][py] and OOBMask[1][px][py], the final Pred is derived through the following processing.

[0250]

[0251] It should be noted that, in the case of true=1 and false=0, bit mask values ​​"0000000000b" and "1111111111b" may be generated from 0 and 1 and the following processing may be performed. Here, "+" may also be "|" (the same applies to the following mask operation processing)

[0252] Pred[px][py]=(formula excl-1)

[0253] (PredBI[px][py]&(0-((!OOBMask[0][px][py]&!OOBMask[1][px][py])?1:0)))+

[0254] (PredL0[px][py]&(0-(!OOBMask[0][px][py]?1:0)))+

[0255] (PredL1[px][py]&(0-(!OOBMask[1][px][py]?1:0)))

[0256] The above formula includes a calculation for converting a binary (0 or 1) mask into 0 (all bits are 0, the pixel value is invalid) or -1 (all bits are 1, the pixel value is valid) in order to generate a mask with a bit width greater than the bit depth of the pixel value. The derivation of the predicted image is not limited to this, as long as the calculation is used in which the items that are invalidated by the OOB mask among the three items corresponding to PredBI, PredL0, and PredL1 are 0. For example, when the bit depth of the pixel value is set to a maximum of 16 bits, the following formula can be used.

[0257]

[0258] Instead of using addition to combine the three items, bitwise OR may be used as described below.

[0259]

[0260] Alternatively, you can use multiplication.

[0261]

[0262] According to the above, the exclusive processing can be simplified. By the exclusive processing, the processing can be simplified compared with the calculation of (formula excl-1).

[0263] The motion compensation unit 3091 (interpolated image generation unit 3091) can pre-allocate processing by referring to the value of isOOB. For example, it can be set as follows. If so, when isOOB[X] is false, it is not necessary to refer to OOBMask[X][px][py] for each pixel, which can reduce the amount of calculation.

[0264]

[0265] (Weighted prediction)

[0266] The weight prediction unit 3094 generates a predicted image of the block by multiplying the interpolation image PredLX by the weight coefficient.

[0267] The inter-frame prediction image generation unit 309 outputs the generated prediction image of the block to the addition unit 312 .

[0268] (Configuration of Moving Image Coding Device)

[0269] Next, the configuration of the moving picture encoding device 11 according to this embodiment will be described. Figure 7 1 is a block diagram showing the structure of the video coding device 11 of the present embodiment. The video coding device 11 includes a prediction image generation unit 101, a subtraction unit 102, a transformation / quantization unit 103, an inverse quantization / inverse transformation unit 105, an addition unit 106, a loop filter 107, a prediction parameter memory (prediction parameter storage unit, frame memory) 108, a reference picture memory (reference picture storage unit, frame memory) 109, a coding parameter determination unit 110, a parameter coding unit 111, a prediction parameter derivation unit 120, and an entropy coding unit 104.

[0270] The predicted image generation unit 101 generates a predicted image for each CU. The predicted image generation unit 101 includes the inter-prediction image generation unit 309 and the intra-prediction image generation unit which have been described, and the description thereof will be omitted.

[0271] The subtraction unit 102 generates a prediction error by subtracting the pixel value of the predicted image of the block input from the predicted image generation unit 101 from the pixel value of the image T. The subtraction unit 102 outputs the prediction error to the transformation / quantization unit 103 .

[0272] The transform / quantization unit 103 calculates transform coefficients by frequency transforming the prediction error input from the subtraction unit 102 , and derives quantized transform coefficients by quantization. The transform / quantization unit 103 outputs the quantized transform coefficients to the parameter encoding unit 111 and the inverse quantization / inverse transform unit 105 .

[0273] The transform / quantization unit 103 includes a separate transform unit (first transform unit), a non-separable transform unit (second transform unit), and a scaling unit.

[0274] The separation transform unit applies separation transform to the prediction error, and the scaling unit performs scaling on the transform coefficient based on the quantization matrix.

[0275] The inverse quantization / inverse transformation unit 105 is the same as the inverse quantization / inverse transformation unit 311 in the moving picture decoding device 31 , and a description thereof is omitted. The calculated prediction error is output to the addition unit 106 .

[0276] The parameter coding unit 111 includes a header coding unit 1110, a CT information coding unit 1111, and a CU coding unit 1112 (prediction mode coding unit). The CU coding unit 1112 further includes a TU coding unit 1114. The following describes the schematic operation of each module.

[0277] The header coding unit 1110 performs coding processing on parameters such as header information, partition information, prediction information, and quantized transform coefficients.

[0278] The CT information encoding unit 1111 encodes QT, MT (BT, TT) division information and the like.

[0279] The CU encoding unit 1112 encodes CU information, prediction information, partition information, and the like.

[0280] When a prediction error is included in a TU, the TU encoding unit 1114 encodes the QP update information and the quantized prediction error.

[0281] The CT information encoding unit 1111 and the CU encoding unit 1112 supply syntax elements such as inter prediction parameters (predMode, general_merge_flag, merge_idx, inter_pred_idc, refIdxLX, mvp_LX_idx, mvdLX), intra prediction parameters, and quantized transform coefficients to the parameter encoding unit 111 .

[0282] The entropy coding unit 104 receives the quantized transform coefficients and coding parameters (division information, prediction parameters) from the parameter coding unit 111. The entropy coding unit 104 performs entropy coding on these, and generates and outputs a coded stream Te.

[0283] The prediction parameter derivation unit 120 includes an inter prediction parameter encoding unit 112 and an intra prediction parameter encoding unit, and derives intra prediction parameters and inter prediction parameters based on the parameters input from the encoding parameter determination unit 110. The derived intra prediction parameters and inter prediction parameters are output to the parameter encoding unit 111.

[0284] (Configuration of Inter-frame Prediction Parameter Coding Unit)

[0285] The inter prediction parameter coding unit 112 includes a parameter coding control unit 1121 and an inter prediction parameter derivation unit 303. The inter prediction parameter derivation unit 303 is a common configuration with the moving picture decoding apparatus. The parameter coding control unit 1121 includes a merge index derivation unit 11211 and a vector candidate index derivation unit 11212.

[0286] The merge index derivation unit 11211 derives merge candidates and the like, and outputs them to the inter prediction parameter derivation unit 303. The vector candidate index derivation unit 11212 derives prediction vector candidates and the like, and outputs them to the inter prediction parameter derivation unit 303 and the parameter encoding unit 111.

[0287] (Configuration of Intra-frame Prediction Parameter Coding Unit)

[0288] The intra prediction parameter coding unit includes a parameter coding control unit and an intra prediction parameter derivation unit. The intra prediction parameter derivation unit is a common configuration of the moving picture decoding apparatus.

[0289] However, unlike the moving picture decoding device, the inputs to the inter-frame prediction parameter derivation unit 303 and the intra-frame prediction parameter derivation unit are the encoding parameter determination unit 110 and the prediction parameter memory 108 , and the outputs are to the parameter encoding unit 111 .

[0290] The adder 106 generates a decoded image by adding the pixel value of the prediction block input from the prediction image generator 101 and the prediction error input from the inverse quantization / inverse transform unit 105 for each pixel. The adder 106 stores the generated decoded image in the reference picture memory 109.

[0291] The loop filter 107 applies a deblocking filter, SAO, and ALF to the decoded image generated by the adding unit 106. It should be noted that the loop filter 107 does not necessarily include the above three filters, and may include only a deblocking filter, for example.

[0292] The prediction parameter memory 108 stores the prediction parameter generated by the encoding parameter determination unit 110 in a predetermined position for each target picture and CU.

[0293] The reference picture memory 109 stores the decoded image generated by the loop filter 107 at a predetermined position for each target picture and CU.

[0294] The coding parameter determination unit 110 selects one of a plurality of sets of coding parameters. The coding parameters refer to the QT, BT or TT partition information, prediction parameters or parameters generated in association with them as the coding target. The predicted image generation unit 101 generates a predicted image using these coding parameters.

[0295] The coding parameter determination unit 110 calculates the RD cost value representing the size of the information amount and the coding error for each of the multiple sets. The RD cost value is, for example, the sum of the code amount and the value obtained by multiplying the square error by the coefficient λ. The code amount is the amount of information of the coded stream Te obtained by entropy encoding the quantization error and the coding parameter. The square error is the sum of the squares of the prediction error calculated in the subtraction unit 102. The coefficient λ is a real number greater than a preset zero. The coding parameter determination unit 110 selects a set of coding parameters with the smallest calculated cost value. The coding parameter determination unit 110 outputs the determined coding parameters to the parameter encoding unit 111 and the prediction parameter derivation unit 120.

[0296] It should be noted that a part of the moving picture encoding device 11 and the moving picture decoding device 31 in the above-mentioned embodiment, for example, the entropy decoding unit 301, the parameter decoding unit 302, the loop filter 305, the predicted image generation unit 308, the inverse quantization / inverse transformation unit 311, the addition unit 312, the prediction parameter derivation unit 320, the predicted image generation unit 101, the subtraction unit 102, the transformation / quantization unit 103, the entropy encoding unit 104, the inverse quantization / inverse transformation unit 105, the loop filter 107, the coding parameter determination unit 110, the parameter encoding unit 111 and the prediction parameter derivation unit 120 can be implemented by a computer. In this case, the program for realizing the control function can be recorded in a computer-readable recording medium, and the computer system can read the program recorded in the recording medium and execute it. It should be noted that the "computer system" mentioned here refers to a computer system built into any one of the moving picture encoding device 11 and the moving picture decoding device 31, and a computer system including hardware such as an OS and peripheral devices is used. In addition, "computer-readable recording medium" refers to removable media such as floppy disks, magneto-optical disks, ROMs, CD-ROMs, and storage devices such as hard disks built into computer systems. Moreover, "computer-readable recording medium" may also include: recording media that dynamically store programs for a short period of time, such as communication lines when sending programs via networks such as the Internet or communication lines such as telephone lines; and recording media that store programs for a fixed period of time, such as volatile memories inside computer systems that serve as servers or clients in this case. In addition, the above-mentioned program may be a program for realizing a part of the above-mentioned functions, or a program that can realize the above-mentioned functions by combining with programs already recorded in the computer system.

[0297] In addition, part or all of the moving picture encoding device 11 and the moving picture decoding device 31 in the above-mentioned embodiment can also be implemented as an integrated circuit such as LSI (Large Scale Integration). Each functional block of the moving picture encoding device 11 and the moving picture decoding device 31 can be individually processor-based, or part or all of them can be integrated to be processor-based. In addition, the method of integrated circuitization is not limited to LSI, and it can also be implemented by a dedicated circuit or a general-purpose processor. In addition, when a technology for integrated circuitization that replaces LSI appears with the advancement of semiconductor technology, an integrated circuit based on this technology can also be used.

[0298] As mentioned above, one embodiment of the present invention has been described in detail with reference to the drawings, but the specific configuration is not limited to the above embodiment, and various design changes and the like can be made within the scope not departing from the gist of the present invention.

[0299] The embodiments of the present invention are not limited to the above-mentioned embodiments, and various modifications can be made within the scope of the claims. That is, embodiments obtained by combining technical solutions appropriately modified within the scope of the claims are also included in the technical scope of the present invention.

[0300] Industrial Applicability

[0301] The embodiments of the present invention can be preferably applied to a motion picture decoding device that decodes coded data obtained by encoding image data, and a motion picture encoding device that generates coded data obtained by encoding image data. In addition, it can be appropriately applied to the data structure of the coded data generated by the motion picture encoding device and referred to by the motion picture decoding device.

[0302] (Cross-reference of related applications)

[0303] This application claims the benefit of priority based on Japanese patent application No. 2022-108070 filed on July 5, 2022, and all the contents are incorporated into this specification by reference thereto.

[0304] Description of Reference Numerals

[0305] 31: Motion picture decoding device

[0306] 301: Entropy decoding unit

[0307] 302: Parameter decoding unit

[0308] 3022: CU decoding unit

[0309] 3024: TU decoding unit

[0310] 303: Inter-frame prediction parameter derivation unit

[0311] 305, 107: Loop filter

[0312] 306, 109: Reference image memory

[0313] 307, 108: Prediction parameter memory

[0314] 308, 101: Prediction image generation unit

[0315] 309: Inter-frame prediction image generation unit

[0316] 3092: OOB Processing Department

[0317] 30921: OOB Judgment Department

[0318] 30922: OOB mask derivation unit

[0319] 311, 105: Inverse quantization / inverse transformation unit

[0320] 312, 106: Addition Department

[0321] 320: Prediction parameter derivation department

[0322] 11: Motion picture encoding device

[0323] 102: Subtraction Department

[0324] 103: Transformation / Quantization Unit

[0325] 104: Entropy coding unit

[0326] 110: Coding parameter determination unit

[0327] 111: Parameter encoding unit

[0328] 112: Inter-frame prediction parameter encoding unit

[0329] 120: Prediction parameter derivation unit.

Claims

1. A moving picture decoding device, comprising: an out-of-bounds OOB determination unit, which determines whether a reference block is an object of OOB processing by comparing the coordinates of the reference block with the coordinates of a picture; and an OOB mask derivation unit, which derives mask data indicating whether each pixel can be used by comparing the coordinates of pixels included in the reference block with the coordinates of the picture, wherein the moving picture decoding device is characterized in that: The OOB determination unit determines whether the reference block is an OOB processing target by using the presence or absence of a process applied to a reference picture including the reference block.

2. The moving picture decoding device according to claim 1, characterized in that The OOB determination unit determines whether the reference block is an OOB processing target by using the presence or absence of a wrap-around process applied to a reference picture including the reference block.

3. The moving picture decoding device according to claim 2, characterized in that The OOB determination unit determines to invalidate the OOB process on the reference block when the wrap-around process is applied to the reference picture including the reference block.

4. The moving picture decoding device according to claim 2, characterized in that: The OOB determination unit changes a method of determining whether the reference block is an OOB processing target when the wrap-around process is applied to a reference picture including the reference block.

5. The moving picture decoding device according to claim 4, characterized in that When the wrap-around process is applied to the reference picture including the reference block, the OOB determination unit determines whether the reference block is an OOB process target by using coordinate values ​​outside the target of the wrap-around process.

6. The moving picture decoding device according to claim 1, characterized in that: The OOB determination unit determines whether the reference block is an OOB processing target by using the presence or absence of a reference picture resampling process applied to a reference picture including the reference block.

7. The moving picture decoding device according to claim 6, characterized in that: The OOB determination unit determines to invalidate the OOB process on the reference block when a reference picture resampling process is applied to a reference picture including the reference block.

8. The moving picture decoding device according to claim 6, characterized in that The OOB determination unit changes a method of determining whether the reference block is an OOB processing target when a reference picture resampling process is applied to a reference picture including the reference block.

9. The moving picture decoding device according to claim 8, characterized in that: When the reference picture including the reference block is subjected to a reference picture resampling process, the OOB determination unit uses coordinate values ​​to determine whether the reference picture is an OOB processing target, wherein the coordinate values ​​are values ​​corrected using the same scale value as that used in the reference picture resampling process.

Images