Video decoding device, video encoding device, and computer-readable recording medium

Abstract

To provide a video encoding and decoding device that can improve encoding efficiency. [Solution] The motion image decoding device comprises an OOB determination unit that determines whether a reference block is subject to OOB processing by comparing the coordinates of a reference block with the coordinates of a picture, and an OOB mask derivation unit that derives mask data indicating the availability of each pixel by comparing the coordinates of pixels belonging to the reference block with the coordinates of the picture plus an offset, wherein the coordinates of the reference block are the coordinates obtained by adding a motion vector to the coordinates of the target block, and the OOB determination unit determines whether a reference block is subject to OOB processing by using whether or not processing is applied to the reference picture containing the reference block.

Description

Technical Field

[0001] Embodiments of the present invention relate to a moving image decoding device, a moving image encoding device, and a computer-readable recording medium.

Background Art

[0002] In order to efficiently transmit or record a moving image, a moving image encoding device that generates encoded data by encoding the moving image, and a moving image decoding device that generates a decoded image by decoding the encoded data are used.

[0003] Specific moving image encoding methods include, for example, H.264 / AVC, HEVC (High-Efficiency Video Coding), Versatile Video Coding (VVC) method, and the like.

[0004] In such a moving image encoding method, an image (picture) constituting a moving image is managed by a hierarchical structure including a slice obtained by dividing the image, a coding tree unit (CTU: Coding Tree Unit) obtained by dividing the slice, a coding unit (sometimes called a Coding Unit: CU) obtained by dividing the coding tree unit, and a transform unit (TU: Transform Unit) obtained by dividing the coding unit, and is encoded / decoded for each CU.

[0005] Furthermore, in such video encoding schemes, a prediction image is usually generated based on a locally decoded image obtained by encoding / decoding the input image, and the prediction error (sometimes called a "difference image" or "residual image") obtained by subtracting the prediction image from the input image (original image) is encoded. Methods for generating the prediction image include inter-screen prediction and intra-screen prediction. In VVC, as shown in Figure 8(a), the coordinates of the motion vector can be wrapped around to make the left and right edges of the picture continuous with respect to the horizontal coordinate system, thereby compensating for motion. Therefore, applying wrap-around processing to images where the left and right edges of the picture are spatially continuous, such as 360-degree panoramic images and 360° images, can improve encoding efficiency. In addition, VVC allows for reference picture resampling (RPR), which compensates for motion by changing the resolution on a picture-by-picture basis, as shown in Figure 8(b). Applying RPR processing to services with fluctuating transmission rates, such as video streaming over the internet, can improve image quality.

[0006] Furthermore, Non-Patent Document 1 discloses an OOB (Out-Of-Boundary) processing technique that replaces part or all of the region of one reference block of a biprediction that extends outside the picture of the reference image with part or all of the region of the other reference block. [Prior art documents] [Non-patent literature]

[0007] [Non-Patent Document 1] “AHG12: Enhanced bi-directional motion compensation”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 25th Meeting, by teleconference, JVET-Y0125 [Overview of the Initiative] [Problems that the invention aims to solve]

[0008] Non-Patent Literature 1 lacks an OOB (Out-of-Block) detection process that corresponds to wrap-around processing, and therefore has the problem that wrap-around processing and OOB processing cannot be applied simultaneously. Furthermore, when wrap-around processing is applied, as shown in Figure 8, if the reference block is outside the left edge of the picture, it references the area to the right of the picture. Therefore, the horizontal coordinate system will never be outside the picture, and there is a problem that the encoding efficiency may decrease when OOB processing is applied. Also, Non-Patent Literature 1 lacks an OOB detection process that corresponds to RPR (Reverse Processing) processing, and therefore has the problem that RPR processing and OOB processing cannot be applied simultaneously. In RPR processing, a reference picture of a different size from the target picture is scaled down to a predetermined size and held in the reference picture buffer. As a result, the coordinate system of the reference picture is different from the coordinate system of the target picture, and there is a problem that correct OOB detection cannot be performed when RPR processing and OOB processing are used together. [Means for solving the problem]

[0009] To solve the above problems, a video decoding device according to one aspect of the present invention includes an OOB determination unit that determines whether a reference block is subject to OOB processing by comparing the coordinates of a reference block with the coordinates of a picture, and an OOB mask derivation unit that derives mask data indicating the availability of each pixel by comparing the coordinates of pixels belonging to the reference block with the coordinates of the picture with an offset added, The coordinates of the aforementioned reference block are the coordinates of the target block plus the motion vector. The OOB determination unit is characterized by determining whether the reference block is subject to OOB processing based on whether or not processing is applied to the reference picture containing the reference block. [Effects of the Invention]

[0010] According to an aspect of the present invention, the amount of computation required for out-of-bounds (OOB) processing in video encoding and decoding can be reduced. [Brief explanation of the drawing]

[0011] [Figure 1] This is a schematic diagram showing the configuration of the image transmission system according to this embodiment. [Figure 2] This diagram shows the hierarchical structure of the encoded stream data. [Figure 3] This is a schematic diagram showing the configuration of a video decoding device. [Figure 4] This is a schematic diagram showing the configuration of the interpretation image generation unit. [Figure 5] This is a schematic diagram showing the configuration of the interpretation parameter derivation unit. [Figure 6] This is a flowchart illustrating the general operation of a video decoding device. [Figure 7] This is a block diagram showing the configuration of a video encoding device. [Figure 8] This figure shows examples of conventional wrap-around processing and reference picture resampling processing. [Figure 9] This is a flowchart illustrating the general operation of OOB processing. [Figure 10] This is a schematic diagram illustrating an example of OOB (Out of bounds) detection processing. [Modes for carrying out the invention]

[0012] (First embodiment) Embodiments of the present invention will be described below with reference to the drawings.

[0013] Figure 1 is a schematic diagram showing the configuration of the image transmission system 1 according to this embodiment.

[0014] The image transmission system 1 is a system that transmits an encoded stream obtained by encoding a target image and decodes the transmitted encoded stream to display an image. The image transmission system 1 includes a moving image encoding device (image encoding device) 11, a network 21, a moving image decoding device (image decoding device) 31, and a moving image display device (image display device) 41.

[0015] An image T is input to the moving image encoding device 11.

[0016] The network 21 transmits the encoded stream Te generated by the moving image encoding device 11 to the moving image decoding device 31. The network 21 is the Internet, a wide area network (WAN), a local area network (LAN), or a combination thereof. The network 21 is not necessarily limited to a two-way communication network and may be a one-way communication network that transmits broadcast waves such as terrestrial digital broadcasting and satellite broadcasting. Also, the network 21 may be replaced by a storage medium that records the encoded stream Te such as a DVD (Digital Versatile Disc: registered trademark) or a BD (Blue-ray Disc: registered trademark).

[0017] The moving image decoding device 31 decodes each of the encoded streams Te transmitted by the network 21 and generates one or more decoded images Td.

[0018] The moving image display device 41 displays all or part of the one or more decoded images Td generated by the moving image decoding device 31. The moving image display device 41 includes, for example, a display device such as a liquid crystal display or an organic EL (Electro-luminescence) display. Examples of the form of the display include a stationary type, a mobile type, and an HMD. Also, when the moving image decoding device 31 has high processing capabilities, it displays an image with high image quality, and when it has only lower processing capabilities, it displays an image that does not require high processing and display capabilities.

[0019] <Operator> The operators used in this specification are described below.

[0020] >> represents a right bit shift, << represents a left bit shift, & represents a bitwise AND, | represents a bitwise OR, |= represents an OR assignment operator, and || represents a logical OR.

[0021] x?y:z is a ternary operator that takes y when x is true (non-zero) and z when x is false (0).

[0022] Clip3(a, b, c) is a function that clips c to a value between a and b (inclusive). If c < a, it returns a; if c > b, it returns b; otherwise, it returns c (where a <= b).

[0023] ClipH(o, W, x) is a function that returns x if x < 0, x - o if x > W - 1, and x otherwise.

[0024] sign(a) is a function that returns 1 if a > 0, 1 if a == 0, and -1 if a < 0.

[0025] abs(a) is a function that returns the absolute value of a.

[0026] Int(a) is a function that returns the integer value of a.

[0027] floor(a) is a function that returns the largest integer less than or equal to a.

[0028] ceil(a) is a function that returns the smallest integer greater than or equal to a.

[0029] a / d represents the division of a by d, rounded down to the nearest integer. ~ represents a bitwise negation (not). It is a function that returns the opposite of each bit, changing 0 to 1 and 1 to 0.

[0030] <Structure of the Encoded Stream Te> Prior to a detailed description of the video encoding device 11 and video decoding device 31 according to this embodiment, the data structure of the encoded stream Te generated by the video encoding device 11 and decoded by the video decoding device 31 will be described.

[0031] Figure 2 shows the hierarchical structure of data in the encoded stream Te. The encoded stream Te includes, exemplarily, a sequence and multiple pictures that make up the sequence. Figures 2(a) to (f) show the encoded video sequence that defines the sequence SEQ, the encoded picture that defines the picture PICT, the encoded slice that defines the slice S, the encoded slice data that defines the slice data, the encoded tree unit contained in the encoded slice data, and the encoded unit contained in the encoded tree unit, respectively.

[0032] (Encoded video sequence) In an encoded video sequence, a set of data that the video decoding device 31 references to decode the sequence SEQ to be processed is defined. As shown in Figure 2, the sequence SEQ includes a Video Parameter Set, a Sequence Parameter Set SPS, a Picture Parameter Set PPS, an Adaptation Parameter Set (APS), a Picture PICT, and Supplemental Enhancement Information SEI.

[0033] The Video Parameter Set (VPS) defines a set of encoding parameters common to multiple video layers in a video composed of multiple layers, as well as a set of encoding parameters associated with the multiple layers included in the video and with each individual layer.

[0034] The sequence parameter set (SPS) defines a set of encoding parameters that the video decoding device 31 references to decode the target sequence. For example, the width and height of the picture are defined. Multiple SPSs may exist. In that case, one of the multiple SPSs is selected from the PPS.

[0035] The Picture Parameter Set (PPS) defines a set of encoding parameters that the video decoding device 31 references to decode each picture in the target sequence. For example, it includes a reference value for the quantization width used for decoding the picture (pic_init_qp_minus26) and a flag indicating the application of weighted prediction (weighted_pred_flag). Multiple PPSs may exist. In that case, one of the multiple PPSs is selected for each picture in the target sequence.

[0036] (Encoded picture) The encoded picture specifies the set of data that the video decoding device 31 references to decode the picture PICT to be processed. As shown in Figure 2, the picture PICT includes slices 0 to NS-1 (NS being the total number of slices included in the picture PICT).

[0037] In the following, if it is not necessary to distinguish between slices 0 through NS-1, the code subscripts may be omitted. The same applies to other data included in the coded stream Te described below that have subscripts.

[0038] (Encoded slice) In an encoded slice, a set of data that the video decoding device 31 references to decode the slice S to be processed is defined. As shown in Figure 2, the slice includes a slice header and slice data.

[0039] The slice header contains a set of encoding parameters that the video decoding device 31 references to determine the decoding method for the target slice. The slice type specification information (slice_type), which specifies the slice type, is an example of the encoding parameters included in the slice header.

[0040] The slice types that can be specified by the slice type specification information include (1) I slices that use only intra prediction during encoding, (2) P slices that use unidirectional prediction or intra prediction during encoding, and (3) B slices that use unidirectional prediction, bidirectional prediction or intra prediction during encoding. Note that interpretation is not limited to single or bidirectional prediction, and prediction images may be generated using more reference pictures. Hereinafter, when referring to P slices and B slices, we mean slices that contain blocks on which interpretation can be used.

[0041] The slice header may also include a reference to the picture parameter set PPS (pic_parameter_set_id).

[0042] (Encoded slice data) The encoded slice data defines the set of data that the video decoding device 31 references in order to decode the slice data to be processed. As shown in Figure 2(d), the slice data includes CTUs. A CTU is a fixed-size (e.g., 64x64) block that makes up a slice, and is sometimes called a Largest Coding Unit (LCU).

[0043] (Code tree unit) Figure 2 defines the set of data that the video decoding device 31 references to decode the CTU to be processed. The CTU is divided into coding units CU, which are the basic units of encoding processing, by recursive quad tree partitioning (QT), binary tree partitioning (BT), or ternary tree partitioning (TT). BT and TT partitioning together are called multi-tree partitioning (MT). The nodes of the tree structure obtained by recursive quad tree partitioning are called coding nodes. The intermediate nodes of the quad tree, binary tree, and ternary tree are coding nodes, and the CTU itself is defined as the highest-level coding node. The lowest-level coding node is defined as the coding unit.

[0044] (Encoding Unit) Figure 2 defines the set of data that the video decoding device 31 references to decode the encoding unit to be processed. Specifically, the CU consists of a CU header CUH, prediction parameters, transformation parameters, quantization transformation coefficients, etc. The CU header defines the prediction mode, etc.

[0045] Prediction processing can be performed at the CU (Unit) level or at the subCU level, which is a further division of the CU. If the size of the CU and the subCU are equal, there is one subCU within the CU. If the CU is larger than the size of the subCU, the CU is divided into subCUs. For example, if the CU is 8x8 and the subCU is 4x4, the CU will be divided into four subCUs, each consisting of two horizontal and two vertical divisions.

[0046] There are three types of prediction (prediction modes): intra-prediction (MODE_INTRA), inter-prediction (MODE_INTER), and intra-block copy (MODE_IBC). Intra-prediction is prediction within the same picture, while inter-prediction refers to prediction processing performed between different pictures (e.g., between display times, between layer images).

[0047] The transformation and quantization processes are performed in units of CUs, but the quantization transformation coefficients may be entropy-encoded in subblock units such as 4x4.

[0048] (Prediction parameters) The predicted image is derived from the prediction parameters associated with the block. These prediction parameters include intra-prediction and inter-prediction parameters.

[0049] (Prediction parameters for interpretation) This section describes the prediction parameters for interpretation. The interpretation parameters consist of prediction list usage flags predFlagL0 and predFlagL1, reference picture indices refIdxL0 and refIdxL1, and motion vectors mvL0 and mvL1. predFlagL0 and predFlagL1 are flags that indicate whether or not a reference picture list (L0 list, L1 list) is used, and if the value is 1, the corresponding reference picture list is used. In this specification, when referring to "flags indicating whether or not XX is true," a flag value other than 0 (e.g., 1) is considered XX, and 0 is considered not XX. In logical negation, logical AND, etc., 1 is treated as true and 0 as false (the same applies below). However, in actual devices and methods, other values ​​may be used as true and false values.

[0050] Syntax elements for deriving inter-prediction parameters include, for example, the merge flag merge_flag (general_merge_flag), merge index merge_idx, merge_subblock_flag, regular_merge_flag, ciip_flag, merge_gpm_partition_idx, merge_gpm_idx0, merge_gpm_idx1, inter_pred_idc, reference picture index refIdxLX, mvp_LX_idx, difference vector mvdLX, and motion vector accuracy mode amvr_mode. merge_subblock_flag is a flag indicating whether to use inter-prediction on a subblock basis. regular_merge_flag is a flag indicating whether to use the normal merge mode or MMVD. ciip_flag is a flag indicating whether to use the CIIP (combined inter-picture merge and intra-picture prediction) mode. merge_gpm_partition_idx is an index indicating the partition shape of the GPM mode. merge_gpm_idx0 and merge_gpm_idx1 are indices indicating the merge index in GPM mode. inter_pred_idc is an interpretation identifier for selecting the reference picture to use in AMVP mode. mvp_LX_idx is a prediction vector index for deriving the motion vector.

[0051] (Reference picture list) The reference picture list is a list of reference pictures stored in the reference picture memory 306. Each CU specifies which picture in the reference picture list RefPicListX (X=0 or 1) to actually reference using refIdxLX. Note that LX is a notation used when there is no distinction between L0 prediction and L1 prediction, and from now on, parameters for the L0 list and parameters for the L1 list will be distinguished by replacing LX with L0 and L1.

[0052] (Merge prediction and AMVP prediction) There are two methods for decoding (encoding) prediction parameters: merge mode and AMVP (Advanced Motion Vector Prediction). general_merge_flag is a flag used to distinguish between these modes. Merge mode is a prediction mode that omits some or all of the motion vector difference. It derives the prediction list usage flag predFlagLX, reference picture index refIdxLX, and motion vector mvLX from the prediction parameters of already processed neighboring blocks, without including them in the encoded data. AMVP mode includes inter_pred_idc, refIdxLX, and mvLX in the encoded data. Note that mvLX is encoded as mvp_LX_idx, which identifies the prediction vector mvpLX, and the difference vector mvdLX. The general term for prediction modes that omit or simplify the motion vector difference is general merge mode, and general_merge_flag can be used to select between general merge mode and AMVP prediction.

[0053] If general_merge_flag is 1, regular_merge_flag may be transmitted separately. If regular_merge_flag is 1, the normal merge mode or MMVD may be selected, and otherwise, the CIIP mode or GPM mode may be selected. CIIP mode generates a predicted image by a weighted sum of the inter-predicted image and the intra-predicted image. GPM mode generates a predicted image using the target CU as two non-rectangular prediction units divided by a line segment.

[0054] inter_pred_idc is a value that indicates the type and number of reference pictures, and can take one of the following values: PRED_L0, PRED_L1, or PRED_BI. PRED_L0 and PRED_L1 indicate single prediction using one reference picture managed in the L0 list and L1 list, respectively. PRED_BI indicates biprediction using two reference pictures managed in the L0 list and L1 list.

[0055] merge_idx is an index that indicates which of the candidate predictive parameters (merge candidates) derived from the completed block will be used as the predictive parameters for the target block.

[0056] (Motion vector) mvLX represents the amount of shift between blocks in two different pictures. The prediction vector and difference vector for mvLX are called mvpLX and mvdLX, respectively.

[0057] (Interpretation identifier inter_pred_idc and prediction list usage flag predFlagLX) The relationship between inter_pred_idc, predFlagL0, and predFlagL1 is as follows, and they are mutually convertible.

[0058] inter_pred_idc = (predFlagL1<<1)+predFlagL0 predFlagL0 = inter_pred_idc & 1 predFlagL1 = inter_pred_idc >> 1 The inter-prediction parameters may use either the prediction list usage flag or the inter-prediction identifier. Furthermore, the determination using the prediction list usage flag may be replaced with the determination using the inter-prediction identifier. Conversely, the determination using the inter-prediction identifier may be replaced with the determination using the prediction list usage flag.

[0059] (Biprediction of biPred) The flag `biPred`, which indicates whether a system uses two prediction lists, can be derived by checking if both of the two prediction list usage flags are set to 1. For example, it can be derived using the following formula.

[0060] biPred = (predFlagL0==1 && predFlagL1==1) Alternatively, biPred can also be derived by determining whether the interpretation identifier is a value that indicates the use of two prediction lists (reference pictures). For example, it can be derived using the following formula:

[0061] biPred = (inter_pred_idc==PRED_BI) ? 1 : 0 (Configuration of the video decoding device) The configuration of the video decoding device 31 (Figure 3) according to this embodiment will be described below.

[0062] The video decoding device 31 includes an entropy decoding unit 301, a parameter decoding unit (predictive image decoding device) 302, a loop filter 305, a reference picture memory 306, a prediction parameter memory 307, a prediction image generation unit (predictive image generation device) 308, an inverse quantization / inverse transform unit 311, and an addition unit 312 and a prediction parameter derivation unit 320. Note that, in accordance with the video encoding device 11 described later, there is also a configuration in the video decoding device 31 that does not include the loop filter 305.

[0063] The parameter decoding unit 302 further comprises a header decoding unit 3020, a CT information decoding unit 3021, and a CU decoding unit 3022 (prediction mode decoding unit), and the CU decoding unit 3022 further comprises a TU decoding unit 3024. These may be collectively referred to as a decoding module. The header decoding unit 3020 decodes parameter set information such as VPS, SPS, PPS, and APS, and slice headers (slice information) from the encoded data. The CT information decoding unit 3021 decodes CT from the encoded data. The CU decoding unit 3022 decodes CU from the encoded data. The TU decoding unit 3024 decodes TU from the encoded data.

[0064] The TU decoding unit 3024 decodes the QP update information and quantization conversion coefficients from the encoded data if the TU contains a prediction error. The QP update information is the difference from the predicted value of the quantization parameter QP, which is the predicted value of the quantization parameter QP, qPpred.

[0065] The predictive image generation unit 308 includes an inter-predictive image generation unit 309 (Figure 4) and an intra-predictive image generation unit 310.

[0066] The prediction parameter derivation unit 320 is comprised of an inter-prediction parameter derivation unit 303 (Figure 5) and an intra-prediction parameter derivation unit.

[0067] Furthermore, while the following examples use CTU and CU as processing units, processing may be performed in sub-CU units, or the system may be reinterpreted as blocks (CTU and CU) and sub-CU as sub-blocks (sub-CU), and processing may be performed in block or sub-block units.

[0068] The entropy decoding unit 301 performs entropy decoding on the encoded stream Te input from the outside to decode individual codes (syntax elements). Entropy coding can be performed in two ways: one method uses a context (probability model) adaptively selected according to the type of syntax element and the surrounding circumstances to encode the syntax elements by a variable length, and the other method uses a predetermined table or calculation formula to encode the syntax elements by a variable length.

[0069] The entropy decoding unit 301 outputs the decoded codes to the parameter decoding unit 302. The decoded codes include, for example, the prediction mode predMode, general_merge_flag, merge_idx, inter_pred_idc, refIdxLX, mvp_LX_idx, mvdLX, amvr_mode, etc. The control of which code to decode is performed based on the instructions of the parameter decoding unit 302.

[0070] (Basic Flow) Figure 6 is a flowchart illustrating the general operation of the video decoding device 31.

[0071] (S1100: Parameter set information decoding) The header decoding unit 3020 decodes parameter set information such as VPS, SPS, and PPS from the encoded data.

[0072] (S1200: Slice information decoding) The header decoding unit 3020 decodes the slice header (slice information) from the encoded data.

[0073] The video decoding device 31 then derives a decoded image for each CTU by repeating the processing from S1300 to S5000 for each CTU included in the target picture.

[0074] (S1300: CTU information decoding) The CT information decoding unit 3021 decodes the CTU from the encoded data.

[0075] (S1400: CT information decoding) The CT information decoding unit 3021 decodes the CT from the encoded data.

[0076] (S1500: CU Decoding) The CU decoding unit 3022 performs S1510 and S1520 to decode the CU from the encoded data.

[0077] (S1510: CU information decoding) The CU decoding unit 3022 decodes CU information, prediction information, TU partitioning flag, CU residual flag, etc. from the encoded data.

[0078] (S1520: TU information decoding) The TU decoding unit 3024 decodes the quantization prediction error, etc., from the encoded data if the TU contains a prediction error.

[0079] (S2000: Predictive image generation) The predictive image generation unit 308 generates a predictive image for each block included in the target CU based on the prediction information.

[0080] (S3000: Inverse Quantization / Inverse Transform) The inverse quantization / inverse transform unit 311 performs inverse quantization / inverse transform processing for each TU included in the target CU.

[0081] (S4000: Decoded image generation) The summing unit 312 generates a decoded image of the target CU by adding the predicted image supplied from the predicted image generation unit 308 and the prediction error supplied from the inverse quantization / inverse transformation unit 311.

[0082] (S5000: Loop filter) Loop filter 305 applies loop filters such as a deblocking filter, SAO, and ALF to the decoded image to generate a decoded image.

[0083] The loop filter 305 is a filter installed within the encoding loop that removes block distortion and ringing distortion, thereby improving image quality. The loop filter 305 applies filters such as a deblocking filter, sample-adaptive offset (SAO), and adaptive loop filter (ALF) to the decoded CU image generated by the summing unit 312.

[0084] The reference picture memory 306 stores the decoded image of the CU generated by the adder 312 at a predetermined location for each target picture and target CU.

[0085] The prediction parameter memory 307 stores prediction parameters at predetermined locations for each CTU or CU to be decoded. Specifically, the prediction parameter memory 307 stores the parameters decoded by the parameter decoding unit 302 and the prediction mode predMode separated by the entropy decoding unit 301.

[0086] The predictive image generation unit 308 receives input such as predMode and prediction parameters. The predictive image generation unit 308 also reads a reference picture from the reference picture memory 306. The predictive image generation unit 308 generates a block or subblock of predicted image using the prediction mode indicated by predMode, along with the prediction parameters and the read reference picture (reference block). Here, a reference block is a set of pixels on the reference picture (usually rectangular, hence called a block), and is the region referenced to generate the predicted image.

[0087] (Configuration of the Interpretation Parameter Derivation Unit) As shown in Figure 5, the inter-prediction parameter derivation unit 303 derives inter-prediction parameters by referring to the prediction parameters stored in the prediction parameter memory 307 based on the syntax elements input from the parameter decoding unit 302. It also outputs the inter-prediction parameters to the inter-prediction image generation unit 309 and the prediction parameter memory 307. The inter-prediction parameter derivation unit 303 and its internal elements, the AMVP prediction parameter derivation unit 3032, the merge prediction parameter derivation unit 3036, the MMVD prediction unit 30376, and the MV addition unit 3038, are common means in both the video encoding device and the video decoding device, and therefore may be collectively referred to as the motion vector derivation unit (motion vector derivation device).

[0088] If general_merge_flag is 1, i.e., indicates merge prediction mode, merge_idx is derived and output to the merge prediction parameter derivation unit 3036.

[0089] If general_merge_flag is 0, i.e., indicates AMVP prediction mode, the AMVP prediction parameter derivation unit 3032 derives mvpLX from inter_pred_idc, refIdxLX or mvp_LX_idx.

[0090] (MV addition section) The MV adder 3038 adds the derived mvpLX and mvdLX to derive mvLX.

[0091] (Merge prediction) The merge prediction parameter derivation unit 3036 derives merge candidates that include prediction parameters (predFlagLX, mvLX, refIdxLX) and constructs a merge candidate list. Furthermore, the merge prediction parameter derivation unit 3036 selects the movement information of merge candidate N indicated by merge_idx (mvLXN[0], mvLXN[1]), predFlagLXN, and refIdxLXN from among the merge candidates included in the merge candidate list as inter-prediction parameters for the target block. The merge prediction parameter derivation unit 3036 stores the inter-prediction parameters of the selected merge candidates in the prediction parameter memory 307 and outputs them to the inter-prediction image generation unit 309.

[0092] (Interpretation image generation unit 309) If predMode indicates inter-prediction, the inter-prediction image generation unit 309 generates a block or sub-block prediction image by inter-prediction using the inter-prediction parameters input from the inter-prediction parameter derivation unit 303 and a reference picture.

[0093] Figure 4 is a schematic diagram showing the configuration of the inter-predictive image generation unit 309 included in the predictive image generation unit 308 according to this embodiment. The inter-predictive image generation unit 309 is composed of a motion compensation unit (predictive image generation device) 3091, an OOB processing unit 3092, and a synthesis unit 3095. The synthesis unit 3095 is composed of a weight prediction unit 3094.

[0094] (Motion compensation) The motion compensation unit 3091 (interpolation image generation unit 3091) generates an interpolated image (motion-compensated image) by reading a reference block from the reference picture memory 306 based on the inter-prediction parameters (predFlagLX, refIdxLX, mvLX) input from the inter-prediction parameter derivation unit 303. The reference block is the block located at a position shifted by mvLX from the position of the target block on the reference picture RefPicLX specified by refIdxLX. If mvLX is not integer precision, a filter called a motion compensation filter is applied to generate pixels at decimal positions, and then the interpolated image is generated.

[0095] The motion compensation unit 3091 derives the integer positions (xInt, yInt) and phases (xFrac, yFrac) corresponding to the top-left coordinates (xPb, yPb), predicted block coordinates (xL, yL), and motion vectors (mvLX[0], mvLX[1]) of a block of size bW*bH using the following formula (MC-P1).

[0096] xInt = xPb+(mvLX[0]>>(log2(MVPREC)))+xL xFrac = mvLX[0]&(MVPREC-1) yInt = yPb+(mvLX[1]>>(log2(MVPREC)))+yL yFrac = mvLX[1]&(MVPREC-1) Here, MVPREC represents the precision of mvLX (1 / MVPREC pixel precision), log2MVPREC = (log2(MVPREC)), where x = 0…bW-1 and y = 0…bH-1. For example, MVPREC = 16 may also be used. Furthermore, in order to perform RPR, the values ​​may be derived as shown in (MC-P2) below. In addition, the motion compensation unit 3091 may correct the position of (xInt, yInt) derived in (MC-P1) and (MC-P2) for wrap-around purposes. If the flag for treating subpicture boundaries as picture boundaries is enabled (sps_subpic_treated_as_pic_flag == 1) AND the number of subpictures in the reference image is greater than 1 (sps_num_subpics_minus1 > 0 for the reference image refPicLX) xInt = Clip3( SubpicLeftBoundaryPos, SubpicRightBoundaryPos, refWraparoundEnabledFlag ? ClipH( ( PpsRefWraparoundOffset ) * MinCbSizeY, picW, xInt ) : xInt ) yInt = Clip3( SubpicTopBoundaryPos, SubpicBotBoundaryPos, yInt ) Here, SubpicLeftBoundaryPos, SubpicRightBoundaryPos, SubpicTopBoundaryPos, and SubpicBottomBoundaryPos represent the left, right, top, and bottom boundary positions of the subpicture, respectively. In all other cases (when the flag for treating subpicture boundaries as picture boundaries is invalid (sps_subpic_treated_as_pic_flag == 0) or when the number of subpictures in the reference image is 1 (sps_num_subpics_minus1 == 0 for the reference image refPicLX) xInt = Clip3( 0, picW ? 1, refWraparoundEnabledFlag ? ClipH( ( PpsRefWraparoundOffset ) * MinCbSizeY, picW, xInt ) : xInt ) yInt = Clip3( 0, picH ? 1, yInt ) Here, refWraparoundEnabledFlag = pps_ref_wraparound_enabled_flag && !refPicIsScaled, PpsRefWraparoundOffset = pps_pic_width_in_luma_samples / MinCbSizeY ? pps_pic_width_minus_wraparound_offset. MinCbSizeY is a predetermined constant or variable (e.g., 4), and pps_pic_width_minus_wraparound_offset is the offset decoded from the encoded data indicating the wraparound position.

[0097] The motion compensation unit 3091 derives a temporary image temp[][] by performing horizontal interpolation on the reference picture refImg using an interpolation filter. The following Σ is the sum with respect to k of k=0..NTAP-1, mcFilter[Frac][k] is the k-th interpolation filter coefficient in phase Frac, shift1 is a normalization parameter that adjusts the range of values, and offset1=1<<(shift1-1).

[0098] temp[x][y] = (ΣmcFilter[xFrac][k]*refImg[xInt+k-NTAP / 2+1][yInt]+offset1)>>shift1 Next, the motion compensation unit 3091 derives the interpolated image Pred[][] from the temporary image temp[][] by vertical interpolation. The following Σ is the sum with respect to k from k=0..NTAP-1, shift2 is a normalization parameter that adjusts the range of values, and offset2=1<<(shift2-1).

[0099] Pred[x][y] = (ΣmcFilter[yFrac][k]*temp[x][y+k-NTAP / 2+1]+offset2)>>shift2 (formula MC-1) (Out-of-Bounds (OOB) processing) The OOB processing unit 3092 includes an OOB determination unit 30921 and an OOB mask derivation unit 30922. The OOB determination unit 30921 determines whether each reference block in dual prediction mode contains an area outside the scope of the reference picture (OOB: Out-Of-Boundary) based on the inter-prediction parameters (predFlagLX, refIdxLX, mvLX) input from the inter-prediction parameter derivation unit 303.

[0100] Please refer to Figure 9 for further explanation. If the OOB determination unit 30921 determines that some pixels of the reference block are outside the picture, i.e., the reference block is out of bounds (isOOB==true, S1002), the OOB mask derivation unit 30922 derives mask data (mask value, OOB mask) indicating which areas of the reference block are out of bounds (S1004). The interpretation image generation unit 309 applies the OOB mask to the OOB reference block and generates a prediction image (S1006). The interpretation image generation unit 309 generates a normal prediction image for reference blocks that are not out of bounds (S1008). Details are explained below.

[0101] (Out-of-Block detection process) The OOB determination unit 30921 determines, in dual prediction, whether at least some pixels of each of the two (first and second) reference blocks are outside the picture (OOB). The region of the LX reference block (X=0 or 1) corresponding to the reference image refLX is represented by the block's top-left coordinate (xRefLX, yRefLX) and block size (width and height: bW, bH). The width and height of the picture are picW and picH, respectively. In the following, the coordinate system uses values ​​obtained by integerizing mvLX according to the precision of mvLX MVPREC. Using these values, the OOB determination unit 30921 derives a truth value isOOB[X] that indicates whether the LX reference block is outside the picture using the following formula.

[0102] An example is shown in FIG. 10. FIG. 10 is a diagram in integer units, and the motion vector mvLX' and the reference pixel position (xRefLX', yRefLX') are parameters obtained by expressing the motion vector mvLX and the reference pixel position (xRefLX, yRefLX) in integer form. xRefLX = (xPb << log2MVPREC) + mvLX[0] yRefLX = (yPb << log2MVPREC) + mvLX[1] picMinX = 0 - MVPREC / 2 picMinY = 0 - MVPREC / 2 picMaxX = ((picW - 1) << log2MVPREC) + MVPREC / 2 picMaxY = ((picH - 1) << log2MVPREC) + MVPREC / 2 isOOB[X] = (xRefLX <= picMinX) || (yRefLX <= picMinY) || (xRefLX + (bW << log2MVPREC) - 1) >= picMaxX) || (yRefLX + (bH << log2MVPREC) - 1) >= picMaxY) picMinX, picMinY, picMaxX, picMaxY are the minimum and maximum coordinates of the picture expressed in the precision of mvLX. The log2MVPREC-bit left shift ("<< log2MVPREC") can be processed as multiplication ("*MVPREC") with MVPREC, or the subsequent "*MVPREC" can be processed as "<< log2MVPREC". <Configuration Example 1> In this configuration example, the OOB determination unit 30921 determines whether a target block is subject to OOB processing, including whether wrap-around processing is applied to the target picture. Similarly, whether a reference block is subject to OOB processing may be determined using whether wrap-around processing is applied to the reference picture containing the reference block. The syntax element pps_ref_wraparound_enabled_flag notified by PPS is a flag indicating whether horizontal wrap-around motion compensation is available. If pps_ref_wraparound_enabled_flag is true (1), it indicates that wrap-around processing of the reference picture is applied to (enabled) the target picture. When wrap-around processing is enabled, the OOB determination unit 30921 sets isOOB[X]=false. if (pps_ref_wraparound_enabled_flag == true) { isOOB[X] = false else { xRefLX = (xPb< <log2MVPREC) + mvLX[0] yRefLX = (yPb< <log2MVPREC) + mvLX[1] picMinX = 0 - MVPREC / 2 picMinY = 0 - MVPREC / 2 picMaxX = ((picW-1)< <log2MVPREC) + MVPREC / 2 picMaxY = ((picH-1)< <log2MVPREC) + MVPREC / 2 isOOB[X] = (xRefLX <= picMinX) || (yRefLX <= picMinY) || (xRefLX + (bW<<log2MVPREC) -1) > = picMaxX) || (yRefLX + (bH<<log2MVPREC) -1) > = picMaxY) } Alternatively, the determination can be made using a logical expression for the out-of-bounds (OOB) check process, rather than conditional branching.

[0103] isOOB[X] = (pps_ref_wraparound_enabled_flag == false) && ((xRefLX <= picMinX) || (yRefLX <= picMinY) || (xRefLX + (bW<<log2MVPREC) -1) > = picMaxX) || (yRefLX + (bH<<log2MVPREC) -1) > = picMaxY)) In the example above, pps_ref_wraparound_enabled_flag is used to determine whether wraparound processing is enabled or disabled, but this is not the only option. You can also use the values ​​of other variables or functions derived from the presence or absence of wraparound processing in the target picture, such as refWraparoundEnabledFlag (the same applies to the configuration below).

[0104] According to the above configuration, OOB processing prevents a decrease in encoding efficiency due to inappropriate wrap-around processing, and has the effect of suppressing the increase in computational load in predictive image generation.

[0105] <Configuration Example 2> The OOB determination unit 30921 may change the OOB determination method depending on whether wrap-around processing is applied to the target picture. Similarly, the presence or absence of wrap-around processing applied to the reference picture containing the reference block may be used to determine whether the reference block is subject to OOB processing. For example, since wrap-around processing in VVC is limited to the horizontal direction, if wrap-around processing is enabled, isOOB[X] is derived by comparing only the vertical coordinates, which are not subject to wrap-around processing, as shown below. If wrap-around processing is disabled, isOOB[X] is derived by comparing both the vertical and horizontal coordinates (X=0,1). xRefLX = (xPb< <log2MVPREC) + mvLX[0] yRefLX = (yPb< <log2MVPREC) + mvLX[1] picMinX = 0 - MVPREC / 2 picMinY = 0 - MVPREC / 2 picMaxX = ((picW-1)< <log2MVPREC) + MVPREC / 2 picMaxY = ((picH-1)< <log2MVPREC) + MVPREC / 2 if (pps_ref_wraparound_enabled_flag == true) { isOOB[X] = (yRefLX <= picMinY) || (yRefLX + (bH<<log2MVPREC) -1) > = picMaxY) else { isOOB[X] = (xRefLX <= picMinX) || (yRefLX <= picMinY) || (xRefLX + (bW<<log2MVPREC) -1) > = picMaxX) || (yRefLX + (bH<<log2MVPREC) -1) > = picMaxY) } Alternatively, the determination can be made using a logical expression for the out-of-bounds (OOB) check process, rather than conditional branching. isOOB[X] = ((pps_ref_wraparound_enabled_flag == false) && ((xRefLX <= picMinX) || ((xRefLX + (bW<<log2MVPREC) -1) > = picMaxX) || (yRefLX <= picMinY) || ((yRefLX + (bH<<log2MVPREC) -1) > = picMaxY))) According to the above configuration, even when wrap-around processing is enabled, OOB processing can be used in conjunction, resulting in improved encoding efficiency.

[0106] <Configuration Example 3> In this configuration example, the OOB determination unit 30921 performs OOB determination on reference pictures to which reference picture resampling (RPR) processing has been applied, after correcting the coordinates by a predetermined scale. Specifically, the coordinates are corrected as follows.

[0107] The motion compensation unit 3091 derives the luminance coordinates (refxSbL,refySbL) and (refxL,refyL), which are represented by the motion vector refMvLX, as follows (MC-P2), instead of (MC-P1) described above. refxSbL = (((xPb ‐ (SubWidthC * pps_scaling_win_left_offset)) << log2MVPREC) + refMvLX[0]) * scalingRatio[0] refxL = ((sign(refxSbL) * ((abs(refxSbL) + offsetP1) >> shiftP1) + xL * ((scalingRatio[0] + offsetP2) >> shiftP2)) + fRefLeftOffset + offsetP3) >> shiftP3 refySbL = (((yPb ‐ (SubHeightC * pps_scaling_win_top_offset)) << log2MVPREC) + refMvLX[1]) * scalingRatio[1] refyL = ((sign(refySbL) * ((abs(refySbL) + offsetP1) >> shiftP1) + yL * ((scalingRatio[1] + offsetP2) >> shiftP2)) + fRefTopOffset + offsetP3) >> shiftP3 Here, offsetP1=(1<<(shiftP1-1), offsetP2=(1<<(shift2-1), offsetP3=(1<<(shiftP3-1). For example, shiftP1=8, shiftP2=4, shiftP3=6, offsetP1, offsetP2, offsetP3=128, 8, 6. xInt = refxL>>log2MVPREC xFrac = refxL &(MVPREC-1) yInt = refyL >> log2MVPREC yFrac = refyL &(MVPREC-1) Here, SubWidthC and SubHeightC are values ​​determined by the sampling method of the color difference format, and are the ratio of the width and height of the color difference pixels to the width and height of the luminance pixels. Alternatively, the syntax element sps_chroma_format_idc in the encoded data may be decoded and derived as follows: sps_chroma_format_idc is a parameter that indicates the color difference format.

[0108] When sps_chroma_format_idc=0 (Monochrome), SubWidthC=1, SubHeightC=1 When sps_chroma_format_idc=1 (4:2:0), SubWidthC=2, SubHeightC=2 When sps_chroma_format_idc=2(4:2:2), SubWidthC=2, SubHeightC=1 When sps_chroma_format_idc=3(4:4:4), SubWidthC=1, SubHeightC=1 pps_scaling_win_left_offset, pps_scaling_win_top_offset, pps_scaling_win_right_offset, and pps_scaling_win_bottom_offset are offset values ​​applied during scaling. refMvLX(X=0,1) is the motion vector on the reference picture, with precision MVPREC(1 / MVPREC). scalingRatio[0] and scalingRatio[1] are the horizontal and vertical scaling factors of the reference picture relative to the target picture, respectively. scalingRatio[0] = ((fRefWidth<<14) + (CurrPicScalWinWidthL>>1)) / CurrPicScalWinWidthL scalingRatio[1] = ((fRefHeight<<14) + (CurrPicScalWinHeightL>>1)) / CurrPicScalWinHeightL CurrPicScalWinWidthL = pps_pic_width_in_luma_samples ? SubWidthC * (pps_scaling_win_right_offset + pps_scaling_win_left_offset) CurrPicScalWinHeightL = pps_pic_height_in_luma_samples ?SubHeightC * (pps_scaling_win_bottom_offset + pps_scaling_win_top_offset) fRefWidth is the CurrPicScalWinWidthL of the j-th referenced picture in RefPicListX, and fRefHeight is the CurrPicScalWinHeightL of the j-th referenced picture in RefPicListX. pps_pic_width_in_luma_samples and pps_pic_height_in_luma_samples are the width and height of the picture.

[0109] fRefLeftOffset and fRefTopOffset are set to the following values: fRefLeftOffset = (SubWidthC * pps_scaling_win_left_offset) << 10) fRefTopOffset = (SubHeightC * pps_scaling_win_top_offset) << 10) xL and yL are relative coordinates, with the top-left coordinate of the referenced block being (0,0).

[0110] The OOB determination unit 30921 derives (xRefLX, yRefLX) as (refxL, refyL) derived using (xL, yL) = (0, 0) as the top-left coordinate of the reference block in the reference picture when scaling by RPR processing is not performed. xRefLX = refxL = ((sign(refxSbL) * ((abs(refxSbL) + 128) >> 8)) + fRefLeftOffset+ 32) >> 6 yRefLX = refyL = ((sign(refySbL) * ((abs(refySbL) + 128) >> 8)) + fRefTopOffset + 32) >> 6 Furthermore, the width bW and height bH of the reference block, corrected for scaling due to RPR processing, are derived using the following formulas. bW = sbWidth * ((scalingRatio[0]+8)>>4) bH = sbHeight * ((scalingRatio[1]+8)>>4) Note that the correction method is not limited to the formula above. Any method that scales the coordinate system using the scaling ratio of the RPR process, such as `scalingRatio`, is acceptable.

[0111] The OOB determination unit 30921 uses the coordinates (xRefLX, xRefLY) corrected by scaling in this way to determine whether the referenced block is out of bounds (OOB). picMinX = 0 - MVPREC / 2 picMinY = 0 - MVPREC / 2 picMaxX = ((picW-1)< <log2MVPREC) + MVPREC / 2 picMaxY = ((picH-1)< <log2MVPREC) + MVPREC / 2 isOOB[X] = (xRefLX <= picMinX) || (yRefLX <= picMinY) || (xRefLX + (bW<<log2MVPREC) - 1) > = picMaxX) || (yRefLY + (bH<<log2MVPREC) - 1) > = picMaxY) The above configuration makes it possible to apply RPR processing and OOB processing simultaneously. In other words, for reference pictures to which RPR processing has been applied, OOB determination processing is performed based on the scaled coordinates, resulting in the effect of improving encoding efficiency through OOB processing.

[0112] <Configuration Example 4> Configuration Example 4 describes an example in which OOB processing is not performed on reference pictures to which RPR processing has been applied. In this configuration example, the OOB determination unit 30921 determines whether the target block is subject to OOB processing, including whether reference picture resampling (RPR) processing has been applied. if (refPicIsScaled == true) { isOOB[X] = false else { isOOB[X] = (xRefLX <= picMinX) || (yRefLX <= picMinY) || (xRefLX + (bW<<log2MVPREC) -1) > = picMaxX) || (yRefLX + (bH<<log2MVPREC) -1) > = picMaxY) } Alternatively, the determination can be made using a logical expression for the out-of-bounds (OOB) check process, rather than conditional branching.

[0113] isOOB[X] = !refPicIsScaled && ((xRefLX <= picMinX) || (yRefLX <= picMinY) || (xRefLX + (bW<<log2MVPREC) -1) > = picMaxX) || (yRefLX + (bH<<log2MVPREC) -1) > = picMaxY)) Here, refPicIsScaled is a variable that returns true if RPR processing is enabled for the referenced picture and the sizes of the target picture and the referenced picture are different. It can also be derived as follows: refPicIsScaled[i][j] = ( pps_pic_width_in_luma_samples != refPicWidth || pps_pic_height_in_luma_samples != refPicHeight || pps_scaling_win_left_offset != refScalingWinLeftOffset || pps_scaling_win_right_offset != refScalingWinRightOffset || pps_scaling_win_top_offset != refScalingWinTopOffset || pps_scaling_win_bottom_offset != refScalingWinBottomOffset || sps_num_subpics_minus1 != fRefNumSubpics) sps_num_subpics_minus1+1 and fRefNumSubpics indicate the number of subpicks of the target picture and the number of subpicks of the reference picture, respectively. refPicWidth and refPicHeight indicate the width and height of the reference picture, while refScalingWinLeftOffset, refScalingWinRightOffset, refScalingWinTopOffset, and refScalingWinBottomOffset indicate the offset of the reference picture.

[0114] According to the above configuration, OOB processing is not applied to the reference picture to which RPR processing is applied, resulting in a reduction in computational complexity during the generation of the predicted image.

[0115] <Configuration Example 5> This configuration example describes a case where both wrap-around processing and reference picture resampling processing are available. The OOB determination unit 30921 in this configuration example determines whether the target block is subject to OOB processing, including determining whether wrap-around processing is applied to the target picture and whether reference picture resampling (RPR) processing is applied. if (pps_ref_wraparound_enabled_flag || refPicIsScaled) { isOOB[X] = false else { xRefLX = (xPb< <log2MVPREC) + mvLX[0] yRefLX = (yPb< <log2MVPREC) + mvLX[1] picMinX = 0 - MVPREC / 2 picMinY = 0 - MVPREC / 2 picMaxX = ((picW-1)< <log2MVPREC) + MVPREC / 2 picMaxY = ((picH-1)< <log2MVPREC) + MVPREC / 2 isOOB[X] = (xRefLX <= picMinX) || (yRefLX <= picMinY) || (xRefLX + (bW<<log2MVPREC) -1) > = picMaxX) || (yRefLX + (bH<<log2MVPREC) -1) > = picMaxY) } Alternatively, the determination can be made using a logical expression for the out-of-bounds (OOB) check process, rather than conditional branching.

[0116] isOOB[X] = (!refWraparoundEnabledFlag && !refPicIsScaled) && ((xRefLX <= picMinX) || (yRefLX <= picMinY) || (xRefLX + (bW<<log2MVPREC) -1) > = picMaxX) || (yRefLX + (bH<<log2MVPREC) -1) > = picMaxY)) According to the above configuration, if both wrap-around processing and reference picture resampling processing are available, OOB processing is not performed if either processing is applied. Therefore, this has the effect of suppressing the increase in computational load in predictive image generation. Note that the OOB determination unit 30921 may be configured with any other combination of Configuration Example 1 or 2 and Configuration Example 3 or 4, not limited to this configuration example.

[0117] (OOB mask derivation process) The OOB mask derivation unit 30922 derives an OOB mask (OOBMask[X]) for the reference block. OOBMask[X] is a binary image (mask) of bWxbH pixels, and the value of OOBMask[X][px][py](px=0..bW-1,py=0..bH-1) indicates that if it is 0 (false), the pixel (px,py) is inside the picture, and if it is 1 (true), it is outside the picture. Note that false and true are not limited to 0 and 1, but may also be other values, for example, bitmask values ​​consisting of a sequence of 0s or 1s with a length of BitDepth or more in binary, such as "0000000000b" or "1111111111b". Alternatively, hexadecimal values ​​such as "000", "3FF", "000", "FFF", or "0000", "FFFF" may also be used. The OOB mask derivation unit 30922 sets all values ​​of OOBMask[X] to false for blocks that the OOB determination unit 30921 has determined are not subject to OOB processing (isOOB[X] == false). For blocks that the OOB determination unit 30921 has determined are subject to OOB processing (isOOB[X] == true), the mask is derived using the following formula. Note that the inequality signs (">=" or "<=") do not necessarily have to include the equality sign (">" or "<"). OOBMask[X][px][py] = (((xRefLX >> log2MVPREC) + px <= (picMinX >> log2MVPREC)) || ((yRefLX >> log2MVPREC) + py <= (picMinY >> log2MVPREC)) || ((xRefLX >> log2MVPREC) + px >= (picMaxX >> log2MVPREC)) || ((yRefLX >> log2MVPREC) + py >= (picMaxY >> log2MVPREC))) ? 1 : 0 or OOBMask[X][px][py] = ((xRefLX + (px< <log2MVPREC) <= picMinX) || (yRefLX + (py<<log2MVPREC) <= picMinY)) || ((xRefLX + (px<<log2MVPREC) > = picMaxX) || (yRefLX + (py<<log2MVPREC) > = picMaxY)) ? 1 : 0 However, px = 0..bW-1 and py = 0..bH-1.

[0118] (OOB prediction image generation) The motion compensation unit 3091 (interpolation image generation unit 3091) sets the Pred[][] derived by (Equation MC-1) above as interpolation images PredL0[][] and PredL1[][] for each L0 list and L1 list in the case of dual prediction. Then, it generates the interpolation image Pred[][] from PredL0[][] and PredL1[][]. If the reference block for dual prediction is subject to OOB processing, the OOBMask[X] corresponding to the reference block is used to generate the prediction image. In this case, Pred[][] is derived as follows. if (OOBMask[0][px][py] && !OOBMask[1][px][py]) { (formula WP-1) Pred[px][py] = (PredL1[px][py] + offset1)>> shift1 } else if (!OOBMask[0][px][py] && OOBMask[1][px][py]) { Pred[px][py] = (PredL0[px][py] + offset1)>> shift1 else { Pred[px][py] = (PredL0[px][py] + PredL1[px][py] + offset2)>> shift2 } Here, px = 0..bW-1 and py = 0..bH-1. Also, shift1 = Max(2, 14-bit Depth), shift2 = Max(3, 15-bit Depth), offset1 = 1 << (shift1-1), and offset2 = 1 << (shift2-1). Note that shift1 is a shift (e.g., 14-bit Depth) to return the interpolated image bit depth (e.g., 14 bits) back to the original bit depth. shift2 is a right shift amount that returns to the original bit depth and also serves as the average, and offset1 and offset2 are the rounding values ​​when right shifting by shift1 and shift2.

[0119] The above processing may be switched depending on whether or not to use explicitly weighted bi-picture prediction (BCW) or combined inter-picture merge and intra-picture prediction (CIIP) at the CU level. bcwIndex is an index indicating the BCW weights. ciip_flag is a flag indicating whether CIIP is on or off. If bcwIndex==0 or the CIIP flag (ciip_flag) is 1, perform the above processing (Equation WP-1). Otherwise, perform the following processing: w1 = bcwWLut[bcwIdx] w0 = (8 - w1) Here, bcwWLut[k] = {4, 5, 3, 10, -2} if (OOBMask[0][px][py] && !OOBMask[1][px][py]) { Pred[px][py] = (PredL1[px][py] + offset1)>> shift1 } else if (!OOBMask[0][px][py] && OOBMask[1][px][py]) { Pred[px][py] = (PredL0[px][py] + offset1)>> shift1 else { Pred[px][py] = ( w0 * PredL0[px][py] + w1 * PredL1[px][py] + offset3 )>>(shift1+3)) } Here, offset3 = 1 << (shift1 + 2).

[0120] By combining the `predFlagLX` flag, which indicates whether to use an LX reference picture on a block-by-block basis, and the processing using `OOBMask` into a single branch, the amount of processing can be reduced as follows. if ((predFlagL0 == 1 && predFlagL1 ==0) || (!OOBMask[0][px][py] && OOBMask[1][px][py])) { (formula WP-2) Pred[px][py] = (PredL0[px][py]*we0 + offset1)>> shift1 } else if ((predFlagL0 == 0 && predFlagL1 ==1) || OOBMask[0][px][py] && !OOBMask[1][px][py]) { Pred[px][py] = (PredL1[px][py]*we1 + offset1)>> shift1 } else if ((bcwIdx ==0 || ciip_flag == 1) { Pred[px][py] = (w0 * PredL0[px][py] + w1 * PredL1[px][py] + offset2)>>shift2 } else { / / bcwIdx !=0 and ciip_flag == 0) Pred[px][py] = (we0 * PredL0[px][py] + we1 * PredL1[px][py] + offset3)>>(shift1+3)) } Here, we0 = we1 = 1 / 2 is also acceptable. <Exclusive configuration> When using the aforementioned mutual exclusion configuration, OOBMask[0] and OOBMask[1] will never be 1, so the motion compensation unit 3091 (interpolation image generation unit 3091) can derive the interpolated image through the following branching process. if (OOBMask[0][px][py]) { Pred[px][py] = (PredL1[px][py] + offset1)>> shift1 } else if (OOBMask[1][px][py]) { Pred[px][py] = (PredL0[px][py] + offset1)>> shift1 else { Pred[px][py] = (PredL0[px][py] + PredL1[px][py] + offset2)>> shift2 } Furthermore, it is appropriate to calculate PredL0, PredL1, and PredBI in advance and then derive the final Pred using the following masking process. First, calculate PredL0, PredL1, and PredBI. PredL0[px][py] = (PredL0[px][py] + offset1) >> shift1 PredL1[px][py] = (PredL1[px][py] + offset1) >> shift1 PredBI[px][py] = (PredL0[px][py] + PredL1[px][py] + offset2) >> shift2 Next, if we use bitmasks with binary bitDepth lengths of "1111111111b" and "0000000000b" as the values ​​of OOBMask[0][px][py] and OOBMask[1][px][py], the final Pred is derived using the following process. Pred[px][py]= (PredBI[px][py] & (~OOBMask[0][px][py] & ~OOBMask[1][px][py])) + (PredL0[px][py] & ~OOBMask[0][px][py]) + (PredL1[px][py] & ~OOBMask[1][px][py]) If the values ​​are true=1 and false=0, you can generate the bitmask values ​​"0000000000b" and "1111111111b" from 0 and 1 and perform the following operation. Here, "+" can be replaced with "|" (the same applies to the mask operation below). Pred[px][py]= (formula excl-1) (PredBI[px][py] & (0-((!OOBMask[0][px][py] & !OOBMask[1][px][py]) ? 1 : 0)))+ (PredL0[px][py] & (0-(!OOBMask[0][px][py] ? 1 : 0))) + (PredL1[px][py] & (0-(!OOBMask[1][px][py] ? 1 : 0))) The above formula involves a calculation that converts a binary (0 or 1) mask to 0 (all bits are 0, pixel value is invalid) or -1 (all bits are 1, pixel value is valid) in order to create a mask with a bit width greater than or equal to the bit depth of the pixel value. The derivation of the predicted image is not limited to this; any calculation that results in 0 being the term invalidated by the OOB mask among the three terms corresponding to PredBI, PredL0, and PredL1 can be used. For example, when the bit depth of the pixel value is a maximum of 16 bits, the following formula may be used. Pred[px][py]= (PredBI[px][py] & ((!OOBMask[0][px][py] & !OOBMask[1][px][py])? 0xFFFF : 0))+ (PredL0[px][py] & (!OOBMask[0][px][py] ? 0xFFFF : 0)) + (PredL1[px][py] & (!OOBMask[1][px][py] ? 0xFFFF : 0)) Instead of using addition to combine the three terms, you may use bitwise OR as shown below. Pred[px][py]= (PredBI[px][py] & ((!OOBMask[0][px][py] & !OOBMask[1][px][py])? 0xFFFF : 0)) | (PredL0[px][py] & (!OOBMask[0][px][py] ? 0xFFFF : 0)) | (PredL1[px][py] & (!OOBMask[1][px][py] ? 0xFFFF : 0)) Alternatively, multiplication can be used. Pred[px][py]= (PredBI[px][py] * ((!OOBMask[0][px][py] & !OOBMask[1][px][py])? 1 : 0)) + (PredL0[px][py] * (!OOBMask[0][px][py] ? 1 : 0)) + (PredL1[px][py] * (!OOBMask[1][px][py] ? 1 : 0)) According to the above, mutual exclusion can be simplified. By using mutual exclusion, the process can be simplified compared to the operation in (expression excl-1).

[0121] The motion compensation unit 3091 (interpolation image generation unit 3091) may pre-process by referring to the value of isOOB. For example, it may do as follows. In this way, when isOOB[X] is false, it is not necessary to refer to OOBMask[X][px][py] for each pixel, which has the effect of reducing the amount of computation. if (isOOB[0] && !isOOB[1]) { if (OOBMask[0][px][py]) { Pred[px][py] = (PredL1[px][py] + offset1)>> shift2 else { Pred[px][py] = (PredL0[px][py] + PredL1[px][py] + offset2)>> shift } } else if (!isOOB[0] && isOOB[1]) { if (OOBMask[1][px][py]) { Pred[px][py] = (PredL0[px][py] + offset1)>> shift2 else { Pred[px][py] = (PredL0[px][py] + PredL1[px][py] + offset2)>> shift } } else if (isOOB[0] && isOOB[1]) { if (OOBMask[0][px][py] && !OOBMask[1][px][py]) { Pred[px][py] = (PredL1[px][py] + offset1)>> shift2 } else if (!OOBMask[0][px][py] && OOBMask[1][px][py]) { Pred[px][py] = (PredL0[px][py] + offset1)>> shift2 else { Pred[px][py] = (PredL0[px][py] + PredL1[px][py] + offset2)>> shift } else { Pred[px][py] = (PredL0[px][py] + PredL1[px][py] + offset2)>> shift } (Weight prediction) The weight prediction unit 3094 generates a predicted block image by multiplying the interpolated image PredLX by a weight coefficient.

[0122] The interpretation image generation unit 309 outputs the predicted image of the generated block to the addition unit 312.

[0123] (Configuration of the video encoding device) Next, the configuration of the video encoding device 11 according to this embodiment will be described. Figure 7 is a block diagram showing the configuration of the video encoding device 11 according to this embodiment. The video encoding device 11 includes a prediction image generation unit 101, a subtraction unit 102, a transformation / quantization unit 103, an inverse quantization / inverse transformation unit 105, an addition unit 106, a loop filter 107, a prediction parameter memory (prediction parameter storage unit, frame memory) 108, a reference picture memory (reference image storage unit, frame memory) 109, an encoding parameter determination unit 110, a parameter encoding unit 111, a prediction parameter derivation unit 120, and an entropy encoding unit 104.

[0124] The prediction image generation unit 101 generates a prediction image for each CU. The prediction image generation unit 101 includes the inter-prediction image generation unit 309 and the intra-prediction image generation unit, which have already been described, so their explanation will be omitted.

[0125] The subtraction unit 102 subtracts the pixel values ​​of the predicted image of the block input from the predicted image generation unit 101 from the pixel values ​​of image T to generate a prediction error. The subtraction unit 102 outputs the prediction error to the conversion / quantization unit 103.

[0126] The conversion / quantization unit 103 calculates conversion coefficients by frequency conversion for the prediction error input from the subtraction unit 102, and derives quantized conversion coefficients by quantization. The conversion / quantization unit 103 outputs the quantized conversion coefficients to the parameter coding unit 111 and the inverse quantization / inverse conversion unit 105.

[0127] The conversion / quantization unit 103 comprises a separation conversion unit (first conversion unit), a non-separation conversion unit (second conversion unit), and a scaling unit.

[0128] The separation transformation unit applies the separation transformation to the prediction error. The scaling unit performs scaling on the transformation coefficients using a quantization matrix.

[0129] The inverse quantization / inverse transformation unit 105 is the same as the inverse quantization / inverse transformation unit 311 in the video decoding device 31, and therefore its explanation is omitted. The calculated prediction error is output to the summing unit 106.

[0130] The parameter coding unit 111 comprises a header coding unit 1110, a CT information coding unit 1111, and a CU coding unit 1112 (predictive mode coding unit). The CU coding unit 1112 further comprises a TU coding unit 1114. The general operation of each module is described below.

[0131] The header encoding unit 1110 performs encoding processing on parameters such as header information, partitioning information, prediction information, and quantization conversion coefficients.

[0132] The CT information encoding unit 1111 encodes QT, MT (BT, TT) division information, etc.

[0133] The CU encoding unit 1112 encodes CU information, prediction information, division information, etc.

[0134] The TU encoding unit 1114 encodes the QP update information and the quantization prediction error when the TU contains a prediction error.

[0135] The CT information coding unit 1111 and the CU coding unit 1112 supply syntax elements such as inter-prediction parameters (predMode, general_merge_flag, merge_idx, inter_pred_idc, refIdxLX, mvp_LX_idx, mvdLX), intra-prediction parameters, and quantization conversion coefficients to the parameter coding unit 111.

[0136] The entropy coding unit 104 receives quantization conversion coefficients and coding parameters (partitioning information, prediction parameters) from the parameter coding unit 111. The entropy coding unit 104 entropy codes these to generate and output a coding stream Te.

[0137] The prediction parameter derivation unit 120 is a means that includes an inter-prediction parameter coding unit 112 and an intra-prediction parameter coding unit, and derives intra-prediction parameters and inter-prediction parameters from parameters input from the coding parameter determination unit 110. The derived intra-prediction parameters and inter-prediction parameters are output to the parameter coding unit 111.

[0138] (Configuration of the interpredictive parameter coding unit) The interprediction parameter coding unit 112 includes a parameter coding control unit 1121 and an interprediction parameter derivation unit 303. The interprediction parameter derivation unit 303 has the same configuration as the video decoding device. The parameter coding control unit 1121 includes a merge index derivation unit 11211 and a vector candidate index derivation unit 11212.

[0139] The merge index derivation unit 11211 derives merge candidates, etc., and outputs them to the inter-prediction parameter derivation unit 303. The vector candidate index derivation unit 11212 derives prediction vector candidates, etc., and outputs them to the inter-prediction parameter derivation unit 303 and the parameter coding unit 111.

[0140] (Configuration of the intra-predictive parameter coding unit) The intra-predictive parameter coding unit comprises a parameter coding control unit and an intra-predictive parameter derivation unit. The intra-predictive parameter derivation unit has the same configuration as the video decoding device.

[0141] However, unlike the video decoding device, the inputs to the inter-prediction parameter derivation unit 303 and the intra-prediction parameter derivation unit are the coding parameter determination unit 110 and the prediction parameter memory 108, and the output is sent to the parameter coding unit 111.

[0142] The addition unit 106 generates a decoded image by adding the pixel values ​​of the prediction blocks input from the prediction image generation unit 101 and the prediction error input from the inverse quantization / inverse transform unit 105 for each pixel. The addition unit 106 stores the generated decoded image in the reference picture memory 109.

[0143] The loop filter 107 applies a deblocking filter, SAO, and ALF to the decoded image generated by the summing unit 106. Note that the loop filter 107 does not necessarily have to include the above three types of filters; for example, it may consist of only a deblocking filter.

[0144] The prediction parameter memory 108 stores the prediction parameters generated by the coding parameter determination unit 110 in predetermined locations for each target picture and CU.

[0145] The reference picture memory 109 stores the decoded images generated by the loop filter 107 at predetermined locations for each target picture and CU.

[0146] The coding parameter determination unit 110 selects one set from among several sets of coding parameters. The coding parameters are the QT, BT, or TT segmentation information, prediction parameters, or parameters that are to be coded and generated in relation to these. The prediction image generation unit 101 generates a prediction image using these coding parameters.

[0147] The coding parameter determination unit 110 calculates an RD cost value for each of the multiple sets, which indicates the magnitude of the information and the coding error. The RD cost value is, for example, the sum of the code amount and the squared error multiplied by a coefficient λ. The code amount is the information amount of the coded stream Te obtained by entropy coding the quantization error and coding parameters. The squared error is the sum of the squares of the prediction errors calculated in the subtraction unit 102. The coefficient λ is a real number greater than zero that is set in advance. The coding parameter determination unit 110 selects the set of coding parameters that minimizes the calculated cost value. The coding parameter determination unit 110 outputs the determined coding parameters to the parameter coding unit 111 and the prediction parameter derivation unit 120.

[0148] Furthermore, some parts of the video encoding device 11 and video decoding device 31 in the above-described embodiment, such as the entropy decoding unit 301, parameter decoding unit 302, loop filter 305, prediction image generation unit 308, inverse quantization / inverse transformation unit 311, addition unit 312, prediction parameter derivation unit 320, prediction image generation unit 101, subtraction unit 102, transformation / quantization unit 103, entropy encoding unit 104, inverse quantization / inverse transformation unit 105, loop filter 107, encoding parameter determination unit 110, parameter encoding unit 111, and prediction parameter derivation unit 120, may be implemented using a computer. In that case, a program for realizing this control function may be recorded on a computer-readable recording medium, and the program recorded on this recording medium may be loaded into a computer system and executed. Hereinafter, "computer system" refers to a computer system built into either the video encoding device 11 or the video decoding device 31, and includes hardware such as an OS and peripheral devices. Furthermore, "computer-readable recording media" refers to portable media such as flexible disks, magneto-optical disks, ROMs, and CD-ROMs, as well as storage devices such as hard disks built into computer systems. In addition, "computer-readable recording media" may also include those that dynamically hold programs for a short period of time, such as communication lines used when transmitting programs over networks such as the Internet or communication lines such as telephone lines, and those that hold programs for a certain period of time, such as volatile memory inside computer systems that act as servers or clients in such cases. Moreover, the above-mentioned programs may be for the purpose of realizing some of the functions described above, and may also be programs that can realize the aforementioned functions in combination with programs already recorded in the computer system.

[0149] Furthermore, some or all of the video encoding device 11 and video decoding device 31 in the above-described embodiment may be implemented as an integrated circuit such as an LSI (Large Scale Integration). Each functional block of the video encoding device 11 and video decoding device 31 may be individually implemented as a processor, or some or all of them may be integrated into a single processor. In addition, the method of implementing the integrated circuit is not limited to LSIs; it may also be implemented using dedicated circuits or general-purpose processors. Furthermore, if an integrated circuit technology that can replace LSIs emerges due to advances in semiconductor technology, an integrated circuit using that technology may be used.

[0150] Although one embodiment of this invention has been described in detail above with reference to the drawings, the specific configuration is not limited to that described above, and various design changes can be made without departing from the spirit of this invention.

[0151] The embodiments of the present invention are not limited to those described above, and various modifications are possible within the scope of the claims. That is, embodiments obtained by combining technical means that have been appropriately modified within the scope of the claims are also included in the technical scope of the present invention. [Industrial applicability]

[0152] Embodiments of the present invention can be suitably applied to a video decoding device that decodes encoded data from image data, and a video encoding device that generates encoded data from image data. Furthermore, they can be suitably applied to the data structure of encoded data generated by the video encoding device and referenced by the video decoding device. [Explanation of Symbols]

[0153] 31. Video Decoder 301 Entropy Decoder 302 Parameter Decoding Unit 3022 CU Decoding Unit 3024 TU Decoding Unit 303 Interpretation parameter derivation section 305, 107 Loop Filter 306, 109 Reference picture memory 307, 108 Prediction parameter memory 308, 101 Predictive Image Generation Unit 309 Interpretation Image Generation Unit 3092 OOB Processing Unit 30921 OOB judgment section 30922 OOB Mask Output Section 311, 105 Inverse Quantization / Inverse Transformation Section 312, 106 Addition section 320 Prediction parameter derivation section 11. Video Encoding Device 102 Subtraction Unit 103 Conversion / Quantization Section 104 Entropy coding unit 110 Encoding parameter determination unit 111 Parameter coding section 112 Interpretation Parameter Coding Unit 120 Prediction parameter derivation section

Claims

1. An OOB decoding device comprising: an OOB determination unit that determines whether a reference block is subject to OOB processing by comparing the coordinates of a reference block with the coordinates of a picture; and an OOB mask derivation unit that derives mask data indicating the availability of each pixel by comparing the coordinates of pixels belonging to the reference block with the coordinates of the picture plus an offset, The coordinates of the aforementioned reference block are the coordinates of the target block plus the motion vector. The video decoding device is characterized in that the OOB determination unit determines whether the reference block is subject to OOB processing using whether or not processing is applied to the reference picture containing the reference block.

2. The motion image decoding apparatus according to claim 1, characterized in that the OOB determination unit determines whether the reference block is subject to the OOB processing using whether or not a reference picture resampling process is applied to the reference picture that includes the reference block.

3. The motion image decoding apparatus according to claim 2, characterized in that the OOB determination unit determines that the OOB processing on the reference block is invalid when the reference picture resampling process is applied to the reference picture which includes the reference block.

4. The video decoding apparatus according to claim 2, characterized in that the OOB determination unit changes the method for determining whether the reference block is subject to the OOB processing when the reference picture resampling process is applied to the reference picture which includes the reference block.

5. The motion image decoding apparatus according to claim 4, characterized in that, when the reference picture containing the reference block is subjected to the reference picture resampling process, the OOB determination unit determines whether or not the reference picture is subject to the OOB process using coordinate values ​​corrected with the same scale value as the reference picture resampling process.

6. An OOB coding device comprising: an OOB determination unit that determines whether a reference block is subject to OOB processing by comparing the coordinates of a reference block with the coordinates of a picture; and an OOB mask derivation unit that derives mask data indicating the availability of each pixel by comparing the coordinates of pixels belonging to the reference block with the coordinates of the picture plus an offset, The coordinates of the aforementioned reference block are the coordinates of the target block plus the motion vector. The video encoding device is characterized in that the OOB determination unit determines whether the reference block is subject to OOB processing using whether or not processing is applied to the reference picture containing the reference block.

7. When executed on a processor, The process involves comparing the coordinates of the reference block, which are the coordinates of the target block plus a motion vector, with the coordinates of the picture to determine whether the reference block is subject to OOB processing. The process involves comparing the coordinates of pixels belonging to a reference block with the coordinates obtained by adding an offset to the picture's coordinates to derive mask data representing the availability of each pixel. The steps include determining whether the reference block is subject to OOB processing based on whether or not processing is applied to the reference picture containing the reference block, and A computer-readable storage medium that stores a program containing instructions to execute a command.

Images