Moving picture decoding device
By combining the weighted prediction processing unit and the usual prediction processing unit in the moving image decoding and encoding device, selecting an appropriate prediction method according to the characteristics of the encoded data, the error application problem of weighted prediction processing in the prior art is solved, and the accuracy of processing is improved Sex and efficiency.
Patent Information
- Application Number
- CN202410976114.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2019-12-03
- Filing Date
- 2020-11-26
- Publication Date
- 2024-11-15
AI Technical Summary
In the prior art, in the process of motion image encoding and decoding, there is an incorrect application of weighted prediction processing, resulting in weighted prediction processing even if the normal prediction processing should be performed.
A moving image decoding device and an encoding device are designed. By combining a weighted prediction processing unit and a normal prediction processing unit, weighted prediction or normal prediction is selectively performed based on the existence or absence of the encoded data, ensuring that weighted coefficients and biases are missing. When shifting the value, perform appropriate prediction processing.
It effectively solves the problem of erroneous application of weighted prediction processing, ensures that weighted prediction is not performed in the motion image encoding/decoding process without weighting coefficients, which improves the accuracy and efficiency of the processing.
Smart Images

Figure CN118972619A_ABST
Abstract
Description
Technical Field
[0001] Embodiments of the present invention relate to a predicted image generation device, a moving image decoding device, and a moving image encoding device. Background Art
[0002] In order to efficiently transmit or record moving images, a moving image encoding device that generates encoded data by encoding the moving image and a moving image decoding device that generates a decoded image by decoding the encoded data are used.
[0003] Specific moving picture coding methods include, for example, H.264 / AVC and H.265 / HEVC (High-Efficiency Video Coding).
[0004] In such a motion image coding method, images (pictures) constituting the motion image are managed through a hierarchical structure and encoded / decoded for each CU. The hierarchical structure includes slices obtained by dividing the image, coding tree units (CTU: Coding Tree Unit) obtained by dividing the slices, coding units (sometimes also called coding units (Coding Unit: CU)) obtained by dividing the coding tree units, and transform units (TU: Transform Unit) obtained by dividing the coding units.
[0005] In such a moving picture coding method, a prediction image is usually generated based on a local decoded image obtained by encoding / decoding an input image, and a prediction error (sometimes also referred to as a "difference image" or "residual image") obtained by subtracting the prediction image from the input image (original image) is encoded. Examples of methods for generating a prediction image include inter-picture prediction (inter-frame prediction) and intra-picture prediction (intra-frame prediction).
[0006] In addition, Non-Patent Document 1 can be cited as a recent moving picture encoding and decoding technology.
[0007] In non-patent document 1, weighted prediction (WeightedPrediction) is switched by the variable weightedPredFlag. If slice_type is equal to P, weightedPredFlag is set to pps_weighted_pred_flag. In other cases (when slice_type is equal to B), weightedPredFlag is set to (pps_weighted_bipred_flag &&!dmvrFlag). Here, pps_weighted_pred_flag is a flag indicating whether weighted prediction is applied in the variables defined by the picture parameter set (Picture ParameterSet) when slice_type is P. In addition, pps_weighted_bipred_flag is a flag indicating whether weighted prediction is applied in the variables defined by the picture parameter set when slice_type is B. The variable dmvrFlag is a variable indicating whether the DMVR processing described later is performed.
[0008] The weighted prediction process is called by the value of the variable weightedPredFlag. If weightedPredFlag is equal to 0 or bcwIdx is not equal to 0, the normal prediction process is called, and in other cases (weightedPredFlag is 1 and bcwIdx is 0), the weighted prediction process is called. Prior art literature Non-patent literature
[0009] Non-patent document 1: “Versatile Video Coding (Draft 7)”, JVET-P2001-vE, Joint Video Exploration Team (JVET) of ITU-T SG 16WP 3and ISO / IEC JTC 1 / SC 29 / WG 11, 2019-11-14 Summary of the invention Problem that the invention aims to solve
[0010] However, sometimes, even when the value of the variable weightedPredFlag is 1 and the weighted prediction process is called, the normal prediction process should be performed as a result. The method described in Non-Patent Document 1 has the following problem: the weighted prediction process is performed even in such a case.
[0011] Specifically, in non-patent document 1, for each reference picture in the L0 list and the L1 list, there is a flag indicating whether there are weighting coefficients and offset values for brightness and color difference, and there is only one right shift value (set to X) equivalent to the denominator of the weighting coefficient for brightness and color difference. Therefore, when there are no weighting coefficients and offset values for the reference list, the value of the weighting coefficient is set to 2 to the power of X, and the offset value is set to 0 to perform weighted prediction processing. Therefore, in the case of L0 prediction or L1 prediction, when there are no weighting coefficients and offset values for the reference list, the value of the weighting coefficient is set to 2 to the power of X, and the offset value is set to 0 to perform weighted prediction processing at the position where the normal prediction processing should be performed. In addition, in the case of bidirectional prediction, when there are no weighting coefficients and offset values for both the L0 list and the L1 list, the value of the weighting coefficient is set to 2 to the power of X, and the offset value is set to 0 to perform weighted prediction processing at the position where the normal bidirectional prediction processing should be performed. In this way, there is a problem that weighted prediction processing is sometimes performed even when the normal prediction processing should be performed. Technical Solution
[0012] A moving picture decoding device according to one aspect of the present invention is characterized by comprising: a weighted prediction processing unit that decodes the weighting coefficient and the offset value based on the coded data, multiplies the interpolation image by the weighting coefficient and adds the offset value to generate a predicted image; and Usually, the prediction processing unit generates a prediction image from the interpolation image. In the weighted prediction processing unit, in the case of bidirectional prediction, if there is no information on weight coefficients and offset values for both reference list 0 and reference list 1, a bidirectional prediction image is generated by the normal prediction processing unit.
[0013] Furthermore, a moving picture decoding device according to one aspect of the present invention is characterized by comprising: a weighted prediction processing unit that decodes the weighting coefficient and the offset value based on the coded data, multiplies the interpolation image by the weighting coefficient and adds the offset value to generate a predicted image; and Usually, the prediction processing unit generates a prediction image from the interpolation image. In the weighted prediction processing unit, in the case of unidirectional prediction (L0 prediction or L1 prediction), when there is no information on the weight coefficient and offset value of the reference list, the unidirectional prediction image is generated by the normal prediction processing unit.
[0014] A moving picture encoding device according to one aspect of the present invention is characterized by comprising: a weighted prediction processing unit that encodes the weighting coefficient and the offset value as encoding data, multiplies the interpolation image by the weighting coefficient and adds the offset value to generate a predicted image; and Usually, the prediction processing unit generates a prediction image from the interpolation image. In the weighted prediction processing unit, in the case of bidirectional prediction, if there is no information on weight coefficients and offset values for both reference list 0 and reference list 1, a bidirectional prediction image is generated by the normal prediction processing unit.
[0015] Furthermore, a moving picture encoding device according to one aspect of the present invention is characterized by comprising: a weighted prediction processing unit that encodes the weighting coefficient and the offset value as encoding data, multiplies the interpolation image by the weighting coefficient and adds the offset value to generate a predicted image; and Usually, the prediction processing unit generates a prediction image from the interpolation image. In the weighted prediction processing unit, in the case of unidirectional prediction, when there is no information on the weight coefficient and offset value of the reference list, the unidirectional prediction image is generated by the normal prediction processing unit. Beneficial Effects
[0016] According to one aspect of the present invention, it is possible to solve the above-mentioned problem by separately specifying for brightness and color difference signals that weighted prediction is not performed when no weighting coefficient exists in the moving picture encoding / decoding process. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] Figure 1 This is a schematic diagram showing the configuration of the image transmission system according to the present embodiment. Figure 2 This is a diagram showing the configuration of a transmitting device equipped with a moving picture encoding device according to the present embodiment and a receiving device equipped with a moving picture decoding device. PROD_A shows a transmitting device equipped with a moving picture encoding device, and PROD_B shows a receiving device equipped with a moving picture decoding device. Figure 3 This is a diagram showing the configuration of a recording device equipped with a moving picture encoding device and a playback device equipped with a moving picture decoding device according to the present embodiment. PROD_C represents a recording device equipped with a moving picture encoding device, and PROD_D represents a playback device equipped with a moving picture decoding device. Figure 4 This is a diagram showing the hierarchical structure of the data of the coded stream. Figure 5 This is a conceptual diagram of an image to be processed in the image transmission system of this embodiment. Figure 6 This is a conceptual diagram showing an example of a reference picture and a reference picture list. Figure 7 This is a schematic diagram showing the structure of a moving picture decoding device. Figure 8This is a flowchart for explaining the schematic operation of the moving picture decoding device. Fig. 9 This is a diagram showing an example of the structure of a syntax table of a slice header. Fig.10 It is a flowchart which shows an example of the process of this application. Fig.11 This is a schematic diagram showing the structure of an inter-frame prediction parameter derivation unit. Fig.12 It is a schematic diagram showing the structure of the combined prediction parameter derivation unit and the AMVP prediction parameter derivation unit. Fig.13 It is a diagram showing the affine predicted motion vector spMvLX[xi][yi]. Fig.14 This is a schematic diagram showing the structure of an inter-frame prediction image generation unit. Fig.15 This is a table for deriving coefficients in interpolation filter processing. Fig.16 It is a table for the derivation of coefficients in the interpolation filter process. Fig.17 It is a table for the derivation of coefficients in the interpolation filter process. Fig.18 It is a table for the derivation of coefficients in the interpolation filter process. Fig.19 It is a table for the derivation of coefficients in the interpolation filter process. Fig. 20 It is a table for the derivation of coefficients in the interpolation filter process. Fig.21 It is a table for the derivation of coefficients in the interpolation filter process. Fig. 22 This is a block diagram showing the structure of a moving picture encoding device. Fig.23 This is a schematic diagram showing the structure of the inter-frame prediction parameter encoding unit. Fig.24 This is a schematic diagram showing the structure of an intra-frame prediction parameter encoding unit. Fig.25 This is a flowchart for explaining the process of deriving a predicted image by the BDOF unit. Fig.26 It is a schematic diagram showing the structure of the BDOF section. Fig. 27 This is a diagram for explaining the syntax of the coding parameters of weighted prediction. Fig.28 This is a diagram for explaining the syntax of the coding parameters of weighted prediction. Fig.29This is a diagram for explaining the syntax of BCW prediction. Fig.30 This is a diagram for explaining the syntax of the coding parameters of weighted prediction. DETAILED DESCRIPTION (First Embodiment)
[0018] Hereinafter, embodiments of the present invention will be described with reference to the drawings.
[0019] Figure 1 It is a schematic diagram showing the configuration of the image transmission system 1 according to the present embodiment.
[0020] The image transmission system 1 is a system for transmitting a coded stream obtained by encoding an image of a different resolution whose resolution is converted, decoding the transmitted coded stream, and inversely converting the image to the original resolution for display. The image transmission system 1 is configured to include: a resolution conversion device (resolution conversion unit) 51, a motion image encoding device (image encoding device) 11, a network 21, a motion image decoding device (image decoding device) 31, a resolution inverse conversion device (resolution inverse conversion unit) 61, and a motion image display device (image display device) 41.
[0021] The resolution conversion device 51 converts the resolution of the image T included in the moving image, and supplies a variable resolution moving image signal including images of different resolutions to the image encoding device 11. In addition, the resolution conversion device 51 supplies information indicating the presence or absence of resolution conversion of the image to the moving image encoding device 11. When the information indicates resolution conversion, the moving image encoding device sets resolution conversion information ref_pic_resampling_enabled_flag described later to 1, and includes it in the sequence parameter set SPS (Sequence Parameter Set) of the encoded data for encoding.
[0022] The image T whose resolution has been converted is input to the moving image encoding device 11 .
[0023] The network 21 transmits the coded stream Te generated by the motion image coding device 11 to the motion image decoding device 31. The network 21 is the Internet, a wide area network (WAN), a small network (LAN) or a combination thereof. The network 21 is not necessarily limited to a two-way communication network, but may also be a one-way communication network that transmits broadcast waves such as terrestrial digital broadcasting and satellite broadcasting. In addition, the network 21 may also be replaced by a storage medium such as a DVD (Digital Versatile Disc: Digital Versatile Disc, registered trademark) or a BD (Blue-ray Disc: Blue-ray Disc, registered trademark) that records the coded stream Te.
[0024] The moving picture decoding device 31 decodes each coded stream Te transmitted through the network 21 , generates a variable resolution decoded picture signal, and supplies the decoded picture signal to the resolution inverse conversion device 61 .
[0025] The resolution inverse conversion device 61 generates a decoded image signal of the original size by inversely converting the resolution-converted image in a case where the resolution conversion information included in the variable-resolution decoded image signal indicates resolution conversion.
[0026] The moving image display device 41 displays all or part of one or more decoded images Td indicated by the decoded image signal input from the resolution inverse conversion unit. The moving image display device 41 includes, for example, a display device such as a liquid crystal display or an organic EL (Electro-luminescence) display. Examples of the display type include a fixed type, a mobile type, and an HMD. In addition, when the moving image decoding device 31 has a high processing capability, a high-quality image is displayed. When the moving image decoding device 31 has only a low processing capability, an image that does not require a high processing capability or a high display capability is displayed.
[0027] Figure 5 is Figure 1 The conceptual diagram of the image to be processed in the image transmission system 1 shown in FIG. 1 is a diagram showing the change in resolution of the image over time. Figure 5 In the , it is not distinguished whether the image is encoded or not. Figure 5 2 shows an example in which the resolution of an image is reduced during the processing of the image transmission system 1 and the image is transmitted to the image decoding device 31. Figure 5 As shown, generally, the resolution conversion device 51 performs conversion to lower the resolution of the image to reduce the amount of information to be transmitted. <operator>
[0028] The following describes operators used in this specification.
[0029] >> indicates a right shift, << indicates a left shift, & indicates a bitwise AND, | indicates a bitwise OR, |= indicates an OR assignment operator, and || represents a logical sum.
[0030] x? y : z is a ternary operator that takes y when x is true (other than 0) and takes z when x is false (0).
[0031] Clip3(a, b, c) is a function that clips c to a value between a and b (inclusive). It returns a when c < a, b when c > b, and c otherwise (where a <= b).
[0032] abs(a) is a function that returns the absolute value of a.
[0033] Int(a) is a function that returns the integer value of a.
[0034] floor(a) is a function that returns the largest integer less than or equal to a.
[0035] ceil(a) is a function that returns the smallest integer greater than or equal to a.
[0036] a / d represents a divided by d (truncating the decimal part). <Structure of the coded stream Te>
[0037] Before explaining the motion picture encoding device 11 and the motion picture decoding device 31 of the present embodiment in detail, the data structure of the coded stream Te generated by the motion picture encoding device 11 and decoded by the motion picture decoding device 31 will be described.
[0038] Figure 4 A diagram showing the hierarchical structure of the data in the coded stream Te. The coded stream Te exemplarily includes a sequence and multiple pictures constituting the sequence. Figure 4 A diagram showing the coded video sequence representing a given sequence SEQ, the coded picture of a specified picture PICT, the coded slice of a specified slice S, the coded slice data of the specified slice data, the coding tree units included in the coded slice data, and the coding units included in the coding tree units is shown. (Coded video sequence)
[0039] In the coded video sequence, a set of data for the motion picture decoding device 31 to refer to for decoding the sequence SEQ to be processed is specified. As Figure 4As shown, the sequence SEQ includes a video parameter set VPS (Video Parameter Set), a sequence parameter set SPS (Sequence Parameter Set), a picture parameter set PPS (Picture Parameter Set), an adaptation parameter set (Adaptation Parameter Set: APS), a picture PICT and supplemental enhancement information SEI (Supplemental Enhancement Information).
[0040] The video parameter set VPS defines a set of coding parameters common to a plurality of moving images in a moving image composed of multiple layers, and a set of coding parameters associated with the multiple layers included in the moving image and with each layer.
[0041] In the sequence parameter set SPS, a set of coding parameters is specified for reference by the motion image decoding device 31 in order to decode the object sequence. For example, the width and height of the picture are specified. It should be noted that there can be multiple SPSs. In this case, any one of the multiple SPSs is selected from the PPS.
[0042] Here, the sequence parameter set SPS includes the following syntax. ref_pic_resampling_enabled_flag: This is a flag that specifies whether to use a function that sets the resolution to be variable (resampling) when decoding each picture included in a single sequence of the reference target SPS. On the other hand, this flag is a flag indicating that the size of the reference picture referred to in the generation of the predicted picture varies between each picture shown in the single sequence. When the value of this flag is 1, the above resampling is applied, and when the value of this flag is 0, the above resampling is not applied. pic_width_max_in_luma_samples: This is a syntax for specifying the width of an image with the largest width in units of luma blocks in an image in a single sequence. In addition, the value of this syntax is required to be non-zero and an integer multiple of Max(8, MinCbSizeY). Here, MinCbSizeY is a value determined based on the minimum size of a luma block. pic_height_max_in_luma_samples: This is a syntax for specifying the height of an image having the maximum height in units of luma blocks among images in a single sequence. In addition, the value of this syntax is required to be non-zero and an integer multiple of Max(8, MinCbSizeY). sps_temporal_mvp_enabled_flag: This is a flag that specifies whether to use temporal motion vector prediction when decoding the target sequence. If the value of this flag is 1, temporal motion vector prediction is used, and if the value is 0, temporal motion vector prediction is not used. In addition, by specifying this flag, it is possible to prevent the coordinate position referenced from being offset when referring to a reference picture of a different resolution.
[0043] In the picture parameter set PPS, a set of coding parameters is specified for reference by the motion image decoding device 31 to decode each picture in the target sequence. For example, it includes a reference value (pic_init_qp_minus26) of the quantization width used for decoding the picture and a flag (weighted_pred_flag) indicating the application of weighted prediction. It should be noted that there can be multiple PPSs. In this case, any one of the multiple PPSs is selected from each picture in the target sequence.
[0044] Here, the picture parameter set PPS includes the following syntax. ppic_width_in_luma_samples: This is a syntax for specifying the width of the target picture. The value of this syntax must not be 0, must be an integer multiple of Max(8, MinCbSizeY), and must be less than or equal to pic_width_max_in_luma_samples. pic_height_in_luma_samples: This is a syntax for specifying the height of the target picture. The value of this syntax must not be 0, must be an integer multiple of Max(8, MinCbSizeY), and must be less than or equal to pic_height_max_in_luma_samples. conformance_window_flag: This is a flag indicating whether to continuously notify the conformance (cropping) window offset parameter, and is a flag indicating the location where the conformance window is displayed. If this flag is 1, the parameter is notified, and if this flag is 0, it indicates that there is no conformance window offset parameter. conf_win_left_offset, conf_win_right_offset, conf_win_top_offset, conf_win_bottom_offset: offset values for specifying the left, right, top, and bottom positions of the picture output by the decoding process with respect to the rectangular area specified by the output picture coordinates. In addition, when the value of conformance_window_flag is 0, the values of conf_win_left_offset, conf_win_right_offset, conf_win_top_offset, and conf_win_bottom_offset are estimated to be 0. ·scaling_window_flag: This is a flag indicating whether the scaling window offset parameter exists in the target PPS, and is a flag related to the specification of the output image size. When this flag is 1, it indicates that the parameter exists in the PPS, and when this flag is 0, it indicates that the parameter does not exist in the PPS. In addition, when the value of ref_pic_resampling_enabled_flag is 0, the value of scaling_window_flag is also required to be 0. scaling_win_left_offset, scaling_win_right_offset, scaling_win_top_offset, scaling_win_bottom_offset: are syntaxes for specifying the offsets applied to the image size for scaling ratio calculation in units of luma samples for the left, right, top, and bottom positions of the target picture, respectively. In addition, when the value of scaling_window_flag is 0, the values of scaling_win_left_offset, scaling_win_right_offset, scaling_win_top_offset, and scaling_win_bottom_offset are estimated to be 0. In addition, the value of scaling_win_left_offset+scaling_win_right_offset is required to be less than pic_width_in_luma_samples, and the value of scaling_win_top_offset+scaling_win_bottom_offset is required to be less than pic_height_in_luma_samples.
[0045] The width PicOutputWidthL and height PicOutputHeightL of the output picture are derived as follows. PicOutputWidthL=pic_width_in_luma_samples- (scaling_win_right_offset+scaling_win_left_offset) PicOutputHeightL=pic_height_in_pic_size_units- (scaling_win_bottom_offset+scaling_win_top_offset) pps_collocated_from_l0_idc: This is a syntax indicating whether collocated_from_l0_flag exists in the slice header of the slice that refers to this PPS. If the value of this syntax is 0, it indicates that collocated_from_l0_flag exists in the slice header, and if the value of this syntax is 1 or 2, it indicates that collocated_from_l0_flag does not exist in the slice header. (Encoded image)
[0046] In the coded picture, a set of data is defined for reference by the moving image decoding device 31 in order to decode the picture PICT to be processed. Figure 4 As shown, the picture PICT includes a picture header PH, and slices 0 to NS-1 (NS is the total number of slices included in the picture PICT).
[0047] In the following, when it is not necessary to distinguish each of slices 0 to NS-1, the code suffix may be omitted for description. The same is true for data included in the coded stream Te described below, that is, other data to which the suffix is attached.
[0048] The picture header includes the following syntax. pic_temporal_mvp_enabled_flag: This is a flag that specifies whether to use temporal motion vector prediction in inter prediction of the slice associated with this picture header. If the value of this flag is 0, the syntax elements of the slice associated with this picture header are restricted so that temporal motion vector prediction is not used in decoding of the slice. If the value of this flag is 1, it indicates that temporal motion vector prediction is used in decoding of the slice associated with this picture header. In addition, if this flag is not specified, it is estimated to be 0. (Encoding slice)
[0049] In the coded slice, a set of data is defined to be referred to by the video decoding device 31 in order to decode the slice S to be processed. Figure 4 As shown, a slice includes a slice header and slice data.
[0050] The slice header includes a coding parameter group that the moving picture decoding device 31 refers to in order to determine a decoding method of a target slice. Slice type designation information (slice_type) that designates a slice type is one example of a coding parameter included in the slice header.
[0051] Examples of slice types that can be specified by the slice type specification information include: (1) I slices that use only intra-frame prediction when encoding, (2) P slices that use unidirectional prediction (L0 prediction) or intra-frame prediction when encoding, and (3) B slices that use unidirectional prediction (L0 prediction or L1 prediction), bidirectional prediction, or intra-frame prediction when encoding. It should be noted that inter-frame prediction is not limited to unidirectional prediction and bidirectional prediction, and more reference pictures can be used to generate predicted images. Hereinafter, the cases referred to as P and B slices refer to slices that include blocks that can use inter-frame prediction.
[0052] It should be noted that the slice header may also include a reference to the picture parameter set PPS (pic_parameter_set_id).
[0053] In addition, if Fig. 9 As shown, the slice header may include the following syntax. collocated_from_l0_flag: This is a flag that specifies from which reference picture list the collocated picture used for temporal motion vector prediction is derived. When the value of this flag is 1, it indicates that the picture is derived from reference picture list 0 (L0 list), and when the value of this flag is 0, it indicates that the picture is derived from reference picture list 1 (L1 list). In addition, when this flag is not specified, a value of 0 or 1 is estimated according to the slice type and pps_collocated_from_l0_idc.
[0054] When the slice type is other than B, collocated_from_10_flag is estimated to be 1. When the slice type is B, collocated_from_10_flag is estimated to be pps_collocated_from_10_idc-1. collocated_ref_idx: is the reference index of the co-located picture used for temporal motion vector prediction. In addition, the value of collocated_ref_idx is specified by the slice type or the value of collocated_from_l0_flag. When the slice type is P, or when the slice type is B and collocated_from_l0_flag is 1, collocated_ref_idx refers to the picture of List0. When the slice type is B and collocated_from_l0_flag is 0, collocated_ref_idx refers to the picture of List1. When collocated_ref_idx does not exist, it is estimated to be 0.
[0055] In this embodiment, if the value of the variable RefPicIsScaled[collocated_from_10_flag?0:1][collocated_ref_idx] indicating that the size of the coded picture is different from the size of the reference picture is 1, the variable SliceTemporalMvpEnableFlag is set to 0. Otherwise, the variable SliceTemporalMvpEnableFlag is set to the value of pic_temporal_mvp_enabled_flag. Therefore, when collocated_ref_idx specifies a reference picture whose size is different from that of the coded picture, the variable SliceTemporalMvpEnableFlag is forcibly set to 0, and encoding and decoding processing is performed in the slice without using motion vector information of the reference picture.
[0056] Fig.10The above-mentioned processing is shown in a flowchart. First. The value of collocated_from_l0_flag, which is a flag that specifies from which reference picture list of L0 and L1 the connected collocated picture (collocated picture) used for temporal motion vector prediction is derived, is substituted into the variable X. (S201) Next, in order to check whether the size of the collocated picture indicated by collocated_ref_idx is different from the size of the encoded picture, compare whether the value of the variable RefPicIsScaled is 1. (S202) If it is 1, it is determined that the size of the collocated picture is different from the size of the encoded picture, and the variable SliceTemporalMvpEnableFlag is set to 0, and the motion vector information of the collocated picture is not used. (S203) If it is not 1, the value of picTemporalMvpEnableFlag, which is a flag in the picture level, is substituted into the variable SliceTemporalMvpEnableFlag. (S204) As another implementation, it can be as follows: if a reference picture whose size is different from that of the encoded picture is specified by collocated_ref_idx, the minimum value of ref_idx is set to collocated_ref_idx in the reference picture of the same size as that of the encoded picture; if there is no reference picture of the same size as that of the encoded picture, the variable SliceTemporalMvpEnableFlag is forcibly set to 0, and in this slice, encoding and decoding processing is performed without using the motion vector information of the reference picture.
[0057] In any case, when collocated_ref_idx specifies a reference picture whose size is different from that of the coded picture, the operation of the decoding process is always set to be the same.
[0058] In this way, rather than prohibiting the generation of coded data for a reference picture whose size specified by collocated_ref_idx is different from that of the coded picture, the processing when a reference picture whose size is different from that of the coded picture is specified by collocated_ref_idx is defined, thereby ensuring that the decoding processing operation is always the same. (Encoded slice data)
[0059] The coded slice data specifies a set of data that the moving image decoding device 31 refers to in order to decode the slice data to be processed. Figure 4As shown in the coded slice header of , the slice data includes CTU. CTU is a fixed-size (for example, 64×64) block constituting a slice, and is also called the Largest Coding Unit (LCU). (Coding Tree Unit)
[0060] exist Figure 4 In the specification, a set of data is specified for reference by the motion picture decoding device 31 in order to decode the CTU of the processing object. The CTU is divided into coding units CU, which are the basic units of encoding processing, by recursive quadtree partitioning (QT (Quad Tree) partitioning), binary tree partitioning (BT (Binary Tree) partitioning) or ternary tree partitioning (TT (Ternary Tree) partitioning). BT partitioning and TT partitioning are collectively referred to as multi-tree partitioning (MT (Multi Tree) partitioning). The nodes of the tree structure obtained by recursive quadtree partitioning are called coding nodes (Coding Node). The intermediate nodes of the quadtree, binary tree and ternary tree are coding nodes, and the CTU itself is also specified as the top-level coding node.
[0061] CT includes the following information as CT information: a CU split flag (split_cu_flag) indicating whether CT splitting is performed, a QT split flag (qt_split_cu_flag) indicating whether QT splitting is performed, an MT split direction (mtt_split_cu_vertical_flag) indicating the split direction of MT splitting, and an MT split type (mtt_split_cu_binary_flag) indicating the split type of MT splitting. split_cu_flag, qt_split_cu_flag, mtt_split_cu_vertical_flag, and mtt_split_cu_binary_flag are transmitted for each coding node.
[0062] Different trees can be used for brightness and color difference. The tree type is indicated by treeType. For example, when a common tree is used for brightness (Y, cIdx=0) and color difference (Cb / Cr, cIdx=1,2), a common single tree is indicated by treeType=SINGLE_TREE. When two different trees (DUAL trees) are used for brightness and color difference, the tree for brightness is indicated by treeType=DUAL_TREE_LUMA, and the tree for color difference is indicated by treeType=DUAL_TREE_CHROMA. (Coding unit)
[0063] exist Figure 4In the CU header, a set of data is specified for reference by the video decoding device 31 to decode the coding unit to be processed. Specifically, the CU is composed of a CU header CUH, prediction parameters, transformation parameters, quantized transformation coefficients, etc. The prediction mode, etc. are specified in the CU header.
[0064] The prediction process may be performed in units of CU or in units of sub-CUs obtained by further dividing the CU. When the size of the CU and the sub-CU are equal, there is one sub-CU in the CU. When the size of the CU is larger than the size of the sub-CU, the CU is divided into sub-CUs. For example, when the CU is 8×8 and the sub-CU is 4×4, the CU is divided into 4 sub-CUs, including two parts divided horizontally and two parts divided vertically.
[0065] There are two types of prediction (prediction modes): intra prediction and inter prediction. Intra prediction is prediction within the same picture, while inter prediction refers to prediction processing between different pictures (for example, between display times, between layer images).
[0066] The transform / quantization process is performed in units of CUs, but the quantized transform coefficients may be entropy-coded in units of 4×4 or other sub-blocks. (Prediction parameters)
[0067] The predicted image is derived from prediction parameters attached to the block. The prediction parameters include prediction parameters for intra-frame prediction and inter-frame prediction.
[0068] The following describes the prediction parameters for inter-frame prediction. The inter-frame prediction parameters are composed of the prediction list utilization flags predFlagL0 and predFlagL1, reference picture indexes refIdxL0 and refIdxL1, and motion vectors mvL0 and mvL1. predFlagL0 and predFlagL1 are flags indicating whether to use the reference picture list (L0 list, L1 list). When the value is 1, the corresponding reference picture list is used. It should be noted that in the present specification, when it is recorded as "a flag indicating whether it is ××", the flag is set to be ×× when it is other than 0 (for example, 1), and the flag is set to be not ×× when it is 0. In logical NOT, logical product, etc., 1 is regarded as true and 0 is regarded as false (the same applies below). However, other values can also be used as true values and false values in actual devices and methods.
[0069] Among the syntax elements used to derive inter-frame prediction parameters, there are, for example, the affine flag affine_flag used in the merge mode, the merge flag merge_flag, the merge index merge_idx, the MMVD flag mmvd_flag, the inter-frame prediction identifier inter_pred_idc used in the AMVP mode for selecting a reference picture, the reference picture index refIdxLX, the prediction vector index mvp_LX_idx used to derive the motion vector, the differential vector mvdLX, and the motion vector accuracy mode amvr_mode. (See picture list)
[0070] The reference picture list is a list consisting of reference pictures stored in the reference picture memory 306 . Figure 6 is a conceptual diagram showing an example of a reference picture and a reference picture list. Figure 6 In the conceptual diagram of an example of a reference picture, a rectangle represents a picture, an arrow represents a reference relationship between pictures, the horizontal axis represents time, I, P, and B in the rectangle represent intra-frame pictures, single-prediction pictures, and dual-prediction pictures, respectively, and the numbers in the rectangle represent the decoding order. Figure 6 As shown in (a), the decoding order of pictures is I0, P1, B2, B3, B4, and the display order is I0, B3, B2, B4, P1. Figure 6 An example of a reference picture list for picture B3 (target picture) is shown in FIG. A reference picture list is a list indicating candidates for reference pictures, and one picture (slice) may have more than one reference picture list. Figure 6 In the example of (b), the target picture B3 has two reference picture lists, namely, L0 list RefPicList0 and L1 list RefPicList1. In each CU, refIdxLX specifies which picture in the reference picture list RefPicListX (X=0 or 1) is actually referenced. Figure 6 (b) is an example of refIdxL0 = 2, refIdxL1 = 0. It should be noted that LX is a description method used when L0 prediction and L1 prediction are not distinguished. In the following, LX is replaced with L0 and L1 to distinguish parameters for the L0 list and parameters for the L1 list. (Combined forecast and AMVP forecast)
[0071] The decoding (encoding) methods of prediction parameters include merge prediction mode and AMVP (Advanced Motion Vector Prediction) mode, and merge_flag is a flag used to identify them. The merge prediction mode is a mode in which the prediction list is not included in the encoded data using the flag predFlagLX, the reference picture index refIdxLX, and the motion vector mvLX, but is derived from the prediction parameters of the processed nearby blocks. The AMVP mode is a mode in which inter_pred_idc, refIdxLX, and mvLX are included in the encoded data. It should be noted that mvLX is encoded as mvp_LX_idx that identifies the prediction vector mvpLX and the differential vector mvdLX. In addition, in addition to the merge prediction mode, there can also be an affine prediction mode and an MMVD prediction mode.
[0072] inter_pred_idc is a value indicating the type and number of reference pictures, and takes any value of PRED_L0, PRED_L1, or PRED_BI. PRED_L0 and PRED_L1 respectively indicate unidirectional prediction using one reference picture managed in the L0 list and the L1 list. PRED_BI indicates bidirectional prediction using two reference pictures managed in the L0 list and the L1 list.
[0073] merge_idx is an index indicating whether any prediction parameter among prediction parameter candidates (merge candidates) derived from a block for which processing has been completed is to be used as a prediction parameter of the target block. (Motion Vector)
[0074] mvLX represents the amount of shift between blocks in two different pictures. The prediction vector and difference vector related to mvLX are referred to as mvpLX and mvdLX, respectively. (Inter prediction identifier inter_pred_idc and prediction list use flag predFlagLX)
[0075] The relationship between inter_pred_idc and predFlagL0 and predFlagL1 can be converted to each other as follows. inter_pred_idc=(predFlagL1<<1)+predFlagL0 predFlagL0=inter_pred_idc&1 predFlagL1=inter_pred_idc>>1
[0076] It should be noted that the inter-frame prediction parameter may use a prediction list utilization flag or an inter-frame prediction identifier. In addition, the determination using the prediction list utilization flag may be replaced by the determination using the inter-frame prediction identifier. Conversely, the determination using the inter-frame prediction identifier may also be replaced by the determination using the prediction list utilization flag. (Bidirectional prediction biPred determination)
[0077] The flag biPred indicating whether bidirectional prediction is used can be derived from whether both prediction list utilization flags are 1. For example, it can be derived by the following formula. biPred=(predFlagL0==1&&predFlagL1==1)
[0078] Alternatively, biPred can also be derived based on whether the inter-frame prediction identifier is a value indicating that two prediction lists (reference pictures) are used. For example, it can be derived using the following formula. biPred=(inter_pred_idc==PRED_BI)? 1:0 (Configuration of Moving Image Decoding Device)
[0079] The video decoding device 31 ( Figure 7 ) is described below.
[0080] The moving picture decoding device 31 is configured to include: an entropy decoding unit 301, a parameter decoding unit (prediction image decoding device) 302, a loop filter 305, a reference picture memory 306, a prediction parameter memory 307, a prediction image generation unit (prediction image generation device) 308, an inverse quantization / inverse transformation unit 311 and an addition unit 312, and a prediction parameter derivation unit 320. It should be noted that according to the moving picture encoding device 11 described later, there is also a configuration in which the loop filter 305 is not included in the moving picture decoding device 31.
[0081] The parameter decoding unit 302 also includes a header decoding unit 3020, a CT information decoding unit 3021, and a CU decoding unit 3022 (prediction mode decoding unit), and the CU decoding unit 3022 also includes a TU decoding unit 3024. They can also be collectively referred to as decoding modules. The header decoding unit 3020 decodes parameter set information such as VPS, SPS, PPS, APS, and slice header (slice information) from the encoded data. The CT information decoding unit 3021 decodes CT from the encoded data. The CU decoding unit 3022 decodes CU from the encoded data. The TU decoding unit 3024 decodes QP update information (quantization correction value) and quantization prediction error (residual_coding) from the encoded data when the prediction error is included in the TU.
[0082] The TU decoding unit 3024 decodes QP update information and quantization prediction error from the coded data when the skip mode is other than (skip_mode==0). More specifically, when skip_mode==0, the TU decoding unit 3024 decodes the flag cu_cbp indicating whether the target block includes the quantization prediction error, and decodes the quantization prediction error when cu_cbp is 1. When cu_cbp does not exist in the coded data, it is derived as 0.
[0083] The TU decoding unit 3024 decodes the index mts_idx indicating the transform basis from the coded data. In addition, the TU decoding unit 3024 decodes the index stIdx indicating the use of the secondary transform and the transform basis from the coded data. In the case of stIdx, 0 indicates that the secondary transform is not applied, in the case of 1, it indicates the transform of one of the set (pair) of secondary transform bases, and in the case of 2, it indicates the transform of the other of the pair.
[0084] In addition, the TU decoding unit 3024 may also decode the sub-block transform flag cu_sbt_flag. When cu_sbt_flag is 1, the CU is divided into multiple sub-blocks, and only the residual of a specific sub-block is decoded. In addition, the TU decoding unit 3024 may also decode the flag cu_sbt_quad_flag indicating whether the number of sub-blocks is 4 or 2, cu_sbt_horizontal_flag indicating the direction of division, and cu_sbt_pos_flag indicating a sub-block including a non-zero transform coefficient.
[0085] The predicted image generation unit 308 is configured to include an inter-predicted image generation unit 309 and an intra-predicted image generation unit 310 .
[0086] The prediction parameter derivation unit 320 is configured to include an inter-frame prediction parameter derivation unit 303 and an intra-frame prediction parameter derivation unit 304 .
[0087] In addition, the following describes an example of using CTU and CU as processing units, but the present invention is not limited thereto, and processing may be performed in sub-CU units. Alternatively, CTU and CU may be replaced with blocks, and sub-CU may be replaced with sub-blocks, and processing may be performed in blocks or sub-blocks.
[0088] The entropy decoding unit 301 performs entropy decoding on the coded stream Te input from the outside, and decodes each code (syntax element). There are the following methods in entropy coding: a method of variable-length coding the syntax elements using a context (probability model) appropriately selected according to the type of syntax element and the surrounding conditions; and a method of variable-length coding the syntax elements using a predetermined table or calculation formula. The former CABAC (Context Adaptive Binary Arithmetic Coding) stores the CABAC state of the context (the category of the dominant symbol (0 or 1) and the probability state index pStateIdx of the specified probability) in the memory. The entropy decoding unit 301 initializes all CABAC states at the starting point of the segment (tile, CTU row, slice). The entropy decoding unit 301 transforms the syntax element into a binary string (Bin String) and decodes each bit of the binary string. When using the context, the context index ctxInc is derived for each bit of the syntax element, the bit is decoded using the context, and the CABAC state of the context used is updated. Bits that do not use context are decoded with equal probability (EP, bypass), and ctxInc derivation and CABAC state are omitted. The decoded syntax elements include prediction information for generating a predicted image and a prediction error for generating a differential image.
[0089] The entropy decoding unit 301 outputs the decoded code to the parameter decoding unit 302. The decoded code is, for example, prediction mode predMode, merge_flag, merge_idx, inter_pred_idc, refIdxLX, mvp_LX_idx, mvdLX, amvr_mode, etc. Based on the instruction of the parameter decoding unit 302, control is performed on which code is decoded. (Basic process)
[0090] Figure 8 This is a flowchart for explaining the schematic operation of the moving picture decoding device 31 .
[0091] (S1100: Parameter set information decoding) The header decoding unit 3020 decodes parameter set information such as VPS, SPS, and PPS from the encoded data.
[0092] (S1200: Slice Information Decoding) The header decoding unit 3020 decodes a slice header (slice information) from the encoded data.
[0093] Thereafter, the video decoding device 31 derives a decoded image of each CTU by repeatedly performing the processes from S1300 to S5000 for each CTU included in the target picture.
[0094] (S1300: CTU Information Decoding) The CT information decoding unit 3021 decodes CTU from the encoded data.
[0095] (S1400: CT Information Decoding) The CT information decoding unit 3021 decodes CT from the encoded data.
[0096] (S1500: CU Decoding) The CU decoding unit 3022 implements S1510 and S1520 to decode a CU from the encoded data.
[0097] (S1510: CU Information Decoding) The CU decoding unit 3022 decodes CU information, prediction information, TU split flag split_transform_flag, CU residual flags cbf_cb, cbf_cr, cbf_luma, etc. from the encoded data.
[0098] (S1520: TU information decoding) When the TU includes a prediction error, the TU decoding unit 3024 decodes QP update information, quantization prediction error, and transform index mts_idx from the encoded data. It should be noted that the QP update information is a difference value from the quantization parameter prediction value qPpred which is a prediction value of the quantization parameter QP.
[0099] (S2000: Prediction Image Generation) The prediction image generation unit 308 generates a prediction image for each block included in the target CU based on the prediction information.
[0100] (S3000: Inverse Quantization / Inverse Transformation) The inverse quantization / inverse transformation unit 311 performs an inverse quantization / inverse transformation process on each TU included in the target CU.
[0101] (S4000: Decoded Image Generation) The addition unit 312 generates a decoded image of the target CU by adding the predicted image supplied from the predicted image generation unit 308 and the prediction error supplied from the inverse quantization / inverse transformation unit 311 .
[0102] (S5000: Loop Filtering) The loop filter 305 applies loop filtering such as deblocking filtering, SAO, and ALF to the decoded image to generate a decoded image. (Configuration of Inter-frame Prediction Parameter Derivation Unit)
[0103] Fig.11, a schematic diagram showing the structure of the inter-frame prediction parameter derivation unit 303 of the present embodiment is shown. The inter-frame prediction parameter derivation unit 303 derives the inter-frame prediction parameters based on the syntax elements input from the parameter decoding unit 302, referring to the prediction parameters stored in the prediction parameter memory 307. In addition, the inter-frame prediction parameters are output to the inter-frame prediction image generation unit 309 and the prediction parameter memory 307. The inter-frame prediction parameter derivation unit 303 and its internal elements, such as the AMVP prediction parameter derivation unit 3032, the merged prediction parameter derivation unit 3036, the affine prediction unit 30372, the MMVD prediction unit 30373, the triangle prediction unit 30377, the DMVR unit 30537, and the MV addition unit 3038, are common units in the motion picture encoding device and the motion picture decoding device, and therefore, they may also be collectively referred to as a motion vector derivation unit (motion vector derivation device).
[0104] The scale parameter derivation unit 30378 derives the horizontal scaling ratio RefPicScale[i][j][0] of the reference picture, the vertical scaling ratio RefPicScale[i][j][1] of the reference picture, and RefPicIsScaled[i][j] indicating whether the reference picture is scaled. Here, i indicates whether the reference picture list is the L0 list or the L1 list, and j is set to the value of the L0 reference picture list or the L1 reference picture list, and the derivation is performed as shown below. RefPicScale[i][j][0]= ((fRefWidth<<14)+(PicOutputWidthL>>1)) / PicOutputWidthL RefPicScale[i][j][1]= ((fRefHeight<<14)+(PicOutputHeightL>>1)) / PicOutputHeightLRefPicIsScaled[i][j]= (RefPicScale[i][j][0]!=(1<<14))||(RefPicScale[i][j][1]!=(1<<14))
[0105] Here, the variable PicOutputWidthL is a value obtained by subtracting the left and right offset values from the number of pixels in the horizontal direction of the brightness of the encoded picture when calculating the horizontal scaling ratio when referring to the encoded picture. The variable PicOutputHeightL is a value obtained by subtracting the upper and lower offset values from the number of pixels in the vertical direction of the brightness of the encoded picture when calculating the vertical scaling ratio when referring to the encoded picture. The variable fRefWidth is set to the value of PicOutputWidthL of the reference picture of reference list value j of list i, and the variable fRefHight is set to the value of PicOutputHeightL of the reference picture of reference list value j of list i.
[0106] When affine_flag is 1, indicating the affine prediction mode, the affine prediction unit 30372 derives the inter-frame prediction parameters in units of sub-blocks.
[0107] When mmvd_flag is 1, indicating the MMVD prediction mode, the MMVD prediction unit 30373 derives the inter-frame prediction parameters from the merge candidate and the differential vector derived by the merge prediction parameter derivation unit 3036 .
[0108] When TriangleFlag is 1, indicating the triangle prediction mode, the triangle prediction unit 30377 derives triangle prediction parameters.
[0109] When merge_flag is 1, indicating the merge prediction mode, merge_idx is derived and output to the merge prediction parameter derivation unit 3036 .
[0110] When merge_flag is 0, indicating the AMVP prediction mode, the AMVP prediction parameter derivation unit 3032 derives mvpLX from inter_pred_idc, refIdxLX or mvp_LX_idx. (MV addition)
[0111] The derived mvpLX and mvdLX are added in the MV adding unit 3038 to derive mvLX. (Affine prediction unit)
[0112] In the affine prediction unit 30372, 1) the motion vectors of two control points CP0, CP1 or three control points CP0, CP1, CP2 of the object block are derived, 2) the affine prediction parameters of the object block are derived, and 3) the motion vectors of each sub-block are derived from the affine prediction parameters.
[0113] In the case of merged affine prediction, the motion vector cpMvLX[] of each control point CP0, CP1, CP2 is derived from the motion vector of the adjacent block of the object block. In the case of inter-frame affine prediction, cpMvLX[] of each control point is derived from the sum of the prediction vector of each control point CP0, CP1, CP2 and the differential vector mvdCpLX[] derived from the encoded data.
[0114] Fig.13 1 is a diagram showing an example of deriving motion vectors spMvLX of each sub-block constituting the target block (bW*bH) from motion vectors cpMvLX[0] of control points CP0, cpMvLX[1] of CP1, and cpMvLX[2] of CP2. Fig.13 As shown, spMvLX is derived as the motion vector of the point (xPosCb, yPosCb) located at the center of each sub-block.
[0115] The affine prediction unit 30372 derives the affine prediction parameters (mvScaleHor, mvScalerVer, dHorX, dHorY, dHorX, dVerY) of the target block from the motion vector of the control point. When mvScaleHor = cpMvLX[0][0] << 7 mvScaleVer=cpMvLX[0][1]<<7 dHorX=(cpMvLX[1][0]-cpMvLX[0][0])<<(7-log2CbW) dVerX=(cpMvLX[1][1]-cpMvLX[0][1])<<(7-log2CbW) When numCpMv==3, In dHorY=(cpMvLX[2][0]-cpMvLX[0][0])<<(7-log2CbH) dVerY=(cpMvLX[2][1]-cpMvLX[0][1])<<(7-log2CbH) numCpMv! =3, dHorY=-dVerX dVerY=dHorX
[0116] Here, log2CbW and log2CbH are logarithmic values of the width bW and the height bH of the target block.
[0117] The affine prediction unit 30372 uses the following formula to derive spMvLX[i][j] (i=0, 1, 2...(bW / sbW)-1, j=0, 1, 2...(bH / sbH)-1) in the object block based on the affine prediction parameters of the object block. xPosCb=2+(i<<2) yPosCb=2+(j<<2) spMvLX[i][j][0]=mvScaleHor+dHorX*xPosCb+dHorY*yPosCb spMvLX[i][j][1]=mvScaleVer+dVerX*xPosCb+dVerY*yPosCb
[0118] It can also be further cropped by performing circular shifting. spMvLX[i][j][0]=Clip3(-2^17, 2^17, (spMvLX[i][j][0]+64)>>7) spMvLX[i][j][1]=Clip3(-2^17, 2^17, (spMvLX[i][j][1]+64)>>7)
[0119] Here, sbW and sbH are the width and height of the target sub-block.
[0120] Then, at the coordinates (xSb, ySb) of the upper left block of the sub-block, spMvLX[i][j] is assigned to mvLX in the corresponding picture. Here, x=0..sbW-1, y=0..sbH-1. mvLX[xSb+x][yXb+y]=spMv[i][j] (Combined Forecast)
[0121] Fig.12 , a schematic diagram showing the structure of the merge prediction parameter derivation unit 3036 of the present embodiment is shown. The merge prediction parameter derivation unit 3036 includes a merge candidate derivation unit 30361 and a merge candidate selection unit 30362. It should be noted that the merge candidate structure includes prediction parameters (predFlagLX, mvLX, refIdxLX) and is stored in a merge candidate list. An index is assigned to a merge candidate stored in the merge candidate list according to a prescribed rule.
[0122] The merge candidate derivation unit 30361 directly uses the decoded motion vector of the adjacent block and refIdxLX to derive the merge candidate. In addition, the merge candidate derivation unit 30361 can apply the spatial merge candidate derivation process, the temporal merge candidate derivation process, the paired merge candidate derivation process, and the zero merge candidate derivation process described later.
[0123] As a spatial merge candidate derivation process, the merge candidate derivation unit 30361 reads the prediction parameters stored in the prediction parameter memory 307 according to a predetermined rule and sets them as merge candidates. In the method of specifying the reference picture, for example, the prediction parameters of each adjacent block (for example, all or part of the blocks adjacent to the left A1, right B1, upper right B0, lower left A0, and upper left B2 of the object block) within a predetermined range from the object block are used. Each merge candidate is called A1, B1, B0, A0, and B2. Here, A1, B1, B0, A0, and B2 are motion information derived from blocks containing the following coordinates. Fig. 9 In the object picture, the positions of A1, B1, B0, A0, and B2 are shown in the configuration of merge candidates. A1: (xCb-1, yCb+cbHeight-1) B1: (xCb+cbWidth-1, yCb-1) B0: (xCb+cbWidth, yCb-1) A0: (xCb-1, yCb+cbHeight) B2: (xCb-1, yCb-1)
[0124] Set the upper left coordinate of the object block to (xCb, yCb), the width to cbWidth, and the height to cbHeight.
[0125] As a time-merged derivation, Fig. 9 As shown in the co-located picture, the merge candidate derivation unit 30361 reads the prediction parameter of the lower right CBR of the object block or the block C in the reference image including the central coordinates from the prediction parameter memory 307 to set it as the merge candidate Col, and stores it in the merge candidate list mergeCandList[].
[0126] Generally, block CBR is added to mergeCandList[] first. If CBR does not have a motion vector (e.g., an intra-frame prediction block) or is located outside the picture, the motion vector of block C is added to the prediction vector candidate. By adding the motion vectors of the same-position blocks with a high probability of different motion as prediction candidates, the options for prediction vectors are increased, and the coding efficiency is improved.
[0127] When SliceTemporalMvpEnabledFlag is 0 or cbWidth*cbHeight is less than or equal to 32, the co-located motion vector mvLXCol of the target block is set to 0, and the availability flag availableFlagLXCol of the co-located block is set to 0.
[0128] In other cases (SliceTemporalMvpEnabledFlag is 1), the following processing is performed.
[0129] For example, the merge candidate derivation unit 30361 can derive the position of C (xColCtr, yColCtr) and the position of CBR (xColCBr, yColCBr) by the following formula. xColCtr=xCb+(cbWidth>>1) yColCtr=yCb+(cbHeight>>1) xColCBr=xCb+cbWidth yColCBr=yCb+cbHeight
[0130] If CBR is available, the merge candidate COL is derived using the motion vector of CBR. If CBR is not available, COL is derived using C. Then, availableFlagLXCol is set to 1. It should be noted that the reference picture may also be the collocated_ref_idx notified in the slice header.
[0131] The paired candidate derivation unit derives the paired candidate avgK from the average value of the two merge candidates (p0Cand, p1Cand) stored in mergeCandList, and stores it in mergeCandList[]. mvLXavgK[0]=(mvLXp0Cand[0]+mvLXp1Cand[0]) / 2 mvLXavgK[1]=(mvLXp0Cand[1]+mvLXp1Cand[1]) / 2
[0132] The merge candidate derivation unit 30361 derives zero merge candidates Z0...ZM in which refIdxLX is 0...M and X and Y components of mvLX are both 0, and stores them in the merge candidate list.
[0133] The order stored in mergeCandList[] is, for example, spatial merge candidates (A1, B1, B0, A0, B2), temporal merge candidates Col, paired candidates avgK, and zero merge candidates ZK. It should be noted that unusable reference blocks (blocks that are intra-predicted, etc.) are not stored in the merge candidate list. i=0 if(availableFlagA1) mergeCandList[i++]=A1 if(availableFlagB1) mergeCandList[i++]=B1 if(availableFlagB0) mergeCandList[i++]=B0 if(availableFlagA0) mergeCandList[i++]=A0 if(availableFlagB2) mergeCandList[i++]=B2 if(availableFlagCol) mergeCandList[i++] = Col if(availableFlagAvgK) mergeCandList[i++] = avgK if(i <MaxNumMergeCand) mergeCandList[i++] = ZK
[0134] The merge candidate selection unit 30362 selects a merge candidate N indicated by merge_idx from among the merge candidates included in the merge candidate list using the following formula. N = mergeCandList[merge_idx]
[0135] Here, N is a label indicating a merge candidate, and may be A1, B1, B0, A0, B2, Col, avgK, ZK, etc. The motion information of the merge candidate indicated by label N is represented by (mvLXN[0], mvLXN[0]), predFlagLXN, and refIdxLXN.
[0136] The selected (mvLXN[0], mvLXN[0]), predFlagLXN, refIdxLXN are selected as the inter prediction parameters of the target block. The merge candidate selection unit 30362 stores the inter prediction parameters of the selected merge candidate in the prediction parameter memory 307 and outputs them to the inter prediction image generation unit 309. (DMVR)
[0137] Next, the DMVR (Decoder side Motion Vector Refinement) processing performed by the DMVR unit 30537 is described. When merge_flag is 1 for the target CU or when the skip flag skip_flag is 1, the DMVR unit 30537 uses the reference image to correct the mvLX of the target CU derived by the merge prediction unit 30374. Specifically, when the prediction parameter derived by the merge prediction unit 30374 is a bidirectional prediction, the motion vector is corrected using a predicted image derived from the motion vectors corresponding to the two reference pictures. The corrected mvLX is supplied to the inter-frame prediction image generation unit 309.
[0138] Furthermore, in the derivation of the flag dmvrFlag that specifies whether to perform DMVR processing, one of the multiple conditions for setting dmvrFlag to 1 includes the value of RefPicIsScaled[0][refIdxL0] being 0 and the value of RefPicIsScaled[1][refIdxL1] being 0. When the value of dmvrFlag is set to 1, DMVR processing by the DMVR unit 30537 is performed.
[0139] Furthermore, in the derivation of the flag dmvrFlag that specifies whether to perform DMVR processing, as one of the conditions for setting dmvrFlag to 1, luma_weight_l0_flag[i], which is a flag indicating whether there is coefficient information of weighted prediction of L0 prediction of luma described later, is 0, and luma_weight_l1_flag[i], which is a flag indicating whether there is coefficient information of weighted prediction of L1 prediction of luma, is 0. When the value of dmvrFlag is set to 1, DMVR processing by the DMVR unit 30537 is performed.
[0140] It should be noted that, in the derivation of the flag dmvrFlag that specifies whether to perform DMVR processing, as one of the multiple conditions for setting dmvrFlag to 1, luma_weight_l0_flag[i] is 0, and the value of luma_weight_l1_flag[i] is 0, and the value of chroma_weight_l0_flag[i] as a flag indicating whether there is weighted prediction coefficient information of L0 prediction of chroma difference described later is 0, and the value of chroma_weight_l1_flag[i] as a flag indicating whether there is weighted prediction coefficient information of L1 prediction of chroma difference is 0. When the value of dmvrFlag is set to 1, the DMVR processing by the DMVR unit 30537 is performed. (Prof)
[0141] Furthermore, if the value of RefPicIsScaled[0][refIdxLX] is 1 and the value of RefPicIsScaled[1][refIdxLX] is 1, the value of cbProfFlagLX is set to FALSE. Here, cbProfFlagLX is a flag that specifies whether prediction refinement (PROF) of affine prediction is performed. (AMVP prediction)
[0142] Fig.12 , a schematic diagram showing the structure of the AMVP prediction parameter derivation unit 3032 of the present embodiment is shown. The AMVP prediction parameter derivation unit 3032 includes a vector candidate derivation unit 3033 and a vector candidate selection unit 3034. The vector candidate derivation unit 3033 derives prediction vector candidates based on refIdxLX and the motion vectors of the decoded adjacent blocks stored in the prediction parameter memory 307, and stores them in the prediction vector candidate list mvpListLX[].
[0143] The vector candidate selection unit 3034 selects the motion vector mvpListLX[mvp_LX_idx] indicated by mvp_LX_idx from among the prediction vector candidates of mvpListLX[] as mvpLX. The vector candidate selection unit 3034 outputs the selected mvpLX to the MV addition unit 3038. (MV addition)
[0144] The MV adding unit 3038 adds the mvpLX input from the AMVP prediction parameter derivation unit 3032 and the decoded mvdLX to calculate mvLX. The adding unit 3038 outputs the calculated mvLX to the inter-frame prediction image generation unit 309 and the prediction parameter memory 307. mvLX[0]=mvpLX[0]+mvdLX[0] mvLX[1]=mvpLX[1]+mvdLX[1] (Detailed classification of sub-block merging)
[0145] The types of prediction processing associated with sub-block merging are summarized as follows. As described above, they are roughly divided into merge prediction and AMVP prediction.
[0146] The combined forecasts are further categorized as follows. Conventional merge prediction (block-based merge prediction) Sub-block merging prediction
[0147] Sub-block merging prediction is further categorized as follows. Sub-block prediction (ATMVP) Affine prediction Inferred affine prediction Constructed affine prediction
[0148] On the other hand, AMVP predictions are categorized as follows. AMVP (Translation) MVD affine prediction
[0149] MVD affine predictions are further classified as follows. ·4-parameter MVD affine prediction 6-parameter MVD affine prediction
[0150] It should be noted that MVD affine prediction refers to affine prediction that decodes and uses a differential vector.
[0151] In sub-block prediction, availabilityFlagSbCol of the co-located sub-block COL of the target sub-block is determined similarly to the temporal merge derivation process, and prediction parameters are derived if available. AvailableFlagSbCol is set to 0 at least when the SliceTemporalMvpEnabledFlag is 0.
[0152] MMVD prediction (Merge with Motion Vector Difference) can be classified as merge prediction or AMVP prediction. For the former, when merge_flag=1, mmvd_flag and MMVD-related syntax elements are decoded, and for the latter, when merge_flag=0, mmvd_flag and MMVD-related syntax elements are decoded.
[0153] The loop filter 305 is a filter provided in the encoding loop, and is a filter for improving image quality by removing block distortion and ringing distortion. The loop filter 305 performs filtering such as deblocking filtering, sample adaptive offset (SAO), and adaptive loop filtering (ALF) on the decoded image of the CU generated by the adding unit 312.
[0154] The reference picture memory 306 stores the decoded image of the CU at a predetermined position for each target picture and target CU.
[0155] The prediction parameter memory 307 stores the prediction parameters at a predetermined location for each CTU or CU. Specifically, the prediction parameter memory 307 stores the parameters decoded by the parameter decoding unit 302 and the parameters derived by the prediction parameter derivation unit 320 .
[0156] The predicted image generation unit 308 receives the parameters derived by the prediction parameter derivation unit 320. In addition, the predicted image generation unit 308 reads the reference picture from the reference picture memory 306. The predicted image generation unit 308 generates a predicted image of a block or a sub-block using the parameters and the reference picture (reference picture block) in the prediction mode indicated by predMode. Here, the reference picture block refers to a set of pixels on the reference picture (usually rectangular, so it is called a block), which is a region referenced to generate a predicted image. (Inter-frame prediction image generation unit 309)
[0157] When predMode indicates the inter prediction mode, the inter prediction image generation unit 309 generates a prediction image of a block or a subblock by inter prediction using the inter prediction parameters input from the inter prediction parameter derivation unit 303 and a reference picture.
[0158] Fig.14This is a schematic diagram showing the structure of the inter-frame prediction image generation unit 309 included in the prediction image generation unit 308 of this embodiment. The inter-frame prediction image generation unit 309 is configured to include a motion compensation unit (prediction image generation device) 3091 and a synthesis unit 3095. The synthesis unit 3095 is configured to include an intra-frame and inter-frame (IntraInter) synthesis unit 30951, a triangle synthesis unit 30952, a BDOF unit 30954, and a weighted prediction unit 3094. (Motion Compensation)
[0159] The motion compensation unit 3091 (interpolation image generation unit 3091) generates an interpolation image (motion compensation image) by reading a reference block from the reference picture memory 306 based on the inter prediction parameters (predFlagLX, refIdxLX, mvLX) input from the inter prediction parameter derivation unit 303. The reference block is a block that is shifted from the position of the target block by mvLX on the reference picture RefPicLX specified by refIdxLX. Here, when mvLX is not an integer precision, filtering for generating pixels at decimal positions, called motion compensation filtering, is performed to generate an interpolation image.
[0160] The motion compensation unit 3091 first derives the integer position (xInt, yInt) and phase (xFrac, yFrac) corresponding to the coordinates (x, y) in the prediction block through the following formula. xInt=xPb+(mvLX[0]>>(log2(MVPREC)))+x xFrac = mvLX[0] & (MVPREC - 1) yInt=yPb+(mvLX[1]>>(log2(MVPREC)))+y yFrac=mvLX[1]&(MVPREC-1)
[0161] Here, (xPb, yPb) is the upper left coordinate of the block of size bW*bH, x=0...bW-1, y=0...bH-1, and MVPREC represents the precision of mvLX (1 / MVPREC pixel precision). For example, MVPREC=16.
[0162] The motion compensation unit 3091 performs horizontal interpolation processing on the reference image refImg using an interpolation filter, thereby deriving a temporary image temp[][]. The following Σ is a sum related to k of k=0..NTAP-1, shift1 is a normalization parameter of the interval of the adjustment value, and offset1=1<<(shift1-1). temp[x][y]= (ΣmcFilter[xFrac][k]*refImg[xInt+k-NTAP / 2+1][yInt]+offset1)>>shift1
[0163] Next, the motion compensation unit 3091 performs vertical interpolation processing on the temporary image temp[][] to derive an interpolated image Pred[][]. Hereinafter, Σ is a sum related to k of k=0..NTAP-1, shift2 is a normalization parameter of the interval of the adjustment value, and offset2=1<<(shift2-1).
[0164] Pred[x][y]=(ΣmcFilter[yFrac][k]*temp[x][y+k-NTAP / 2+1]+offset2)>>shift2 It should be noted that in the case of bidirectional prediction, the above-mentioned Pred[][] (called interpolated images PredL0[][] and PredL1[][]) is derived according to each L0 list and L1 list, and Pred[][] is generated based on the interpolated images PredL0[][] and PredL1[][].
[0165] It should be noted that the motion compensation unit 3091 has the function of scaling the interpolated image according to the horizontal scaling ratio RefPicScale[i][j][0] of the reference image and the vertical scaling ratio RefPicScale[i][j][1] of the reference image derived by the scale parameter derivation unit 30378.
[0166] The synthesis unit 3095 includes an intra-frame and inter-frame synthesis unit 30951 , a triangle synthesis unit 30952 , a weighted prediction unit 3094 , and a BDOF unit 30954 . (Interpolation filter processing)
[0167] Hereinafter, the interpolation filter processing performed by the prediction image generation unit 308 , that is, the interpolation filter processing when the size of the reference picture changes in a single sequence by applying the above-mentioned resampling will be described. It should be noted that this processing can be performed by the motion compensation unit 3091 , for example.
[0168] When the value of RefPicIsScaled[i][j] input from the inter prediction parameter derivation unit 303 indicates that the reference picture is scaled, the prediction image generation unit 308 performs the following interpolation filter processing.
[0169] When correcting the size of the reference picture for luma, the prediction image generation unit 308 derives the value of the horizontal luma interpolation filter coefficient fLH[p] and the value of the vertical luma interpolation filter coefficient fLV[p]. Here, the value p is a value corresponding to each 1 / 16 position of the luma sample in the horizontal or vertical direction. The horizontal luma interpolation filter coefficient fLH[p] at the 1 / 16 pixel position p of xFracL or yFracL is derived as follows.
[0170] In affine prediction, when the block size is equal to 4 in both the horizontal and vertical directions, the luminance interpolation filter coefficient fLH[p] is used Fig.18 .
[0171] In all other cases, when scalingRatio[0] is greater than 28672, the luma interpolation filter coefficients fLH[p] are used. Fig.16 .
[0172] In all other cases, when scalingRatio[0] is greater than 20480, the luma interpolation filter coefficients fLH[p] are used. Fig.17 .
[0173] In other cases, the luminance interpolation filter coefficients fLH[p] are calculated according to hpelIfIdx. Fig.15 to specify.
[0174] The vertical luminance interpolation filter coefficient flV[p] for each 1 / 16 fractional sample position p equal to yFracL is derived as shown below.
[0175] In affine prediction, when the block size is equal to 4 in both the horizontal and vertical directions, the luminance interpolation filter coefficient flV[p] is used Fig.18 .
[0176] In all other cases, when scalingRatio[1] is greater than 28672, the luma interpolation filter coefficients flV[p] use Fig.16 .
[0177] In all other cases, when scalingRatio[1] is greater than 20480, the luma interpolation filter coefficients flV[p] are used. Fig.17 .
[0178] In all other cases, the luma interpolation filter coefficients flV[p] are used according to hpelIfIdx. Fig.15 .
[0179] When i=0..7, the integer pixel position of brightness is set to (xInti, yInti), the fractional pixel position of brightness is set to (xFracL, yFracL), the reference pixel arrangement refPicLXL of brightness, and the index of the 1 / 2 pixel interpolation filter is set to hpelIfIdx, and the pixel value predSampleLXL of the brightness interpolation image is derived as shown below.
[0180] When both xFracL and yFracL are equal to 0 and both scalingRatio[0] and scalingRatio[1] are less than 20481, the value of predSampleLXL is derived as follows: In the following, Σ represents the sum associated with i=0..7. predSampleLXL=refPicLXL[xInt3][yInt3]< <shift3
[0181] In other cases, when yFracL is 0 and scalingRatio[1] is less than 20481, the value of predSampleLXL is derived as shown below. predSampleLXL=(ΣfLH[xFracL][i]*refPicLXL[xInti][yInt3])>>shift1
[0182] In other cases, when xFracL is 0 and scalingRatio[0] is less than 20481, the value of predSampleLXL is derived as shown below. predSampleLXL=(ΣfLV[yFracL][i]*refPicLXL[xInt3][yInti])>>shift1
[0183] In other cases, the value of predSampleLXL is derived as follows.
[0184] The pixel arrangement temp[n] for n=0..7 is derived as shown below. temp[n]=(ΣfLH[xFracL][i]*refPicLXL[xInti][yIntn])>>shift1
[0185] The pixel value predSampleLXL of the luminance interpolation image is derived as follows. predSampleLXL=(ΣfLV[yFracL][i]*temp[i])>>shift2
[0186] In addition, when correcting the size of the reference picture for color difference, the predicted image generation unit 308 derives the value of the horizontal color difference interpolation filter coefficient fCH[p] and the value of the vertical color difference interpolation filter coefficient fCV[p]. The value p here is a value corresponding to each 1 / 32 position in the horizontal or vertical direction of the color difference sample. The horizontal color difference interpolation filter coefficient fCH[p] at the 1 / 32 pixel position p of xFracC is derived as shown below.
[0187] When scalingRatio[0] is greater than 28672, the chromatic aberration interpolation filter coefficient fCH[p] uses Fig.21 .
[0188] In all other cases, when scalingRatio[0] is greater than 20480, the color difference interpolation filter coefficient fCH[p] uses Fig. 20 .
[0189] In all other cases, the color difference interpolation filter coefficient fCH[p] is used Fig.19 .
[0190] The vertical color difference interpolation filter coefficient fCV[p] at each 1 / 32 fractional pixel position p equal to yFracC is derived as shown below.
[0191] When scalingRatio[1] is greater than 28672, the color difference interpolation filter coefficient fCV[p] uses Fig.21 .
[0192] In all other cases, when scalingRatio[1] is greater than 20480, the color difference interpolation filter coefficients fCV[p] are used Fig. 20 .
[0193] In all other cases, the chroma interpolation filter coefficients fCV[p] are used Fig.19 .
[0194] Assuming the integer pixel position of the chromatic aberration when i=0..3 to be (xInti, yInti), the fractional pixel position of the chromatic aberration (xFracL, yFracL) and the reference pixel arrangement refPicLXL of the chromatic aberration, the pixel value predSampleLXLC of the chromatic aberration interpolated image is derived as shown below.
[0195] The pixel value predSampleLXC of the color difference interpolation image is derived as follows.
[0196] When both xFracC and yFracC are 0 and both scalingRatio[0] and scalingRatio[1] are less than 20481, the value of predSampleLXC is derived as follows: Hereinafter, Σ represents the sum of i=0..3. predSampleLXC=refPicLXC[xInt1][yInt1]< <shift3
[0197] In other cases, when yFracC is 0 and scalingRatio[1] is less than 20481, the value of predSampleLXC is derived as shown below. predSampleLXC=(ΣfCH[xFracC][i]*refPicLXC[xInti][yInt1])>>shift1
[0198] In other cases, when xFracC is 0 and scalingRatio[0] is less than 20481, the value of predSampleLXC is derived as shown below. predSampleLXC=(ΣfCV[yFracC][i]*refPicLXC[xInt1][yInti])>>shift1
[0199] In other cases, the value of predSampleLXC is derived as follows:
[0200] The pixel arrangement temp[n] for n=0..3 is derived as follows. temp[n]=(ΣfCH[xFracC][i]*refPicLXC[xInti][yIntn])>>shift1
[0201] The pixel value predSampleLXLC of the color difference interpolation image is derived as follows: predSampleLXC = (fCV[yFracC][0]*temp[0]+fCV[yFracC][1]*temp[1]) +fCV[yFracC][2]*temp[2]+fCV[yFracC][3]*temp[3])>>shift2 (intra-frame and inter-frame synthesis processing)
[0202] The intra-frame inter-frame synthesis unit 30951 generates a predicted image by performing a weighted sum of the inter-frame predicted image and the intra-frame predicted image. (Triangle synthesis processing)
[0203] The triangle synthesis unit 30952 generates a predicted image using the above-mentioned triangle prediction. (BDOF forecast)
[0204] Next, the details of BDOF prediction (Bi-Directional Optical Flow: bidirectional optical flow, BDOF processing) performed by the BDOF unit 30954 are described. The BDOF unit 30954 generates a predicted image by referring to two predicted images (a first predicted image and a second predicted image) and a gradient correction item in a bidirectional prediction mode.
[0205] Fig.25 This is a flowchart illustrating the process of deriving a predicted image.
[0206] When the inter-frame prediction parameter decoding unit 303 determines that it is a unidirectional prediction of L0 (inter_pred_idc is 0 in S101), the motion compensation unit 3091 generates PredL0[x][y] (S102). When the inter-frame prediction parameter decoding unit 303 determines that it is a unidirectional prediction of L1 (inter_pred_idc is 1 in S101), the motion compensation unit 3091 generates PredL1[x][y] (S103). On the other hand, when the inter-frame prediction parameter decoding unit 303 determines that it is a bidirectional prediction mode (inter_pred_idc is 2 in S101), the following process of S104 is performed. In S104, the synthesis unit 3095 refers to bdofAvailableFlag indicating whether to perform BDOF processing to determine whether to perform BDOF processing. When bdofAvailableFlag indicates true (TRUE), the BDOF unit 30954 performs BDOF processing to generate a bidirectional prediction image (S106), and when bdofAvailableFlag indicates false, the synthesis unit 3095 generates a prediction image by normal bidirectional prediction image generation (S105).
[0207] In the derivation of the flag bdofAvailableFlag that specifies whether to perform BDOF processing, one of the multiple conditions for setting bdofAvailableFlag to 1 includes the value of RefPicIsScaled[0][refIdxL0] being 0 and the value of RefPicIsScaled[1][refIdxL1] being 0.
[0208] In addition, in the derivation of the flag bdofAvailableFlag that specifies whether BDOF processing is to be performed, as one of the multiple conditions for setting bdofAvailableFlag to 1, luma_weight_l0_flag[i], which is a flag indicating whether there is coefficient information of weighted prediction of L0 prediction of luminance described later, is 0, and luma_weight_l1_flag[i], which is a flag indicating whether there is coefficient information of weighted prediction of L1 prediction of luminance, is 0.
[0209] It should be noted that, in the derivation of the flag bdofAvailableFlag that specifies whether BDOF processing is performed, as one of the multiple conditions for setting bdofAvailableFlag to 1, the inter-frame prediction parameter decoding unit 303 may also include luma_weight_l0_flag[i] being 0, and the value of luma_weight_l1_flag[i] being 0, and the chroma_weight_l0_flag[i] as a flag indicating whether there is weighted prediction coefficient information of L0 prediction of chrominance described later is 0, and the value of chroma_weight_l1_flag[i] as a flag indicating whether there is weighted prediction coefficient information of L1 prediction of chrominance is 0.
[0210] use Fig.26 The specific processing performed by the BDOF unit 30954 is described below. The BDOF unit 30954 includes an L0, L1 prediction image generation unit 309541, a gradient image generation unit 309542, a related parameter calculation unit 309543, a motion compensation correction value derivation unit 309544, and a bidirectional prediction image generation unit 309545. The BDOF unit 30954 generates a prediction image based on the interpolation image received from the motion compensation unit 3091 and the inter-frame prediction parameter received from the inter-frame prediction parameter decoding unit 303, and outputs it to the addition unit 312. It should be noted that the process of deriving the motion compensation correction value bdofOffset (motion compensation correction image) from the gradient image and correcting and deriving PredL0 and PredL1 is called a bidirectional gradient change process.
[0211] In the L0, L1 prediction image generation unit 309541, interpolation images predSamplesL0 and predSamplesL1 of L0 and L1 predictions used for BDOF processing are generated.
[0212] It should be noted that the settings are: variable shift1 is 6, variable shift2 is 4, variable shift3 is 1, variable shift4 is Max(3,15-BitDepth), variable offset4 is 1<<(shift4-1), and variable mvRefineThres is 1<<4. predSamplesL0 is the interpolated image predicted by L0, and predSamplesL1 is the interpolated image predicted by L1.
[0213] When bdofAvailableFlag is false, the bidirectional prediction image generation unit 309545 derives the pixel value of the prediction image in units of 4×4 pixel blocks as shown below. pbSamples[x][y]=Clip3(0,(1< <BitDepth)-1, (predSamplesL0[x+1][y+1]+offset4+predSamplesL1[x+1][y+1])>>shift4)
[0214] In other cases (bdofAvailableFlag is equal to true), the pixel values of the current predicted image are derived as shown below.
[0215] The position (hx, vy) of the corresponding pixel position (x, y) in the prediction image array is derived as follows. hx=Clip3(1,W,x) vy=Clip3(1,H,y)
[0216] Here, W is the number of pixels in the horizontal direction of the processing block, and H is the number of pixels in the vertical direction.
[0217] The gradient image generation unit 309542 generates a gradient image. Specifically, the gradient image variables gradientHL0[x][y], gradientVL0[x][y], gradientHL1[x][y], gradientVL1[x][y] are derived as shown below. gradientHL0 and gradientHL1 represent the gradient images in the horizontal direction, and gradientVL0 and gradientVL1 represent the gradient images in the vertical direction. gradientHL0[x][y]=(predSamplesL0[hx+1][vy]>>shift1) -(predSampleL0[hx-1][vy])>>shift1) gradientVL0[x][y]=(predSampleL0[hx][vy+1]>>shift1) -(predSampleL0[hx][vy-1])>>shift1) gradientHL1[x][y]=(predSamplesL1[hx+1][vy]>>shift1) -(predSampleL1[hx-1][vy])>>shift1) gradientVL1[x][y]=(predSampleL1[hx][vy+1]>>shift1) -(predSampleL1[hx][vy-1])>>shift1)
[0218] In addition, the variables diff[x][y], tempH[x][y], and tempV[x][y] are derived. diff[x][y]=(predSamplesL0[hx][vy]>>shift2)-(predSamplesL1[hx][vy]>>shift2) tempH[x][y]=(gradientHL0[x][y]+gradientHL1[x][y])>>shift3 tempV[x][y]=(gradientVL0[x][y]+gradientVL1[x][y])>>shift3
[0219] Next, the correlation parameter calculation unit 309543 derives the gradient product and variables sGx2, sGy2, sGxGy, sGxdI, and sGydI for each 4×4 pixel block as shown below. sGx2=ΣΣabs(tempH[xSb+i][ySb+j]) sGy2=ΣΣabs(tempV[xSb+i][ySb+j]) sGxGy=ΣΣ(Sign(tempV[xSb+i][ySb+j])*tempH[xSb+i][ySb+j]) sGxdI=ΣΣ(-Sign(tempH[xSb+i][ySb+j])*diff[xSb+i][ySb+j]) sGydI=ΣΣ(-Sign(tempV[xSb+i][ySb+j])*diff[xSb+i][ySb+j])
[0220] Here, ΣΣ means finding the total value of a block of 6×6 pixels including 1 pixel surrounding 4×4 pixels when i=-1..4, j=-1..4, abs(x) is a function that returns the absolute value of x, and Sign(x) is a function that returns 1 when the value of x is greater than 0, returns 0 when the value of x is 0, and returns -1 when the value of x is less than 0.
[0221] Next, the motion compensation correction value derivation unit 309544 uses the derived gradient products and sGx2, sGy2, sGxGy, sGxdI and sGydI to derive a correction weight vector (u, v) in units of 4×4 pixels. u=sGx2>0? Clip3(-mvRefineThres+1, mvRefineThres-1, (sGxdI<<2)>>Floor(Log2(sGx2))): 0 v=sGy2>0? Clip3(-mvRefineThres+1, mvRefineThres-1, ((sGydI<<2)-((vx*sGxGy)>>1))>>Floor(Log2(sGy2))):0
[0222] The motion compensation correction value derivation unit 309544 derives bdofOffset using the correction weight vector (u, v) in units of 4×4 pixels and the gradient images gradientHL0, gradientVL0, gradientHL1, gradientVL1. bdofOffset=u*(gradientHL0[x+1][y+1]-gradientHL1[x+1][y+1]) +v*(gradientVL0[x+1][y+1]-gradientVL1[x+1][y+1])
[0223] The bidirectional prediction image generation unit 309545 uses the above parameters to derive the pixel value Pred of the 4×4 pixel prediction image by the following formula. pbSamples[x][y]=Clip3(0,(1< <BitDepth)-1, (predSamplesL0[x+1][y+1]+offset4+predSamplesL1[x+1][y+1]+bdof Offset)>>shift4) (Weighted Forecast)
[0224] The weighted prediction unit 3094 generates a prediction image pbSamples of the block from the interpolation image PredLX. First, the syntax associated with the weighted prediction unit 3094 will be described.
[0225] Fig. 27 (a) shows an example of the syntax of sps_weighted_pred_flag and sps_weighted_bipred_flag in a sequence parameter set (Sequence Paramenter Set: SPS).
[0226] sps_weighted_pred_flag equal to 1 indicates that weighted prediction may be applied to P slices referring to SPS. sps_weighted_pred_flag equal to 0 indicates that weighted prediction is not applied to P slices referring to SPS. sps_weighted_bipred_flag equal to 1 indicates that weighted prediction may be applied to B slices referring to SPS. sps_weighted_bipred_flag equal to 0 indicates that weighted prediction is not applied to B slices referring to SPS.
[0227] Fig. 27 (b) represents an example of the syntax of pps_weighted_pred_flag and pps_weighted_bipred_flag in a picture parameter set (PPS).
[0228] pps_weighted_pred_flag equal to 0 indicates that weighted prediction is not applied to the P slice that refers to the PPS. pps_weighted_pred_flag equal to 1 indicates that weighted prediction is applied to the P slice that refers to the PPS. When sps_weighted_pred_flag is equal to 0, the weighted prediction unit 3094 sets the value of pps_weighted_pred_flag to 0. If pps_weighted_pred_flag does not exist, the value is set to 0.
[0229] pps_weighted_bipred_flag equal to 0 indicates that weighted prediction is not applied to B slices referring to PPS. pps_weighted_bipred_flag equal to 1 indicates that weighted prediction is applied to B slices referring to PPS. When sps_weighted_bipred_flag is equal to 0, the weighted prediction unit 3094 sets the value of pps_weighted_bipred_flag to 0. If pps_weighted_bipred_flag does not exist, the value is set to 0.
[0230] Fig.28 Represents the weighted prediction parameters syntax () of non-patent document 1 called through the slice header.
[0231] Here, luma_log2_weight_denom is the base 2 logarithm of the denominators of all luma weighting coefficients. The value of luma_log2_weight_denom must be in the range of 0 to 7. delta_chroma_log2_weight_denom is the difference of the base 2 logarithms of the denominators of all chroma weighting coefficients. In the absence of delta_chroma_log2_weight_denom, it is presumed to be equal to 0. The inference is that the variable ChromaLog2WeightDenom is equal to luma_log2_weight_denom + delta_chroma_log2_weight_denom, and the value must be in the range of 0 to 7.
[0232] When luma_weight_l0_flag[i] is 1, it indicates that there is a weighting coefficient of the luminance component of the L0 prediction. When luma_weight_l0_flag[i] is 0, it indicates that there is no weighting coefficient of the luminance component of the L0 prediction. In the absence of luma_weight_l0_flag[i], the weighted prediction unit 3094 infers that it is equal to 0. When chroma_weight_l0_flag[i] is 1, it indicates that there is a weighting coefficient of the chrominance prediction value of the L0 prediction. When chroma_weight_l0_flag[i] is 0, it indicates that there is no weighting coefficient of the chrominance prediction value of the L0 prediction. In the absence of chroma_weight_l0_flag[i], the weighted prediction unit 3094 infers that it is equal to 0.
[0233] delta_luma_weight_l0[i] is the difference of the weighted coefficient of the luma prediction value for L0 prediction using RefPicList[0][i]. It is derived that the variable LumaWeightL0[i] is equal to (1<<luma_log2_weight_denom)+delta_luma_weight_l0[i]. When luma_weight_l0_flag[i] is equal to 1, the value of delta_luma_weight_l0[i] must be in the range of -128 to 127. When luma_weight_l0_flag[i] is equal to 0, the weighted prediction unit 3094 assumes that LumaWeightL0[i] is equal to the power value of 2 to the luma_log2_weight_denom (2^luma_log2_weight_denom).
[0234] luma_offset_l0[i] is the additive offset of the luma prediction value for L0 prediction using RefPicList[0][i]. The value of luma_offset_l0[i] must be in the range of -128 to 127. When luma_weight_l0_flag[i] is equal to 0, the weighted prediction unit 3094 assumes that luma_offset_l0[i] is equal to 0.
[0235] delta_chroma_weight_l0[i][j] is the difference of the weighted coefficient of the chroma prediction value for the L0 prediction using RefPicList0[i] where j is 0 in Cb and j is 1 in Cr. It is derived that the variable ChromaWeightL0[i][j] is equal to (1<<ChromaLog2WeightDenom)+delta_chroma_weight_l0[i][j]. When chroma_weight_l0_flag[i] is equal to 1, the value of delta_chroma_weight_l0[i][j] must be in the range of -128 to 127. In the case where chroma_weight_l0_flag[i] is 0, the weighted prediction unit 3094 assumes that ChromaWeightL0[i][j] is a power of 2 equal to ChromaLog2WeightDenom (2^ChromaLog2WeightDenom). delta_chroma_offset_l0[i][j] is the difference of the addition offset of the chroma prediction value for the L0 prediction using RefPicList0[i] where j is 0 in Cb and j is 1 in Cr. The variable ChromaOffsetL0[i][j] is derived as follows. ChromaOffsetL0[i][j] = Clip3(-128, 127, (128 + delta_chroma_offset_l0[i][j] - ((128 * ChromaWeightL0[i][j]) >> ChromaLog2WeightDenom)))
[0236] The value of delta_chroma_offset_l0[i][j] must be in the range of -4 * 128 to 4 * 127. In the case where chroma_weight_l0_flag[i] is 0, the weighted prediction unit 3094 assumes that ChromaOffsetL0[i][j] is equal to 0.
[0237] It should be noted that luma_weight_l1_flag[i], chroma_weight_l1_flag[i], delta_luma_weight_l1[i], luma_offset_l1[i], delta_chroma_weight_l1[i][j] and delta_chroma_offset_l1[i][j] are replaced by luma_weight_l0_flag[i], chroma_weight_l0_flag[i], delta_luma_weight_l0[i], luma_offset_l0[i], delta_chroma_weight_l0[i][j] and delta_chroma_offset_l0[i][j] respectively for interpretation, and l0, L0, list0 and List0 are replaced by l1, l1, list1 and List1 respectively for interpretation.
[0238] The following describes the processing of the inter-frame prediction parameter derivation unit 303.
[0239] In Non-Patent Document 1, a variable weightedPredFlag indicating whether or not to perform weighted prediction processing is derived as follows.
[0240] When slice_type is equal to P, weightedPredFlag is set equal to pps_weighted_pred_flag defined by PPS.
[0241] In addition, when slice_type is equal to B, weightedPredFlag is set to be equal to pps_weighted_bipred_flag&&(!dmvrFlag) defined by PPS.
[0242] Hereinafter, bcw_idx is a weight index of bidirectional prediction with weights in CU units. When bcw_idx is not notified, bcw_idx=0 is set. bcwIdx sets bcwIdxN of a neighboring block in merge prediction mode and sets bcw_idx of a target block in AMVP prediction mode.
[0243] If the value of the variable weightedPredFlag is equal to 0 or the value of the variable bcwIdx is 0, the predicted image pbSamples is derived as shown below and processed as a normal predicted image.
[0244] When one of the prediction list utilization flags (predFlagL0 or predFlagL1) is 1 (unidirectional prediction) (weighted prediction is not used), the following calculation is performed to match PredLX (LX is L0 or L1) with the pixel bit number bitDepth. pbSamples[x][y]=Clip3(0,(1<<bitDepth)-1,(PredLX[x][y]+offset1)> >shift1)
[0245] Here, shift1=14-bitDepth, offset1=1<<(shift1-1). PredLX is the interpolation image of L0 or L1 prediction.
[0246] When both prediction list utilization flags (predFlagL0 and predFlagL1) are 1 (bidirectional prediction PRED_BI) and weighted prediction is not used, the following calculation is performed to average PredL0 and PredL1 and match the average with the number of pixel bits. pbSamples[x][y]=Clip3(0,(1<<bitDepth)-1,(PredL0[x][y]+PredL1[x][y]+offset2)> >shift2)
[0247] Here, shift2=15-bitDepth, offset2=1<<(shift2-1).
[0248] If the value of the variable weightedPredFlag is equal to 1 and the value of the variable bcwIdx is equal to 0, the prediction image pbSamples is derived as shown below as a weighted prediction process.
[0249] The variable shift1 is set equal to Max(2, 14-bitDepth). The variables log2Wd, o0, o1, w0, and w1 are derived as follows.
[0250] If cIdx is 0 and is brightness, the following formula is applied. log2Wd=luma_log2_weight_denom+shift1 w0=LumaWeightL0[refIdxL0] w1=LumaWeightL1[refIdxL1] o0=luma_offset_l0[refIdxL0]<<(bitDepth-8) o1=luma_offset_l1[refIdxL1]<<(bitDepth-8)
[0251] In other cases (color difference where cIdx is not equal to 0), the following formula is applied. log2Wd=ChromaLog2WeightDenom+shift1 w0=ChromaWeightL0[refIdxL0][cIdx-1] w1=ChromaWeightL1[refIdxL1][cIdx-1] o0=ChromaOffsetL0[refIdxL0][cIdx-1]<<(bitDepth-8) o1=ChromaOffsetL1[refIdxL1][cIdx-1]<<(bitDepth-8)
[0252] The pixel values pbSamples[x][y] of the predicted image with x=0..nCbW-1 and y=0..nCbH-1 are derived as shown below.
[0253] Next, if predFlagL0 is equal to 1 and predFlagL1 is equal to 0, the pixel value pbSamples[x][y] of the predicted image is derived as shown below.
[0254] In addition, when predFlagL0 is 0 and predFlagL1 is 1, the pixel value pbSamples[x][y] of the predicted image is derived as shown below.
[0255] In addition, when predFlagL0 is equal to 1 and predFlagL1 is equal to 1, the pixel value pbSamples[x][y] of the predicted image is derived as shown below. pbSamples[x][y]=Clip3(0,(1< <bitDepth)-1, (predSamplesL0[x][y]*w0+predSamplesL1[x][y]*w1+ ((o0+o1+1)<<log2Wd))> >(log2Wd+1))
[0256] However, in the method described in non-patent document 1, for example, for each reference picture in the L0 list, there is a flag luma_weight_l0_flag[i] indicating whether there is a luminance weighting coefficient LumaWeightL0[i] and an offset value luma_offset_l0[refIdxL0], and as for the right shift value equivalent to the denominator of the weighting coefficient, there is only one luma_log2_weight_denom of luminance. Therefore, in the absence of a weighting coefficient and an offset value for the reference list, the value of the weighting coefficient is set to 2 to the power of luma_log2_weight_denom, and the offset value is set to 0 to perform weighted prediction processing. Therefore, in the case of L0 prediction or L1 prediction, in the case of the absence of a weighting coefficient and an offset value for the reference list, weighted prediction processing is performed at a position where normal prediction processing should be performed. In addition, in the case of bidirectional prediction, in the case of the absence of weighting coefficients and offset values for both the L0 list and the L1 list, weighted prediction processing is performed at a position where normal bidirectional prediction processing should be performed. Thus, there is a problem that weighted prediction processing may be performed even when normal prediction processing should be performed.
[0257] Therefore, as this embodiment, a function WeightedPredFlag(slice_type, cIdx, refIdxL0, refIdxL1) is defined as follows. WeightedPredFlag(slice_type, cIdx, refIdxL0, refIdxL1)= (slice_type==P&&pps_weighted_pred_flag)? ((cIdx==0)?luma_weight_l0_flag[refIdxL0]: chroma_weight_l0_flag[refIdxL0]): ((slice_type==B&&pps_weighted_bipred_flag)? ((refIdxL0>=0&&refIdxL1>=0)? ((cIdx==0)?(luma_weight_l0_flag[refIdxL0]|| luma_weight_l1_flag[refIdxL1]): (chroma_weight_l0_flag[refIdxL0]|| chroma_weight_l1_flag[refIdxL1])): ((refIdxL0>=0&&refIdxL1<0)? ((cIdx==0)?luma_weight_l0_flag[refIdxL0]: chroma_weight_l0_flag[refIdxL0]): ((refIdxL0<0&&refIdxL1>=0)? ((cIdx==0)?luma_weight_l1_flag[refIdxL1]: chroma_weight_l1_flag[refIdxL1]): 0)))
[0258] This function WeightedPredFlag(slice_type, cIdx, refIdxL0, refIdxL1) can be used when performing exclusive processing with weighted prediction. Fig.29 The syntax of is an example for notifying bcw_idx[x0][y0] only when there is no coefficient of weighted prediction in order to make BCW prediction and weighted prediction operate exclusively.
[0259] In addition, when the DMVR process and the BDOF process are applied without a luminance weighting coefficient in the weighted prediction process, the condition that the value of the function WeightedPredFlag (B, 0, refIdxL0, refIdxL1) is equal to 0 can be set.
[0260] When DMVR processing and BDOF processing can be applied when there are no weighting coefficients for brightness and color difference in weighted prediction processing, the condition that the values of both functions WeightedPredFlag(B, 0, refIdxL0, refIdxL1) and WeightedPredFlag(B, 1, refIdxL0, refIdxL1) are set to 0.
[0261] The variable weightedPredFlag indicating whether to perform weighted prediction processing only needs to set the value of the function WeightedPredFlag (slice_type, cIdx, refIdxL0.refIdxL1).
[0262] Furthermore, the method described in Non-Patent Document 1 has a problem in that even when the value of the variable weightedPredFlag is 1 and the weighted prediction process is called, the weighted prediction process is performed when the normal prediction process should be performed as a result.
[0263] Therefore, in another implementation of this embodiment, the weighted prediction unit 3094 derives the variable weightedPredFlag as shown below.
[0264] When slice_type is equal to P, the variable weightedPredFlag is set to weightedPredFlag=(cIdx==0)? luma_weight_l0_flag[refIdxL0]: chroma_weight_l0_flag[refIdxL0].
[0265] In other cases (slice_type is equal to B), setting is performed as follows.
[0266] If refIdxL0 is greater than 0 and refIdxL1 is greater than 0, the variable weightedPredFlag is set to weightedPredFlag=(cIdx==0)? (luma_weight_l0_flag[refIdxL0]||luma_weight_l1_flag[refIdxL1]): (chroma_weight_l0_flag[refIdxL0]||chroma_weight_l1_flag[refIdxL1]).
[0267] If refIdxL0 is greater than 0 and refIdxL1 is less than 0, the variable weightedPredFlag is set to weightedPredFlag=(cIdx==0)? luma_weight_l0_flag[refIdxL0]: chroma_weight_l0_flag[refIdxL0].
[0268] If refIdxL0 is less than 0 and refIdxL1 is greater than 0, the variable weightedPredFlag is set to weightedPredFlag=(cIdx==0)? luma_weight_l1_flag[refIdxL1]: chroma_weight_l0_flag[refIdxL1].
[0269] In all other cases, the variable weightedPredFlag is set to weightedPredFlag=0.
[0270] By performing derivation as described above, it is possible to specify for each luminance signal and color difference signal that when there is no weighting coefficient in the moving picture encoding / decoding process, weighted prediction is not performed, thereby solving the above-mentioned problem.
[0271] In addition, as another embodiment of the present embodiment, a method of solving the above-mentioned problem within the weighted prediction process is shown.
[0272] If the value of the variable weightedPredFlag is equal to 1 and the value of the variable bcwIdx is equal to 0, the prediction image pbSamples is derived as shown below as a weighted prediction process.
[0273] The variable shift1 is set equal to Max(2, 14-bitDepth). The variables log2Wd, o0, o1, w0, and w1 are derived as follows.
[0274] If cIdx is 0 and is brightness, the following formula is applied. log2Wd=luma_log2_weight_denom+shift1 WeightL0Flag=luma_weight_l0_flag[refIdxL0] WeightL1Flag=luma_weight_l1_flag[refIdxL1] w0=LumaWeightL0[refIdxL0] w1=LumaWeightL1[refIdxL1] o0=luma_offset_l0[refIdxL0]<<(bitDepth-8) o1=luma_offset_l1[refIdxL1]<<(bitDepth-8)
[0275] In other cases (color difference where cIdx is not equal to 0), the following formula is applied. log2Wd=ChromaLog2WeightDenom+shift1 WeightL0Flag=chroma_weight_l0_flag[refIdxL0] WeightL1Flag=chroma_weight_l1_flag[refIdxL1] w0=ChromaWeightL0[refIdxL0][cIdx-1] w1=ChromaWeightL1[refIdxL1][cIdx-1] o0=ChromaOffsetL0[refIdxL0][cIdx-1]<<(bitDepth-8) o1=ChromaOffsetL1[refIdxL1][cIdx-1]<<(bitDepth-8)
[0276] The pixel values pbSamples[x][y] of the predicted image with x=0..nCbW-1 and y=0..nCbH-1 are derived as shown below.
[0277] If predFlagL0 is equal to 1 and predFlagL1 is equal to 0, the pixel value pbSamples[x][y] of the predicted image is derived as shown below. if(WeightL0Flag==0)
[0278] In addition, when predFlagL0 is 0 and predFlagL1 is 1, the pixel value pbSamples[x][y] of the predicted image is derived as shown below.
[0279] In addition, when predFlagL0 is equal to 1 and predFlagL1 is equal to 1, the pixel value pbSamples[x][y] of the predicted image is derived as shown below.
[0280] By performing derivation as described above, it is possible to specify for each luminance signal and color difference signal that when a weighting coefficient does not exist in the moving picture encoding / decoding process, weighted prediction is not performed, thereby solving the above-mentioned problem.
[0281] As another problem, in Non-Patent Document 1, a concept of a picture header is newly introduced, but there is a problem that it does not correspond to the concept.
[0282] Therefore, as another implementation of this embodiment, flag information for weighted prediction is newly introduced into the picture header. Fig.30 As shown in (b), pic_weighted_pred_flag and pic_weighted_bipred_flag are imported in picture_header_rbsp(), and in the slice header (Slice header), as shown in Fig.30 As shown in (c), when pic_weighted_pred_flag or pic_weighted_bipred_flag is 1, pred_weight_table() is called.
[0283] By setting the syntax as described above, the weighted prediction process is switched for each picture.
[0284] As another problem, in non-patent document 1, for the variable weightedPredFlag indicating whether weighted prediction processing is performed, when slice_type is equal to B, weightedPredFlag is set to be equal to pps_weighted_bipred_flag&&(!dmvrFlag) defined by PPS. However, one of the conditions for dmvrFlag, which is a variable indicating whether DMVR processing is performed, to be true is defined only for the weighted prediction processing of the luminance signal when both luma_weight_l0_flag[refIdxL0] and luma_weight_l1_flag[refIdxL1] are 0. Therefore, there is a problem that the application conditions of the weighted prediction processing for the color difference signal are not clarified.
[0285] Therefore, as another implementation of this embodiment, when slice_type is equal to B, the variable weightedPredFlag is set to be equal to pps_weighted_bipred_flag &&!(dmvrFlag &&cIdx==0) defined by PPS.
[0286] By setting the condition as described above, even when dmvrFlag is 1, the weighted prediction process for the color difference signal is appropriately performed, and the application condition of the weighted prediction process is clarified.
[0287] Alternatively, it is also possible to set the weighted prediction process of brightness or color difference not to be performed in either the DMVR process or the BDOF process. Furthermore, when slice_type is equal to B, weightedPredFlag is set to be equal to pps_weighted_bipred_flag && !dmvrFlag && !bdofAvailableFlag defined in PPS.
[0288] By setting the conditions as described above, the application conditions of the weighted prediction process are clarified. (BCW forecast)
[0289] BCW (Bi-prediction with CU-level Weights) prediction is a prediction method that can switch weighting coefficients predetermined at the CU level.
[0290] Input two variables nCbW and nCbH that specify the width and height of the current coding block, two arrangements predSamplesL0 and predSamplesL1 of (nCbW)x(nCbH), flags predFlagL0 and predFlagL1 indicating whether to use the prediction list, reference indexes refIdxL0 and refIdxL1, BCW prediction index bcw_idx, and variable cIdx that specifies the index of the luminance and chrominance components, perform BCW prediction processing, and output the pixel values of the predicted image of the arrangement pbSamples of (nCbW)x(nCbH).
[0291] When sps_bcw_enabled_flag indicating whether to use the prediction at the SPS level is true (TURE), the variable weightedPredFlag is 0, and there is no weighted prediction coefficient in the reference picture indicated by the two reference indexes refIdxL0 and refIdxL1, and the coding block size is less than a certain value, the bcw_idx of the CU-level syntax is explicitly notified and the value is substituted into the variable bcwIdx. If bcw_idx does not exist, 0 is substituted into the variable bcwIdx.
[0292] When the variable bcwIdx is 0, the pixel value of the predicted image is derived as shown below. pbSamples[x][y]=Clip3(0,(1< <bitDepth)-1, (predSamplesL0[x][y]+predSamplesL1[x][y]+offset2)>>shift2)
[0293] In other cases (when bcwIdx is not equal to 0), the following formula is applied.
[0294] Set the variable w1 to be equal to bcwWLut[bcwIdx]. bcwWLut[k]={4, 5, 3, 10, -2}.
[0295] The variable w0 is set to (8-w1). In addition, the pixel value of the predicted image is derived as shown below. pbSamples[x][y]=Clip3(0,(1< <bitDepth)-1, (w0*predSamplesL0[x][y]+ w1*predSamplesL1[x][y]+offset3)>>(shift2+3))
[0296] When BCW prediction is used in the AMVP prediction mode, the inter-frame prediction parameter decoding unit 303 decodes bcw_idx and sends it to the BCW unit 30955. In addition, when BCW prediction is used in the merge prediction mode, the inter-frame prediction parameter decoding unit 303 decodes the merge index merge_idx, and the merge candidate derivation unit 30361 derives bcwIdx for each merge candidate. Specifically, the merge candidate derivation unit 30361 uses the weighting coefficient of the adjacent block used for the derivation of the merge candidate as the weighting coefficient of the merge candidate for the object block. That is, in the merge mode, the weighting coefficient used in the past is inherited as the weighting coefficient of the object block. (Intra-frame prediction image generation unit 310)
[0297] When predMode indicates an intra prediction mode, the intra prediction image generation unit 310 performs intra prediction using the intra prediction parameters input from the intra prediction parameter derivation unit 304 and the reference pixels read from the reference picture memory 306 .
[0298] The inverse quantization / inverse transformation unit 311 inversely quantizes the quantized transformation coefficient input from the parameter decoding unit 302 to obtain a transformation coefficient.
[0299] The adder 312 adds the predicted image of the block input from the predicted image generator 308 and the prediction error input from the inverse quantization / inverse transform unit 311 for each pixel to generate a decoded image of the block. The adder 312 stores the decoded image of the block in the reference picture memory 306 and outputs it to the loop filter 305.
[0300] The inverse quantization / inverse transformation unit 311 inversely quantizes the quantized transformation coefficient input from the parameter decoding unit 302 to obtain a transformation coefficient.
[0301] The adder 312 adds the predicted image of the block input from the predicted image generator 308 and the prediction error input from the inverse quantization / inverse transform unit 311 for each pixel to generate a decoded image of the block. The adder 312 stores the decoded image of the block in the reference picture memory 306 and outputs it to the loop filter 305. (Configuration of Moving Image Coding Device)
[0302] Next, the configuration of the moving picture encoding device 11 according to this embodiment will be described. Fig. 22 1 is a block diagram showing the structure of the video coding device 11 of the present embodiment. The video coding device 11 includes a prediction image generation unit 101, a subtraction unit 102, a transformation / quantization unit 103, an inverse quantization / inverse transformation unit 105, an addition unit 106, a loop filter 107, a prediction parameter memory (prediction parameter storage unit, frame memory) 108, a reference picture memory (reference picture storage unit, frame memory) 109, a coding parameter determination unit 110, a parameter coding unit 111, a prediction parameter derivation unit 120, and an entropy coding unit 104.
[0303] The predicted image generation unit 101 generates a predicted image for each CU. The predicted image generation unit 101 includes the inter-prediction image generation unit 309 and the intra-prediction image generation unit 310 which have been described above, and the description thereof will be omitted.
[0304] The subtraction unit 102 generates a prediction error by subtracting the pixel value of the predicted image of the block input from the predicted image generation unit 101 from the pixel value of the image T. The subtraction unit 102 outputs the prediction error to the transformation / quantization unit 103 .
[0305] The transform / quantization unit 103 calculates transform coefficients by frequency transforming the prediction error input from the subtraction unit 102 , and derives quantized transform coefficients by quantization. The transform / quantization unit 103 outputs the quantized transform coefficients to the parameter encoding unit 111 and the inverse quantization / inverse transform unit 105 .
[0306] The inverse quantization / inverse transformation unit 105 and the inverse quantization / inverse transformation unit 311 ( Figure 7 The calculated prediction error is input to the adding unit 106.
[0307] The parameter coding unit 111 includes a header coding unit 1110, a CT information coding unit 1111, and a CU coding unit 1112 (prediction mode coding unit). The CU coding unit 1112 further includes a TU coding unit 1114. The following describes the schematic operation of each module.
[0308] The header coding unit 1110 performs coding processing on parameters such as header information, partition information, prediction information, and quantized transform coefficients.
[0309] The CT information encoding unit 1111 encodes QT, MT (BT, TT) division information and the like.
[0310] The CU encoding unit 1112 encodes CU information, prediction information, partition information, and the like.
[0311] When a prediction error is included in a TU, the TU encoding unit 1114 encodes the QP update information and the quantized prediction error.
[0312] The CT information encoding unit 1111 and the CU encoding unit 1112 provide syntax elements such as inter-frame prediction parameters (predMode, merge_flag, merge_idx, inter_pred_idc, refIdxLX, mvp_LX_idx, mvdLX), intra-frame prediction parameters (intra_luma_mpm_flag, intra_luma_mpm_idx, intra_luma_mpm_reminder, intra_chroma_pred_mode), and quantized transform coefficients to the parameter encoding unit 111.
[0313] The entropy coding unit 104 receives the quantized transform coefficients and coding parameters (division information, prediction parameters) from the parameter coding unit 111. The entropy coding unit 104 performs entropy coding on these, and generates and outputs a coded stream Te.
[0314] The prediction parameter derivation unit 120 includes an inter prediction parameter encoding unit 112 and an intra prediction parameter encoding unit 113, and derives intra prediction parameters and intra prediction parameters from the parameters input from the encoding parameter determination unit 110. The derived intra prediction parameters and intra prediction parameters are input to the parameter encoding unit 111. (Configuration of Inter-frame Prediction Parameter Encoding Unit)
[0315] like Fig.23 As shown, the inter-frame prediction parameter coding unit 112 includes a parameter coding control unit 1121 and an inter-frame prediction parameter derivation unit 303. The inter-frame prediction parameter derivation unit 303 is a common structure with the moving picture decoding device. The parameter coding control unit 1121 includes a merge index derivation unit 11211 and a vector candidate index derivation unit 11212.
[0316] The merge index derivation unit 11211 derives merge candidates and the like, and outputs them to the inter prediction parameter derivation unit 303. The vector candidate index derivation unit 11212 derives prediction vector candidates and the like, and outputs them to the inter prediction parameter derivation unit 303 and the parameter encoding unit 111. (Configuration of Intra-frame Prediction Parameter Coding Unit 113)
[0317] like Fig.24 As shown, the intra prediction parameter coding unit 113 includes a parameter coding control unit 1131 and an intra prediction parameter derivation unit 304. The intra prediction parameter derivation unit 304 is a common configuration of the moving picture decoding apparatus.
[0318] The parameter encoding control unit 1131 derives IntraPredModeY and IntraPredModeC, and further refers to mpmCandList[] to determine intra_luma_mpm_flag, and outputs these prediction parameters to the intra prediction parameter derivation unit 304 and the parameter encoding unit 111 .
[0319] However, unlike the moving picture decoding device, the inputs to the inter-frame prediction parameter derivation unit 303 and the intra-frame prediction parameter derivation unit 304 are the encoding parameter determination unit 110 and the prediction parameter memory 108 , and the outputs are to the parameter encoding unit 111 .
[0320] The adder 106 generates a decoded image by adding the pixel value of the prediction block input from the prediction image generator 101 and the prediction error input from the inverse quantization / inverse transform unit 105 for each pixel. The adder 106 stores the generated decoded image in the reference picture memory 109.
[0321] The loop filter 107 performs deblocking filtering, SAO, and ALF on the decoded image generated by the adding unit 106. It should be noted that the loop filter 107 does not necessarily include the above three filters, and may include only a deblocking filter, for example.
[0322] The prediction parameter memory 108 stores the prediction parameter generated by the encoding parameter determination unit 110 in a predetermined position for each target picture and CU.
[0323] The reference picture memory 109 stores the decoded image generated by the loop filter 107 at a predetermined position for each target picture and CU.
[0324] The coding parameter determination unit 110 selects one of a plurality of sets of coding parameters. The coding parameters refer to the QT, BT or TT partition information, prediction parameters or parameters generated in association with them as the coding target. The predicted image generation unit 101 generates a predicted image using these coding parameters.
[0325] The coding parameter determination unit 110 calculates the RD cost value representing the size of the information amount and the coding error for each of the multiple sets. The RD cost value is, for example, the sum of the code amount and the value obtained by multiplying the square error by the coefficient λ. The code amount is the amount of information of the coded stream Te obtained by entropy encoding the quantization error and the coding parameter. The square error is the sum of the squares of the prediction error calculated in the subtraction unit 102. The coefficient λ is a real number greater than a preset zero. The coding parameter determination unit 110 selects a set of coding parameters with the smallest calculated cost value. The coding parameter determination unit 110 outputs the determined coding parameters to the parameter encoding unit 111 and the prediction parameter derivation unit 120.
[0326] It should be noted that a part of the moving image encoding device 11 and the moving image decoding device 31 in the above-mentioned embodiment can be implemented by a computer, for example, the entropy decoding unit 301, the parameter decoding unit 302, the loop filter 305, the prediction image generation unit 308, the inverse quantization / inverse transformation unit 311, the addition unit 312, the prediction parameter derivation unit 320, the prediction image generation unit 101, the subtraction unit 102, the transformation / quantization unit 103, the entropy encoding unit 104, the inverse quantization / inverse transformation unit 105, the loop filter 107, the coding parameter determination unit 110 and the prediction parameter encoding unit 111. In this case, the program for realizing the control function can be recorded in a computer-readable recording medium, and the computer system can read the program recorded in the recording medium and execute it. It should be noted that the "computer system" mentioned here refers to a computer system built into any one of the moving image encoding device 11 and the moving image decoding device 31, and a computer system including hardware such as an OS and peripheral devices is used. In addition, "computer-readable recording medium" refers to removable media such as floppy disks, magneto-optical disks, ROMs, CD-ROMs, and storage devices such as hard disks built into computer systems. Moreover, "computer-readable recording medium" may also include: recording media that dynamically store programs for a short period of time, such as communication lines when sending programs via networks such as the Internet or communication lines such as telephone lines; and recording media that store programs for a fixed period of time, such as volatile memories inside computer systems that serve as servers or clients in this case. In addition, the above-mentioned program may be a program for realizing a part of the above-mentioned functions, or a program that can realize the above-mentioned functions by combining with programs already recorded in the computer system.
[0327] In addition, part or all of the moving picture encoding device 11 and the moving picture decoding device 31 in the above-mentioned embodiment can also be implemented as an integrated circuit such as LSI (Large Scale Integration). Each functional block of the moving picture encoding device 11 and the moving picture decoding device 31 can be individually processor-based, or part or all of them can be integrated to be processor-based. In addition, the method of integrated circuitization is not limited to LSI, and it can also be implemented by a dedicated circuit or a general-purpose processor. In addition, when a technology for integrated circuitization that replaces LSI appears with the advancement of semiconductor technology, an integrated circuit based on this technology can also be used.
[0328] As mentioned above, one embodiment of the present invention has been described in detail with reference to the drawings, but the specific configuration is not limited to the above embodiment, and various design changes and the like can be made within the scope not departing from the gist of the present invention. [Application Examples]
[0329] The above-mentioned moving picture encoding device 11 and moving picture decoding device 31 can be mounted on various devices for sending, receiving, recording, and reproducing moving pictures. It should be noted that the moving pictures can be natural moving pictures taken by a camera or the like, or artificial moving pictures (including CG and GUI) generated by a computer or the like.
[0330] First, refer to Figure 2 A case where the above-mentioned moving picture encoding device 11 and moving picture decoding device 31 can be used for transmission and reception of moving pictures will be described.
[0331] Figure 2 PROD_A is a block diagram showing the structure of a transmitting device PROD_A equipped with a motion picture encoding device 11. Figure 2 As shown, the transmitting device PROD_A includes: an encoding unit PROD_A1 for encoding a moving image to obtain encoded data, a modulation unit PROD_A2 for modulating a carrier wave using the encoded data obtained by the encoding unit PROD_A1 to obtain a modulated signal, and a transmitting unit PROD_A3 for transmitting the modulated signal obtained by the modulation unit PROD_A2. The moving image encoding device 11 described above is used as the encoding unit PROD_A1.
[0332] As a source of motion images input to the encoding unit PROD_A1, the transmitting device PROD_A may further include: a camera PROD_A4 for shooting motion images, a recording medium PROD_A5 for recording motion images, an input terminal PROD_A6 for inputting motion images from the outside, and an image processing unit A7 for generating or processing images. Figure 2The example in which the transmitting device PROD_A has all of these structures is shown, but some of them may be omitted.
[0333] It should be noted that the recording medium PROD_A5 may be a medium on which an uncoded moving image is recorded, or may be a medium on which a moving image coded in a recording coding method different from a transmission coding method is recorded. In the latter case, it is preferred that a decoding unit (not shown) that decodes coded data read from the recording medium PROD_A5 in accordance with the recording coding method is interposed between the recording medium PROD_A5 and the coding unit PROD_A1.
[0334] Figure 2 PROD_B is a block diagram showing the structure of a receiving device PROD_B equipped with a motion picture decoding device 31. Figure 2 As shown, the receiving device PROD_B includes a receiving unit PROD_B1 that receives a modulated signal, a demodulating unit PROD_B2 that demodulates the modulated signal received by the receiving unit PROD_B1 to obtain coded data, and a decoding unit PROD_B3 that decodes the coded data obtained by the demodulating unit PROD_B2 to obtain a moving image. The moving image decoding device 31 described above is used as the decoding unit PROD_B3.
[0335] The receiving device PROD_B, as a supply destination of the moving image output by the decoding unit PROD_B3, may further include a display PROD_B4 for displaying the moving image, a recording medium PROD_B5 for recording the moving image, and an output terminal PROD_B6 for outputting the moving image to the outside. Figure 2 The example in which the receiving device PROD_B has all of these structures is shown, but some of them may be omitted.
[0336] It should be noted that the recording medium PROD_B5 may be a medium for recording uncoded moving images, or may be a medium coded in a recording coding method different from a transmission coding method. In the latter case, it is preferred that a coding unit (not shown) that codes the moving images obtained from the decoding unit PROD_B3 in a recording coding method is interposed between the decoding unit PROD_B3 and the recording medium PROD_B5.
[0337] It should be noted that the transmission medium for transmitting the modulated signal can be wireless or wired. In addition, the transmission scheme for transmitting the modulated signal can be broadcast (here, it refers to a transmission scheme in which the transmission destination is not predetermined) or communication (here, it refers to a transmission scheme in which the transmission destination is predetermined). That is, the transmission of the modulated signal can be achieved through any one of wireless broadcasting, wired broadcasting, wireless communication and wired communication.
[0338] For example, a broadcasting station (broadcasting equipment, etc.) / receiving station (television receiver, etc.) of terrestrial digital broadcasting is an example of a transmitting device PROD_A / receiving device PROD_B that transmits and receives modulated signals via wireless broadcasting. In addition, a broadcasting station (broadcasting equipment, etc.) / receiving station (television receiver, etc.) of cable television broadcasting is an example of a transmitting device PROD_A / receiving device PROD_B that transmits and receives modulated signals via wired broadcasting.
[0339] In addition, a server (workstation, etc.) / client (television receiver, personal computer, smart phone, etc.) of a VOD (Video On Demand) service or a motion picture sharing service using the Internet is an example of a transmitter PROD_A / receiver PROD_B that transmits and receives a modulated signal through communication (usually, a wireless or wired transmission medium is used as a transmission medium in a LAN, and a wired transmission medium is used as a transmission medium in a WAN). Here, a personal computer includes a desktop PC, a laptop PC, and a tablet PC. In addition, a smart phone also includes a multifunctional portable phone terminal.
[0340] It should be noted that the client of the moving image sharing service has the function of decoding the encoded data downloaded from the server and displaying it on the display, and also has the function of encoding the moving images captured by the camera and uploading them to the server. That is, the client of the moving image sharing service plays the role of both the sending device PROD_A and the receiving device PROD_B.
[0341] Next, refer to Figure 3 , a description is given of a case where the above-mentioned motion image encoding device 11 and motion image decoding device 31 can be used for recording and reproducing motion images.
[0342] Figure 3 PROD_C is a block diagram showing the structure of a recording device PROD_C equipped with the above-mentioned motion picture encoding device 11. Figure 3 As shown, the recording device PROD_C includes a coding unit PROD_C1 for obtaining coded data by coding a moving picture, and a writing unit PROD_C2 for writing the coded data obtained by the coding unit PROD_C1 into the recording medium PROD_M. The moving picture coding device 11 described above is used as the coding unit PROD_C1.
[0343] It should be noted that the recording medium PROD_M can be (1) a type of recording medium built into the recording device PROD_C such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive), or (2) a type of recording medium connected to the recording device PROD_C such as an SD memory card or a USB (Universal Serial Bus) flash memory, or (3) a recording medium loaded into a drive device (not shown) built into the recording device PROD_C such as a DVD (Digital Versatile Disc, registered trademark) or a BD (Blu-ray Disc, registered trademark).
[0344] In addition, as a supply source of moving images input to the encoding unit PROD_C1, the recording device PROD_C may further include: a camera PROD_C3 for shooting moving images, an input terminal PROD_C4 for inputting moving images from the outside, a receiving unit PROD_C5 for receiving moving images, and an image processing unit PROD_C6 for generating or processing images. Figure 3 The example in which the recording device PROD_C has all of these structures is shown, but some of them may be omitted.
[0345] It should be noted that the receiving unit PROD_C5 can receive uncoded moving images, or can receive coded data coded in a transmission coding method different from the recording coding method. In the latter case, it is preferable to place a transmission decoding unit (not shown) that decodes the coded data coded in the transmission coding method between the receiving unit PROD_C5 and the coding unit PROD_C1.
[0346] Examples of such a recording device PROD_C include a DVD recorder, a BD recorder, and a HDD (Hard Disk Drive) recorder (in this case, the input terminal PROD_C4 or the receiving unit PROD_C5 is the main source of motion images). In addition, a portable camera (in this case, the camera PROD_C3 is the main source of motion images), a personal computer (in this case, the receiving unit PROD_C5 or the image processing unit C6 is the main source of motion images), and a smart phone (in this case, the camera PROD_C3 or the receiving unit PROD_C5 is the main source of motion images) are also examples of such a recording device PROD_C.
[0347] Figure 3PROD_D is a block diagram showing the structure of a playback device PROD_D equipped with the above-mentioned moving picture decoding device 31. Figure 3 As shown, the playback device PROD_D includes a reading unit PROD_D1 for reading coded data written in the recording medium PROD_M and a decoding unit PROD_D2 for obtaining a moving picture by decoding the coded data read by the reading unit PROD_D1. The moving picture decoding device 31 described above is used as the decoding unit PROD_D2.
[0348] It should be noted that the recording medium PROD_M can be (1) a type of recording medium built into the reproduction device PROD_D such as an HDD, SSD, etc., or (2) a type of recording medium connected to the reproduction device PROD_D such as an SD memory card, USB flash memory, etc., or (3) a recording medium loaded into a drive device (not shown) built into the reproduction device PROD_D such as a DVD, BD, etc.
[0349] In addition, as a supply destination of the moving image output by the decoding unit PROD_D2, the reproduction device PROD_D may further include: a display PROD_D3 for displaying the moving image, an output terminal PROD_D4 for outputting the moving image to the outside, and a transmission unit PROD_D5 for transmitting the moving image. Figure 3 The example in which the playback device PROD_D has all of these components is shown, but some of them may be omitted.
[0350] It should be noted that the sending unit PROD_D5 can send uncoded moving images or can send coded data coded in a transmission coding method different from the recording coding method. In the latter case, it is better to place a coding unit (not shown) that encodes the moving images in the transmission coding method between the decoding unit PROD_D2 and the sending unit PROD_D5.
[0351] Examples of such a reproduction device PROD_D include a DVD player, a BD player, and an HDD player (in this case, the output terminal PROD_D4 connected to a television receiver or the like is the main supply destination of the motion image). In addition, a television receiver (in this case, the display PROD_D3 is the main supply destination of the motion image), a digital signage (also called an electronic billboard, an electronic bulletin board, etc., the display PROD_D3 or the transmission unit PROD_D5 is the main supply destination of the motion image), a desktop PC (in this case, the output terminal PROD_D4 or the transmission unit PROD_D5 is the main supply destination of the motion image), a laptop or tablet PC (in this case, the display PROD_D3 or the transmission unit PROD_D5 is the main supply destination of the motion image), and a smartphone (in this case, the display PROD_D3 or the transmission unit PROD_D5 is the main supply destination of the motion image) are also examples of such a reproduction device PROD_D. (Hardware implementation and software implementation)
[0352] Furthermore, each block of the above-described moving picture decoding device 31 and moving picture encoding device 11 may be implemented in hardware by a logic circuit formed on an integrated circuit (IC chip), or may be implemented in software by using a CPU (Central Processing Unit).
[0353] In the latter case, each of the above-mentioned devices has: a CPU that executes commands of a program that realizes each function, a ROM (Read Only Memory) that stores the above-mentioned program, a RAM (Random Access Memory) that expands the above-mentioned program, and a storage device (recording medium) such as a memory that stores the above-mentioned program and various data. Then, the purpose of the embodiment of the present invention can also be achieved in the following manner: a recording medium that records the program code (executable form program, intermediate code program, source program) of the control program of the software that realizes the above-mentioned functions in a computer-readable manner is supplied to each of the above-mentioned devices, and the computer (or CPU, MPU) reads the program code recorded in the recording medium and executes it.
[0354] As the above-mentioned recording medium, for example, the following tapes can be used: tapes such as magnetic tapes and cassette tapes; disks including floppy disks (registered trademark) / hard disks, CD-ROMs (Compact Disc Read-Only Memory) / MO disks (Magneto-Optical disc) / MD (Mini Disc) / DVD (Digital Versatile Disc) / CD-R (CD Recordable) / Blu-ray Discs (registered trademark) and the like; cards such as IC cards (including memory cards) / optical cards; semiconductor memories such as mask ROMs / EPROMs (Erasable Programmable Read-Only Memory) / EEPROMs (Electrically Erasable and Programmable Read-Only Memory) / flash ROMs; or logic circuits such as PLDs (Programmable logic device) and FPGAs (Field Programmable Gate Array), etc.
[0355] In addition, each of the above-mentioned devices can also be configured to be connected to a communication network, and the above-mentioned program code can be supplied via the communication network. The communication network can transmit program codes without being particularly limited. For example, the Internet, intranet, extranet, LAN (Local Area Network), ISDN (Integrated Services Digital Network), VAN (Value-Added Network), CATV (Community Antenna television / Cable Television) communication network, virtual private network (Virtual Private Network), telephone line network, mobile communication network, satellite communication network, etc. can be used. In addition, the transmission medium constituting the communication network can also be a medium capable of transmitting program codes, and is not limited to a specific structure or type. For example, it can be used in wired communication such as IEEE (Institute of Electrical and Electronic Engineers) 1394, USB, power line transmission, wired TV line, telephone line, ADSL (Asymmetric Digital Subscriber Line) line, or wireless communication such as IrDA (Infrared Data Association), infrared such as remote controller, BlueTooth (registered trademark), IEEE802.11 wireless, HDR (High Data Rate), NFC (Near Field Communication), DLNA (Digital Living Network Alliance, registered trademark), mobile phone network, satellite line, terrestrial digital broadcasting network, etc. It should be noted that the embodiments of the present invention can be implemented in the form of a computer data signal embedded in a carrier wave to embody the above-mentioned program code through electronic transmission.
[0356] The embodiments of the present invention are not limited to the above-mentioned embodiments, and various modifications can be made within the scope of the claims. That is, embodiments obtained by combining technical solutions appropriately modified within the scope of the claims are also included in the technical scope of the present invention. Industrial Applicability
[0357] The embodiments of the present invention can be preferably applied to a motion picture decoding device that decodes coded data obtained by encoding image data, and a motion picture encoding device that generates coded data obtained by encoding image data. In addition, the embodiments of the present invention can be preferably applied to a data structure of coded data generated by a motion picture encoding device and referred to by a motion picture decoding device. (Cross-reference of related applications)
[0358] This application claims the benefit of priority to Japanese Patent Application No. 2019-218449 filed on December 3, 2019, the entire contents of which are incorporated herein by reference. Description of the accompanying symbols 31 Image decoding device 301 Entropy Decoding Unit 302 parameter decoding unit 303 Inter-frame prediction parameter derivation unit 304 Intra-frame prediction parameter derivation unit 305, 107 loop filter 306, 109 reference picture memory 307, 108 prediction parameter memory 308, 101 prediction image generation unit 309 inter-frame prediction image generation unit 310 Intra-frame prediction image generation unit 311, 105 inverse quantization / inverse transformation unit 312, 106 addition unit 320 Prediction Parameter Derivation Department 11Image Coding Device 102 Subtraction Department 103 Transformation / Quantization Unit 104 Entropy Coding Unit 110 Coding parameter determination unit 111 parameter encoding unit 112 Inter-frame prediction parameter encoding unit 113 Intra-frame prediction parameter encoding unit 120 prediction parameter derivation unit.
Claims
1. A moving picture decoding device, characterized in that: have: a weighted prediction circuit configured to decode a first flag indicating whether there is a weighted coefficient of L0 prediction of a luminance component, a second flag indicating whether there is a weighted coefficient of L1 prediction of the luminance component, a third flag indicating whether there is a weighted coefficient of L0 prediction of a color difference component, and a fourth flag indicating whether there is a weighted coefficient of L1 prediction of the color difference component; a prediction parameter derivation circuit configured to derive a first scaling parameter indicating whether an L0 reference picture specified by a first reference picture index is scaled and a second scaling parameter indicating whether an L1 reference picture specified by a second reference picture index is scaled; and A prediction image generation circuit is configured to perform bidirectional optical flow (BDOF) prediction to generate a prediction image, wherein If a condition is met, the predicted image generation circuit sets the fifth flag to be equal to true, wherein the condition includes: the value of the first scaling parameter is 0, the value of the second scaling parameter is 0, the value of the first flag is 0, the value of the second flag is 0, the value of the third flag is 0 and the value of the fourth flag is 0, wherein the predicted image generation circuit performs the BDOF prediction based on the fifth flag being true.
