Image decoding device, image encoding device, and computer readable medium

Abstract

The invention provides an image decoding apparatus, an image encoding apparatus, and a computer readable medium. According to the present invention, in a manner in which only a part of one picture is intra-predicted and other regions are inter-predicted, it is possible to avoid the occurrence of a delay due to the generation of a large amount of code in a specific picture, but since only a part of a picture is encoded by intra-prediction, it is not possible to enlarge the amount of code in the specific picture. As a result, the encoding efficiency is reduced compared to an I picture in which the entire picture is encoded by intra prediction. A picture is divided into a restricted region and a non-restricted region, blocks of the restricted region are predicted using intra prediction referring to only pixels of the restricted region or inter prediction referring to a restricted reference region of a reference picture, and blocks of the non-restricted region are predicted using intra prediction referring to only pixels of the restricted region or inter prediction referring to a restricted reference region of the reference picture. The prediction is performed using intra prediction of decoded pixels in a reference picture or inter prediction of a reference picture, and after picture decoding, the restricted region is set as a restricted reference region.

Description

This application is a divisional application of application No. 201980052552.8 entitled “Motion Image Decoding Device and Motion Image Encoding Device” filed with the State Intellectual Property Office of China on August 6, 2019. Technical Field

[0001] One aspect of an embodiment of the present invention relates to a predicted image generation device, a moving image decoding device, and a moving image encoding device. Background Art

[0002] In order to efficiently transmit or record moving images, a moving image encoding device that generates encoded data by encoding the moving image and a moving image decoding device that generates a decoded image by decoding the encoded data are used.

[0003] As specific moving picture coding methods, for example, methods proposed in H.264 / AVC and HEVC (High-Efficiency Video Coding) can be cited.

[0004] In such a motion image coding method, images (pictures) constituting the motion image are managed through a hierarchical structure and encoded / decoded for each CU. The hierarchical structure includes slices obtained by dividing the image, coding tree units (CTU: Coding Tree Unit) obtained by dividing the slices, coding units (sometimes also called coding units (Coding Unit: CU)) obtained by dividing the coding tree units, and transform units (TU: Transform Unit) obtained by dividing the coding units.

[0005] In such a moving picture coding method, a prediction image is usually generated based on a local decoded image obtained by encoding / decoding an input image, and a prediction error (sometimes also referred to as a "difference image" or "residual image") obtained by subtracting the prediction image from the input image (original image) is encoded. Examples of methods for generating a prediction image include inter-picture prediction (inter-frame prediction) and intra-picture prediction (intra-frame prediction).

[0006] In addition, Non-Patent Document 1 can be cited as a recent moving picture encoding and decoding technology. Prior art literature Non-patent literature

[0007] Non-patent document 1: “Algorithm Description of Joint Exploration Test Model 7”, JVET-G1001, Joint Video Exploration Team (JVET) of ITU-TSG 16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11, 2017-08-19 Non-patent document 2: “Improved Cyclic Intra Refresh”, JVET-K0212, Joint Video Exploration Team (JVET) of ITU-T SG 16WP 3 and ISO / IEC JTC I / SC 29 / WG 11, 2018-07-10 Summary of the invention

[0008] Figure 1 This is a diagram showing an example of the amount of generated code for each picture when encoding a certain moving image, where the horizontal axis represents time (continuous pictures) and the vertical axis represents the amount of generated code for each picture. Problem that the invention aims to solve

[0009] However, the ratio of the amount of code for intra-frame prediction and inter-frame prediction is very different. Figure 1 This is a diagram showing an example of the amount of code generated for each picture when encoding a certain moving image, where the horizontal axis represents time (continuous pictures) and the vertical axis represents the amount of code generated for each picture. Figure 1 As shown, an intra picture (I picture) generates several times the amount of code as an inter picture. Therefore, a significant delay occurs when an I picture is transmitted. On the other hand, there is a method in which, instead of performing intra-frame prediction on an entire picture, intra-frame prediction is performed on only a portion of a picture, and inter-frame prediction is performed on the remaining area (there may also be an intra-frame predicted area) for intra-frame refresh. In this method, by performing intra-frame prediction on multiple pictures little by little, it is possible to avoid the situation in which a huge amount of code is generated in a specific picture and a delay occurs. However, since only a portion of a picture is encoded by intra-frame prediction, there is also a disadvantage in that the encoding efficiency is reduced compared to an I picture in which the entire picture is encoded by intra-frame prediction.

[0010] Therefore, the present invention has been made in view of the above-mentioned problems, and an object of the present invention is to provide a mechanism for encoding and decoding moving images with less delay while suppressing a decrease in encoding efficiency compared to I pictures without impairing the functionality of random access. Technical Solution

[0011] A motion image decoding device of one embodiment of the present invention divides a picture into a restricted area and a non-restricted area, and predicts the blocks included in the restricted area using intra-frame prediction with reference only to pixels in the restricted area of ​​the picture or inter-frame prediction with reference to a restricted reference area of ​​a reference picture of the picture, and predicts the blocks included in the non-restricted area using intra-frame prediction with reference to decoded pixels in the picture or inter-frame prediction with reference to a reference picture of the picture, and after decoding the picture, the restricted area of ​​the picture is set as the restricted reference area. Beneficial Effects

[0012] According to one embodiment of the present invention, a refresh mechanism can be introduced into a moving image to generate coded data with a small change in the amount of code while enabling random access and suppressing the reduction in coding efficiency. Thus, coding with a substantially fixed bit rate can be achieved through the coded data, and the delay generated in each I picture can be avoided. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 This is a diagram showing an example of the amount of generated code for each picture when encoding a certain moving image. Figure 2 This is a diagram showing the hierarchical structure of the data of the coded stream. Figure 3 This is a diagram showing an example of CTU segmentation. Figure 4 This is a conceptual diagram showing an example of a reference picture and a reference picture list. Figure 5 This is a schematic diagram showing the types (mode numbers) of intra prediction modes. Figure 6 It is a diagram for explaining the restricted area and the non-restricted area of ​​the present invention. Figure 7 This is a diagram for explaining a range that can be referred to by a target block of the present invention. Figure 8 This is a diagram for explaining the setting of the restricted reference area. Fig. 9 It is a diagram for explaining the phased refresh area of ​​the present invention. Fig.10 It is a diagram for explaining the new refresh area, restricted area, and non-restricted area of ​​the present invention. Fig.11 It is a diagram for explaining the new refresh area of ​​the present invention. Fig.12 This is a diagram illustrating the relationship between a new refresh area and a CU. Fig.13 It is a diagram for explaining the types of new refresh areas of the present invention. Fig.14is another diagram illustrating the new refresh area of ​​the present invention. Fig.15 This is a diagram that explains the syntax required for phased refresh. Fig.16 It is a diagram for explaining the relationship between the restriction area and the overlap area. Fig.17 This is a schematic diagram showing the structure of a moving picture decoding device. Fig.18 This is a block diagram showing the structure of a moving picture encoding device. Fig.19 This is a schematic diagram showing the structure of an inter-frame prediction parameter decoding unit. Fig. 20 It is a schematic diagram showing the structure of the combined prediction parameter derivation unit and the AMVP prediction parameter derivation unit. Fig.21 This is a schematic diagram showing the structure of an intra-frame prediction parameter decoding unit. Fig. 22 This is a schematic diagram showing the structure of an inter-frame prediction image generation unit. Fig.23 is a diagram illustrating border filling and border motion vector clipping. Fig.24 This is a diagram showing a reference area used for intra prediction. Fig.25 This is a diagram showing the structure of an intra-frame prediction image generation unit. Fig.26 is a diagram for explaining reference pixels of a reference pixel filter Fig. 27 This is a diagram showing the relationship between the restricted area and the intra prediction mode (mode number) during intra prediction. Fig.28 This is a schematic diagram showing the structure of the inter-frame prediction parameter encoding unit. Fig.29 This is a schematic diagram showing the structure of an intra-frame prediction parameter encoding unit. Fig.30 This is a diagram showing the configuration of a transmitting device equipped with a moving picture encoding device according to this embodiment and a receiving device equipped with a moving picture decoding device. Fig.30 (a) represents a transmitting device equipped with a motion picture encoding device, Fig.30 (b) represents a receiving device equipped with a motion picture decoding device. Fig.31 This is a diagram showing the configuration of a recording device equipped with a moving picture encoding device according to this embodiment and a reproducing device equipped with a moving picture decoding device. Fig.31 (a) represents a recording device equipped with a motion picture encoding device, Fig.31(b) shows a reproduction device equipped with a motion picture decoding device. Fig.32 This is a schematic diagram showing the configuration of the image transmission system according to the present embodiment. Fig.33 It is a diagram illustrating deblocking filtering. Fig.34 This is a diagram for explaining the relationship between the restriction area and the deblocking filter. Fig.35 This is a diagram for explaining the relationship between the restriction area and the deblocking filter area. Fig.36 This is a flowchart illustrating the operation of deblocking filtering. DETAILED DESCRIPTION (First Embodiment)

[0014] Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[0015] Fig.32 It is a schematic diagram showing the configuration of the image transmission system 1 according to the present embodiment.

[0016] The image transmission system 1 is a system for transmitting a coded stream obtained by encoding a coding target image, decoding the transmitted coded stream and displaying the image. The image transmission system 1 is configured to include: a moving image encoding device (image encoding device) 11, a network 21, a moving image decoding device (image decoding device) 31, and a moving image display device (image display device) 41.

[0017] The video image encoding device 11 receives the image T as input.

[0018] The network 21 transmits the coded stream Te generated by the motion image coding device 11 to the motion image decoding device 31. The network 21 is the Internet, a wide area network (WAN), a small network (LAN) or a combination thereof. The network 21 is not necessarily limited to a two-way communication network, but may also be a one-way communication network that transmits broadcast waves such as terrestrial digital broadcasting and satellite broadcasting. In addition, the network 21 may also be replaced by a storage medium such as a DVD (Digital Versatile Disc: Digital Versatile Disc, registered trademark) or a BD (Blu-ray (registered trademark) Disc: Blu-ray Disc, registered trademark) that records the coded stream Te.

[0019] The moving picture decoding device 31 decodes each of the coded streams Te transmitted through the network 21 , and generates one or more decoded pictures Td after decoding.

[0020] The moving image display device 41 displays all or part of one or more decoded images Td generated by the moving image decoding device 31. The moving image display device 41 includes, for example, a liquid crystal display, an organic EL (Electro-luminescence) display, and other display devices. Examples of the display type include a fixed type, a mobile type, and an HMD. In addition, when the moving image decoding device 31 has high processing power, a high-quality image is displayed, and when it has only low processing power, an image that does not require high processing power or high display power is displayed. <Operator>

[0021] The operators used in this specification are described below.

[0022] >> is a right shift, << is a left shift, & is a bitwise AND, | is a bitwise OR, |= is an OR substitution operator, and || represents a logical AND.

[0023] x?y:z is a three-term operator that takes y when x is true (other than 0) and takes z when x is false (0).

[0024] Clip3(a, b, c) is a function that clips c to a value greater than a and less than b. It returns a when c<a, returns b when c>b, and returns c in other cases (where a<=b).

[0025] abs(a) is a function that returns the absolute value of a.

[0026] Int(a) is a function that returns the integer value of a.

[0027] floor(a) is a function that returns the largest integer below a.

[0028] ceil(a) is a function that returns the largest integer above a.

[0029] a / d means a divided by d (discarding decimal places). <Structure of coded stream Te>

[0030] Before describing in detail the moving picture encoding device 11 and the moving picture decoding device 31 of the present embodiment, the data structure of the coded stream Te generated by the moving picture encoding device 11 and decoded by the moving picture decoding device 31 will be described.

[0031] Figure 2 : is a diagram showing the hierarchical structure of data in the coded stream Te. The coded stream Te exemplarily includes a sequence and a plurality of pictures constituting the sequence. Figure 2(a) to (f) are diagrams respectively representing a coded video sequence of a given sequence SEQ, a coded picture of a given picture PICT, a coded slice of a given slice S, coded slice data of the given slice data, a coding tree unit CTU included in the coded slice data, and a coding unit CU included in the coding tree unit. (Encoded video sequence)

[0032] In the coded video sequence, a set of data is defined for reference by the moving image decoding device 31 in order to decode the sequence SEQ to be processed. Figure 2 As shown in (a), it includes: video parameter set (Video Parameter Set), sequence parameter set SPS (Sequence Parameter Set), picture parameter set PPS (Picture Parameter Set), picture PICT and supplemental enhancement information SEI (Supplemental Enhancement Information).

[0033] The video parameter set VPS defines a set of coding parameters common to a plurality of moving images in a moving image composed of multiple layers, and a set of coding parameters associated with the multiple layers included in the moving image and with each layer.

[0034] In the sequence parameter set SPS, a set of coding parameters is specified for reference by the motion image decoding device 31 in order to decode the object sequence. For example, the width and height of the picture are specified. It should be noted that there can be multiple SPSs. In this case, any one of the multiple SPSs is selected from the PPS.

[0035] In the picture parameter set PPS, a set of coding parameters is specified for reference by the motion image decoding device 31 to decode each picture in the target sequence. For example, it includes a reference value (pic_init_qp_minus26) of the quantization width used for decoding the picture and a flag (weighted_pred_flag) indicating the application of weighted prediction. It should be noted that there can be multiple PPSs. In this case, any one of the multiple PPSs is selected from each picture in the target sequence. (Encoded image)

[0036] In the coded picture, a set of data is defined to be referred to by the video decoding device 31 in order to decode the picture PICT to be processed. Figure 2 As shown in (b), it includes slices 0 to NS-1 (NS is the total number of slices included in the picture PICT).

[0037] It should be noted that, when there is no need to distinguish between slices 0 to NS-1, the code subscript may be omitted for description. The same applies to other data with subscripts included in the coded stream Te described below. (Encoding slice)

[0038] In the coded slice, a set of data is defined for reference by the video decoding device 31 to decode the slice S to be processed. Figure 2 (c) includes a slice header and slice data.

[0039] The slice header includes a coding parameter group that the moving picture decoding device 31 refers to in order to determine a decoding method of a target slice. Slice type designation information (slice_type) that designates a slice type is one example of a coding parameter included in the slice header.

[0040] Examples of slice types that can be specified by the slice type specification information include: (1) I slices that use only intra-frame prediction when encoding, (2) P slices that use unidirectional prediction or intra-frame prediction when encoding, and (3) B slices that use unidirectional prediction, bidirectional prediction, or intra-frame prediction when encoding. It should be noted that inter-frame prediction is not limited to unidirectional prediction and bidirectional prediction, and more reference pictures can be used to generate predicted images. Hereinafter, the cases referred to as P and B slices refer to slices that include blocks that can use inter-frame prediction.

[0041] It should be noted that the slice header may also include a reference to the picture parameter set PPS (pic_parameter_set_id). (Encoded slice data)

[0042] The coded slice data specifies a set of data that the video decoding device 31 refers to in order to decode the slice data to be processed. Figure 2 (d) includes CTU. CTU is a block of a fixed size (for example, 64×64) constituting a slice, and is also called the largest coding unit (LCU). (Coding Tree Unit CTU)

[0043] exist Figure 2In (e), a set of data is specified for reference by the motion image decoding device 31 in order to decode the CTU of the processing object. The CTU is divided into coding units CU, which are basic units of encoding processing, by recursive quadtree partitioning (QT (Quad Tree) partitioning), binary tree partitioning (BT (Binary Tree) partitioning) or ternary tree partitioning (TT (Ternary Tree) partitioning). BT partitioning and TT partitioning are collectively referred to as multi-tree partitioning (MT (Multi Tree) partitioning). The nodes of the tree structure obtained by recursive quadtree partitioning are called coding nodes (Coding Node). The intermediate nodes of the quadtree, binary tree and ternary tree are coding nodes, and the CTU itself is also specified as the top-level coding node.

[0044] CT includes the following information as CT information: a QT split flag (cu_split_flag) indicating whether QT splitting is performed, an MT splitting mode (split_mt_mode) indicating whether MT splitting is performed, an MT splitting direction (split_mt_dir) indicating the splitting direction of MT splitting, and an MT splitting type (split_mt_type) indicating the splitting type of MT splitting. cu_split_flag, split_mt_flag, split_mt_dir, and split_mt_type are transmitted for each coding node.

[0045] When cu_split_flag is 1, the coding node is split into 4 coding nodes ( Figure 3 (b)). When cu_split_flag is 0 and split_mt_flag is 0, the coding node is not split and one CU is kept as a node ( Figure 3 (a)). CU is the terminal node of the coding node and is not further divided. CU is the basic unit of the coding process.

[0046] When split_mt_flag is 1, the coding node is MT split as follows. When split_mt_type is 0 and split_mt_dir is 1, the coding node is horizontally split into two coding nodes ( Figure 3 (d)), when split_mt_dir is 0, the coding node is vertically split into 2 coding nodes ( Figure 3 In addition, when split_mt_type is 1 and split_mt_dir is 1, the coding node is horizontally split into 3 coding nodes ( Figure 3(f)), when split_mt_dir is 0, the coding node is vertically split into 3 coding nodes ( Figure 3 (e)).

[0047] In addition, when the size of CTU is 64×64 pixels, the size of CU can be any one of 64×64 pixels, 64×32 pixels, 32×64 pixels, 32×32 pixels, 64×16 pixels, 16×64 pixels, 32×16 pixels, 16×32 pixels, 16×16 pixels, 64×8 pixels, 8×64 pixels, 32×8 pixels, 8×32 pixels, 16×8 pixels, 8×16 pixels, 8×8 pixels, 64×4 pixels, 4×64 pixels, 32×4 pixels, 4×32 pixels, 16×4 pixels, 4×16 pixels, 8×4 pixels, 4×8 pixels and 4×4 pixels. (Coding Unit CU)

[0048] like Figure 2 As shown in (f), a set of data is specified for reference by the video decoding device 31 to decode the CU to be processed. Specifically, the CU is composed of a CU header CUH, prediction parameters, transform parameters, quantized transform coefficients, etc. The prediction mode, etc. are specified in the CU header.

[0049] The prediction process may be performed in units of CU or in units of sub-CUs obtained by further dividing the CU. When the size of the CU and the sub-CU are equal, there is one sub-CU in the CU. When the size of the CU is larger than the size of the sub-CU, the CU is divided into sub-CUs. For example, when the CU is 8×8 and the sub-CU is 4×4, the CU is divided into 4 sub-CUs, including two parts divided horizontally and two parts divided vertically.

[0050] There are two types of prediction (prediction modes): intra prediction and inter prediction. Intra prediction is prediction within the same picture, while inter prediction refers to prediction processing between different pictures (for example, between display times, between layer images).

[0051] The transform / quantization process is performed in units of CUs, but the quantized transform coefficients may be entropy-coded in units of 4×4 or other sub-blocks. (Prediction parameters)

[0052] The predicted image is derived from prediction parameters attached to the block. The prediction parameters include prediction parameters for intra-frame prediction and inter-frame prediction.

[0053] The following describes the prediction parameters for inter-frame prediction. The inter-frame prediction parameters are composed of prediction list utilization flags predFlagL0 and predFlagL1, reference picture indexes refIdxL0 and refIdxL1, and motion vectors mvL0 and mvL1. The prediction list utilization flags predFlagL0 and predFlagL1 are flags indicating whether to use the reference picture lists called L0 list and L1 list, respectively. When the value is 1, the corresponding reference picture list is used. It should be noted that in the present specification, when it is described as "a flag indicating whether it is ××", a flag other than 0 (for example, 1) is set to be ××, and a flag of 0 is set to be not ××. In logical NOT, logical product, etc., 1 is regarded as true and 0 is regarded as false (the same applies below). However, other values ​​can also be used as true values ​​and false values ​​in actual devices and methods.

[0054] The syntax elements used to derive the inter-frame prediction parameters include, for example, a merge flag merge_flag, a merge index merge_idx, an inter-frame prediction identifier inter_pred_idc, a reference picture index refIdxLX, a prediction vector index mvp_LX_idx, and a difference vector mvdLX. (See picture list)

[0055] The reference picture list is a list consisting of reference pictures stored in the reference picture memory 306 . Figure 4 : is a conceptual diagram showing an example of a reference picture and a reference picture list in a picture structure for low delay. Figure 4 In (a), the rectangle represents the picture, the arrow represents the reference relationship of the picture, the horizontal axis represents time, the I, P, and B in the rectangle represent intra-frame pictures, unidirectional prediction pictures, and bidirectional prediction pictures, respectively, and the numbers in the rectangle represent the decoding order. Figure 4 As shown in (a), the decoding order of the pictures is I0, P1 / B1, P2 / B2, P3 / B3, P4 / B4, and the display order is also the same. Figure 4 (b) of FIG. 4 shows an example of a reference picture list of picture B3 (target picture). A reference picture list is a list indicating candidates for reference pictures, and one picture (slice) can have more than one reference picture list. Figure 4 In the example of (b), the target picture B3 has two reference picture lists, L0 list RefPicList0 and L1 list RefPicList1. In each CU, a reference picture index refIdxLX specifies which picture in the reference picture list RefPicListX (X=0 or 1) is actually referenced. Figure 4(b) is an example of refIdxL0=2, refIdxL1=0. When the target picture is P3, the reference picture list is only the L0 list. It should be noted that LX is a description method used when L0 prediction and L1 prediction are not distinguished. In the following, the parameters for the L0 list and the parameters for the L1 list are distinguished by replacing LX with L0 and L1. (Combined forecast and AMVP forecast)

[0056] The decoding (encoding) methods of the prediction parameters include a merge prediction (merge) mode and an AMVP (Adaptive Motion Vector Prediction) mode, and the merge flag merge_flag is a flag for identifying them.

[0057] The merge prediction mode is a mode in which the prediction list utilization flag predFlagLX (or inter-frame prediction identifier inter_pred_idc), the reference picture index refIdxLX, and the motion vector mvLX are not included in the coded data, but are derived from the prediction parameters of the processed nearby blocks and used. The merge index merge_idx is an index indicating whether any prediction parameter among the prediction parameter candidates (merge candidates) derived from the processed block is used as the prediction parameter of the target block.

[0058] The AMVP mode is a mode that includes the inter-frame prediction identifier inter_pred_idc, the reference picture index refIdxLX, and the motion vector mvLX in the encoded data. It should be noted that the motion vector mvLX is encoded as the prediction vector index mvp_LX_idx and the differential vector mvdLX that identify the prediction vector mvpLX. The inter-frame prediction identifier inter_pred_idc is a value indicating the type and number of reference pictures, and takes any value among PRED_L0, PRED_L1, and PRED_BI. PRED_L0 and PRED_L1 respectively represent unidirectional prediction using a reference picture managed in the L0 list and the L1 list. PRED_BI represents bidirectional prediction BiPred using two reference pictures managed in the L0 list and the L1 list. (Motion Vector)

[0059] The motion vector mvLX indicates the amount of displacement between blocks on two different pictures. The prediction vector and the difference vector related to the motion vector mvLX are referred to as a prediction vector mvpLX and a difference vector mvdLX, respectively.

[0060] The prediction parameters of the intra prediction are described below. The intra prediction parameters are composed of a luma prediction mode IntraPredModeY and a chroma prediction mode IntraPredModeC. Figure 5 is a schematic diagram showing the types (mode numbers) of intra-frame prediction modes. Figure 5 As shown, there are 67 intra prediction modes (0 to 66), for example, Planar prediction (0), DC prediction (1), Angular prediction (2 to 66). Furthermore, LM mode (67 to 72) can be added in chrominance.

[0061] The syntax elements for deriving intra prediction parameters include, for example, prev_intra_luma_pred_flag, mpm_idx, rem_selected_mode_flag, rem_selected_mode, rem_non_selected_mode, and the like. (MPM)

[0062] prev_intra_luma_pred_flag is a flag indicating whether the luminance prediction mode Intra Pred ModeY of the object block is consistent with MPM (Most Probable Mode). MPM is a prediction mode included in the MPM candidate list mpmCandList[]. The MPM candidate list is a list that stores candidates with high probability of being applied to the object block based on the intra prediction mode of the adjacent block and the specified intra prediction mode. When prev_intra_luma_pred_flag is 1, the luminance prediction mode IntraPredModeY of the object block is derived using the MPM candidate list and the index mpm_idx. IntraPredModeY=mpmCandList[mpm_idx] (REM)

[0063] When prev_intra_luma_pred_flag is 0, the intra prediction mode is selected from the mode RemIntraPredMode remaining after removing the intra prediction mode included in the MPM candidate list from all intra prediction modes. The intra prediction mode selectable as RemIntraPredMode is called "non-MPM" or "REM". The flag rem_selected_mode_flag is a flag that specifies whether the intra prediction mode is selected with reference to rem_selected_mode or with reference to rem_non_selected_mode. RemIntraPredMode is derived using rem_selected_mode or rem_non_selected_mode. (Additional restricted encoding / decoding areas)

[0064] A motion image encoding and decoding method is described, which is characterized in that a partial area is set in the same picture, pixels of other areas are not used for encoding and decoding in the partial area, and the entire picture is used for encoding and decoding in other areas.

[0065] Figure 6 : This is a diagram for explaining area A and area B of the present invention. In the motion image encoding device and decoding device of the present invention, area A and area B in a picture are set. In area A, prediction processing can be performed only based on area A, and padding and other processing can be performed outside the area. In area B, prediction processing can be performed based on the entire picture including area A. Here, prediction processing refers to intra-frame prediction, inter-frame prediction, loop filtering processing, etc. In area A, encoding processing / decoding processing is turned off in area A, so only decoding can be performed on area A.

[0066] Hereinafter, area A is referred to as a restricted area (first area, control area, clean area, refreshed area, area A). Conversely, areas outside the restricted area are referred to as non-restricted areas (second area, non-control area, dirty area, unrefreshed area, area B, outside the restricted area).

[0067] For example, an area that is encoded / decoded only by intra-frame prediction and has been encoded in intra-frame prediction (a new refresh area FRA described later) is a restricted area. An area that is further encoded / decoded with reference to the restricted area formed by intra-frame prediction is also a restricted area. Moreover, an area that is encoded / decoded with reference to a restricted area in a reference picture is also a restricted area. That is, a restricted area refers to an area that is encoded / decoded with reference to a restricted area.

[0068] Hereinafter, the upper left position of the restriction area is represented by (xRA_st, yRA_st), the lower right position is represented by (xRA_en, yRA_en), and the size is represented by (wRA, hRA). In addition, the position and the size have the following relationship, so one can be derived from the other. xRA_en=xRA_st+wRA-1 yRA_en=yRA_st+hRA-1

[0069] Alternatively, derivation can be performed as follows. wRA=xRA_en-xRA_st+1 hRA=yRA_en-yRA_st+1

[0070] Furthermore, the upper left position of the restricted reference area at time j is represented by (xRA_st[j], yRA_st[j]), the lower right position is represented by (xRA_en[j], yRA_en[j]), and the size is represented by (wRA[j], hRA[j]). In addition, the position of the restricted reference area of ​​the reference picture Ref may be represented by (xRA_st[Ref], yRA_st[Ref]), the lower right position is represented by (xRA_en[Ref], yRA_en[Ref]), and the size is represented by (wRA[Ref], hRA[Ref]). (Determination of restricted areas)

[0071] For example, when a certain image is at time i and the position of a certain block is (x, y), the following formula can be used to determine whether the pixel at the position is within the restricted area. IsRA(x,y)=(xRA_st[i]<=x&&x<=xRA_en[i]&&yRA_st[i]<=y&&y<=yRA_en[i])

[0072] Alternatively, the following determination formula may be used. IsRA(x,y)=(xRA_st[i]<=x&&x<xRA_st[i]+wRA[i]&&yRA_st[i]<=y&&y<yRA_st[i]+hRA[i]) IsRA(xRef, yRef)=(xRA_st[Ref]<=xRef&&xRef<=xRA_en[Ref]&&yRA_st[Ref]<=yRef&&yRef<=yRA_en[Ref])

[0073] For example, when the object picture is at time i, the upper left coordinate of the object block Pb is (xPb, yPb), and the width and height are bW and bH, the intra-frame prediction unit, motion compensation unit, and loop filter of the motion image decoding device and the motion image encoding device derive IsRA(Pb) through the following judgment formula when the object block Pb is within the restricted area. IsRA(Pb)=(xRA_st[i]<=xPb&&xPb<=xRA_en[i]&&yRA_st[i]<=yPb&&yPb<=yRA_en[i])

[0074] Alternatively, the following determination formula may be used. IsRA(Pb)=(xRA_st[i]<=xPb&&xPb<xRA_st[i]+wRA[i]&&yRA_st[i]<=yPb&&yPb<yRA_st[i]+hRA[i]) (Basic action of limiting the reference area)

[0075] The moving picture encoding device and the moving picture decoding device of this specification perform the following operations.

[0076] Figure 7 This is a diagram showing a range in which a restricted area can be referred to in intra-frame prediction, inter-frame prediction, and loop filtering in the present invention. Figure 7 (a) indicates a range that the object block included in the restricted area can refer to. The coded / decoded area included in the restricted area in the same picture (object picture) as the object block is a range that the object block can refer to in intra-frame prediction, inter-frame prediction, and loop filtering. Similarly, the restricted area in the reference picture (restricted reference area) is a range that the object block can refer to in inter-frame prediction and loop filtering. Figure 7 (b) indicates the range that the object block included in the unrestricted area can refer to. The coded / decoded area in the object picture is the range that the object block can refer to in intra-frame prediction and inter-frame prediction. Similarly, all areas in the reference picture are the range that can be referenced in inter-frame prediction. It should be noted that in the case of parallel processing and reference restrictions such as tiles, slices, and wavefronts, other restrictions are also added in addition to the above. .The object block included in the restricted area is intra-frame predicted with reference only to pixels in the restricted area in the object picture or inter-frame predicted with reference to the restricted reference area in the reference picture. The target block included in the restricted area refers to the coding parameters (such as intra prediction direction, motion vector, reference picture index) of the restricted area in the target picture or refers to the coding parameters of the restricted reference area of ​​the reference picture to derive the coding parameters of the target block. The target block included in the restricted area is subjected to loop filtering processing with reference to only the pixels in the restricted area in the target picture. (Determination of restricted areas and availability of use)

[0077] In the MPM derivation of intra prediction, the merge candidate derivation of inter prediction, etc., the prediction parameters of the adjacent area are used to derive the prediction parameters (intra prediction mode, motion vector) of the object block. The following processing can be performed in such a case. In the case where the object block is in the restricted area (IsRA(xPb, yPb) is true) and the reference position (xNbX, yNbX) of the adjacent block of the object block is in the non-restricted area (IsRA(xNbX, yNbX) is false) in the intra prediction and inter prediction, the value of the adjacent block is not used in the prediction parameter derivation. That is, when the object block is in the restricted area (IsRA(xPb, yPb) is true) and the reference position (xNbX, yNbX) of the adjacent block of the object block is in the restricted area (IsRA(xNbX, yNbX) is true), the prediction parameters of the position (xNbX, yNbX) are used for the derivation of the prediction parameters of the object block.

[0078] It should be noted that, although the above description is made by deriving the prediction candidate, it can also be applied to the loop filtering process. In addition, the determination of the restricted area can also be used for the determination outside the entire screen in the same way as the determination outside the screen and the parallel processing unit (slice boundary, tile boundary). That is, when the object block is in the restricted area (IsRA (xPb, yPb) is true) and the reference position (xNbX, yNbX) of the object block is in the non-restricted area (IsRA (xNbX, yNbX) is false), the reference position (xNbX, yNbX) is determined to be unreferenceable (unavailable) (availableNbX = 0). That is, when the object block is in the screen, and the object block and the reference position are not in different parallel processing units, the object block is in the restricted area or the reference position (xNbX, yNbX) of the object block is in the restricted area (IsRA (xNbX, yNbX) is true), the reference position (xNbX, yNbX) is determined to be referenceable (availableNbX = 1). In intra-frame prediction and inter-frame prediction, when the reference position (xNbX, yNbX) is referenceable (available) (availableNbX=1), the prediction parameters of the reference position are used to derive the prediction parameters of the object block. In loop filtering, when the reference position (xNbX, yNbX) is available (availableNbX=1), deblocking filtering is performed. (Determination of restricted reference area and cropping of restricted reference area)

[0079] In addition, when the upper left position of the reference pixel in the reference picture at time j is (xRef, yRef), the motion compensation unit derives that the reference pixel is within the restricted reference area using the following determination formula. IsRA(xRef, yRef)=(xRA_st[j]<=xRef&&xRef<=xRA_en[j]&&yRA_st[j]<=yRef&&yRef<=yRA_en[j])

[0080] Alternatively, the following determination formula may be used. IsRA(xRef, yRef)=(xRA_st[j]<=xRef&&xRef<xRA_st[j]+wRA[j]&&yRA_st[i]<=yRef&&yRef<yRA_st[j]+hRA[j])

[0081] Alternatively, the motion compensation unit may use the following formula to clip the reference pixel to a position within the restricted area. xRef=Clip3(xRA_st[j],xRA_en[j],xRef) yRef=Clip3(yRA_st[j], yRA_en[j], yRef)

[0082] Or it can be the following derivation. xRef=Clip3(xRA_st[j],xRA_st[j]+wRA[j]-1,xRef) yRef=Clip3(yRA_st[j], yRA_st[j]+hRA[j]-1, yRef)

[0083] It should be noted that the position of the restricted area is transmitted from the motion image encoding device to the motion image decoding device through the staged refresh information described later. It should be noted that the position and size of the restricted area can also be derived not based on the time (such as POC), but by setting the reference image Ref in the reference memory after decoding the target image or at the decoding start time of the target image. In this case, the position and size of the restricted area can be derived by specifying the reference image Ref. (Setting / updating of restricted reference area) .Set the above-mentioned restricted area of ​​the image to the above-mentioned restricted reference area when the image is set as a reference image.

[0084] For example, when a picture of a restricted area having upper left coordinates (xRA_st, yRA_st), lower right coordinates (xRA_en, yRA_en), and size (wRA, hRA) is referenced as a reference picture at time j, the restricted reference area can be set by the following formula. xRA_st[j]=xRA_st, xRA_en[j]=xRA_en, wRA[j]=wRA yRA_st[j]=yRA_st, yRA_en[j]=xRA_en, hRA[j]=hRA

[0085] Figure 8 This is a diagram showing updates of the restricted reference area of ​​the reference picture in the reference memory. For example, the restricted reference area of ​​the reference picture j in the reference memory decodes the restricted area of ​​the object picture at time i, that is, the upper left coordinates (xRA_st[j], yRA_st[j]), the lower right coordinates (xRA_en[j], yRA_en[j]), and the size (wRA[j], hRA[j]). The restricted reference area when the object picture i is decoded and stored as the reference picture i in the reference memory can be as follows.

[0086] Restricted reference area of ​​reference image j: upper left coordinate (xRA_st[j], yRA_st[j]), lower right coordinate (xRA_en[j], yRA_en[j]), size (wRA[j], hRA[j])

[0087] Restricted reference area of ​​reference picture i: upper left coordinate (xRA_st[i], yRA_st[i]), lower right coordinate (xRA_en[i], yRA_en[i]), size (wRA[i], hRA[i])

[0088] Furthermore, as described later, when the restricted regions are overlapped, the restricted reference region of the reference picture may be smaller than the restricted reference region at the time when the target picture is decoded.

[0089] Restricted reference area of ​​reference image j: upper left coordinate (xRA_st[j], yRA_st[j]), lower right coordinate (xRA_en[j], yRA_en[j]), size (wRA[j], hRA[j])

[0090] Restricted reference area of ​​reference image i: upper left coordinate (xRA_st[i], yRA_st[i]), lower right coordinate (xRA_en[i]-wOVLP, yRA_en[i]), size (wRA[i]-wOVLP, hRA[i])

[0091] Furthermore, as described later, the restricted region of the reference picture at time j (=i) can be set using the restricted region at time i-1 and the new refresh area FRA of the target picture. In a sequentially decoder refresh picture (SDR picture), all referenceable areas (restricted reference areas) are cleared from restricted areas in the reference picture and set as non-referenceable.

[0092] For example, a flag indicating whether a restricted reference area is set may be set to 0 in the reference picture at time j, or the restricted reference area may be removed using the following formula. xRA_st[j]=0, xRA_en[j]=0, wRA[j]=0 yRA_st[j]=0, yRA_en[j]=0, hRA[j]=0

[0093] In addition, the restricted area in the reference image is also referred to as the restricted reference area (first reference area, non-controlled reference area, clean reference area, refreshed reference area, reference area A), and the non-restricted area in the reference image is also referred to as the non-restricted reference area (second reference area, non-controlled reference area, dirty reference area, unrefreshed reference area, reference area B). (Phase refresh)

[0094] The refresh (stage refresh, staged decoding refresh) of the present invention is briefly described. The stage refresh of the present invention refers to the following technology: a restricted area consisting of intra-frame prediction is transmitted in stages, and then the size of the restricted area is expanded to the entire screen in subsequent pictures, thereby refreshing the entire picture in stages. It should be noted that in this specification, the area newly added as a restricted area in the target picture is called a fresh refresh area (Fresh Refresh Area).

[0095] Fig. 9This is a diagram for explaining the refresh (staged refresh) of the present invention. In the staged refresh of this specification, a restricted area (new refresh area FRA) composed of intra-frame prediction is inserted into the picture as the starting point of the refresh. The picture as the starting point of the refresh is called an SDR picture (Sequentially Decoder Refresh, random access point). The area encoded using only the intra-frame prediction mode is called a new refresh area FRA (Forced Refresh Area: forced refresh area).

[0096] exist Fig. 9 (a) shows the restricted area at each time t. Fig. 9 As shown in (a), in the SDR picture (t=0), all restricted areas composed of intra-frame predictions are decoded. In the SDR picture, all restricted reference areas in the reference picture are cleared and set as non-referenceable. Then, the restricted area is expanded as it extends from t=0 to numIR-1, and the entire picture is refreshed at t=numIR-1. Fig. 9 (b) represents the restricted reference area on the reference image. Fig. 9 In (b), when the picture at time t is used as a reference picture, the restricted reference area is indicated by the horizontal line area. It can also be a structure in which SDR pictures are inserted periodically. In this specification, SDR pictures are inserted according to each SDR period SDRperiod.

[0097] Fig.10 This is a diagram that clearly shows the newly added area (new refresh area) in the target picture in the restricted area. In the staged refresh, the new refresh area FRA can be added in stages, and the restricted area can be newly set including the added FRA, thereby refreshing the entire picture. It should be noted that all parts of the new refresh area FRA can also be set as an area consisting only of intra-frame prediction (Intra Refresh Area, IRA). By setting the new refresh area FRA to the intra-frame prediction IRA in this way, the possibility of normal decoding (error tolerance) can be further improved.

[0098] It should be noted that the restricted area shall not refer to pictures before the starting picture of the SDR cycle. Fig.10 At time t=2 in (a), the picture referenced by the region k=0 or k=1 is Fig.10 (b) is the picture at time t=0 or t=1 in the reference picture memory. (Example of phased refresh configuration)

[0099] Fig.11 (a) is an example of dividing one picture into numIR regions (refresh unit regions). In the following, it is assumed that numIR=5. Fig.11 The number k (k=0..4) in (a) is an index indicating the refresh unit area.

[0100] Fig.11 (b) is a diagram showing an example of a new refresh area FRA (the shaded area in the figure) added to the restricted area shifting from left to right. At time t=0, the area with k=0 at the left end is FRA, and at time t=1, the area with k=1 is FRA. As t increases, FRA shifts to the right, and at time t=4, the area with k=4 (=numIR-1) at the right end of the picture is FRA. From t=5 to t=N-1 is a normal picture that does not include FRA. After t=N, the same picture structure as from t=0 to t=N-1 is repeated. This N is called the SDR period SDRperiod. At time t, the picture with (t%SDRperiod)=0 is a random access point, and random access can be performed with this as the starting point. (Random access processing using periodic refresh)

[0101] The motion picture decoding device decodes the restricted area (FRA) predicted intra-frame in the SDR picture, and generates an appropriate image by error concealment in the area outside the restricted area. In the picture following the SDR picture, the restricted area predicted intra-frame and the restricted area of ​​the restricted reference area of ​​the reference picture are also decoded, and an appropriate image is generated by error concealment in the other areas. The entire picture can be correctly decoded by repeating this action until the new refresh area (such as FRA) moves from one end of the picture to the other end and the restricted area becomes the entire picture (for numIR pictures). (Shape and shift direction of refresh area)

[0102] Fig.12 It is a diagram showing the FRA area. Fig.12 (a) represents the case of shifting from left to right, in which case FRA is a rectangle with width wIR and height hPict with (xIR, yIR) as the upper left coordinate. The width wIR of FRA is an integer multiple of the width minCU of the minimum CU. wIR=minCU*a

[0103] Here, a is a positive integer.

[0104] also, Fig.12 (b) shows the case of shifting from top to bottom. In this case, FRA is a rectangle with a height of hIR and a width of wPict, with (xIR, yIR) being the upper left coordinate. Moreover, the height hIR of FRA is an integer multiple of the width minCU of the minimum CU. hIR=minCU*a

[0105] Here, a is a positive integer. That is, the granularity of the width or height of FRA is greater than or equal to the minimum value minCU of the width or height of CU and is set to an integer multiple of minCU.

[0106] Fig.13 is a diagram illustrating the refresh direction. Fig.13 In (a), FRA is inserted vertically and shifted from left to right. In this case, the upper left coordinate (xIR[t], yIR[t]) and lower right coordinate (xRA_en[t], yRA_en[t]) of the new refresh area of ​​the picture at time t can be expressed as follows. (xIR[t], yIR[t])=(wIR*t-1, hPict-1) (xRA_en[t], yRA_en[t])=(wIR*(t+1)-1, hPict-1)

[0107] Furthermore, the upper left coordinate (xRA_st[t], yRA_st[t]) and lower right coordinate (xRA_en[t], yRA_en[t]) of the restricted area of ​​the image at time t can be expressed by (Formula IR-1). (xRA_st[t], yRA_st[t])=(0, 0) (calculation IR-1) (xRA_en[t], yRA_en[t])=(wIR*(t+1)-1, hPict-1)

[0108] Here, wIR is the width of FRA and hPict is the height of the picture.

[0109] In addition, (Formula IR-1) can be expressed as follows using the width wOVLP associated with the pixel derived by referring to the pixel value outside the restricted area and the encoding parameter. (xRA_st[t], yRA_st[t])=(0,0) (xRA_en[t], yRA_en[t])=(wIR*(t+1)-1-wOVLP, hPict-1)

[0110] Fig.13 (b) is an example of inserting FRA in the vertical direction and shifting from right to left (seq_refresh_direction=1).

[0111] Fig.13(c) is an example of inserting FRA in the horizontal direction and shifting from top to bottom (seq_refresh_direction = 0). In this case, the upper left coordinate of the restricted area of ​​the picture at time t can be expressed by (Formula IR-2). (xRA_st[t], yRA_st[t])=(0,0) (Formula IR-2) (xRA_en[t], yRA_en[t])=(wPict-1, hIR*(t+1)-1)

[0112] Here, hIR is the height of FRA and wPict is the width of the picture.

[0113] In addition, (Formula IR-2) can be expressed as follows using the height wOVLP associated with the pixel derived by referring to the pixel value outside the restricted area and the encoding parameter. (xRA_st[t], yRA_st[t])=(0,0) (xRA_en[t], yRA_en[t])=(wPict-1, hIR*(t+1)-1-hOVLP)

[0114] Fig.13 (d) is an example of inserting FRA in the horizontal direction and shifting it from bottom to top.

[0115] Fig.14 is an example of a FRA composed of multiple regions. As mentioned above, region k is a rectangle on a certain image, but Fig.14 As shown in (a), region k can also be a plurality of rectangles. In this case, Fig.14 As shown in (b), the restricted area is composed of multiple areas of the image. (Configuration of Moving Image Decoding Device)

[0116] The video decoding device 31 ( Fig.17 ) is described below.

[0117] The moving image decoding device 31 is configured to include an entropy decoding unit 301, a parameter decoding unit (prediction image decoding device) 302, a loop filter 305, a reference picture memory 306, a prediction parameter memory 307, a prediction image generation unit (prediction image generation device) 308, an inverse quantization / inverse transformation unit 311, and an addition unit 312. It should be noted that, according to the moving image encoding device 11 described later, there is also a configuration in which the loop filter 305 is not included in the moving image decoding device 31.

[0118] The parameter decoding unit 302 includes a restricted area control unit 320, and the restricted area control unit 320 includes a header decoding unit 3020, a CT information decoding unit 3021, and a CU decoding unit 3022 (prediction mode decoding unit) which are not shown in the figure. The CU decoding unit 3022 also includes a TU decoding unit 3024. The header decoding unit 3020 decodes parameter set information such as VPS, SPS, and PPS from the encoded data. The header decoding unit 3020 decodes a slice header (slice information) from the encoded data. The CT information decoding unit 3021 decodes CT from the encoded data. The CU decoding unit 3022 decodes CU from the encoded data. The TU decoding unit 3024 decodes QP update information (quantization correction value) and quantization prediction error (residual_coding) from the encoded data when the prediction error is included in the TU.

[0119] The parameter decoding unit 302 includes an inter prediction parameter decoding unit 303 and an intra prediction parameter decoding unit 304 (not shown). The predicted image generation unit 308 includes an inter prediction image generation unit 309 and an intra prediction image generation unit 310 .

[0120] In addition, the following describes an example of using CTU and CU as processing units, but the present invention is not limited thereto, and processing may be performed in sub-CU units. Alternatively, CTU, CU, and TU may be replaced with blocks, and sub-CU may be replaced with sub-blocks, and processing may be performed in blocks or sub-blocks.

[0121] The entropy decoding unit 301 performs entropy decoding on the coded stream Te input from the outside, separates each code (syntax element) and decodes it. There are the following methods in entropy coding: a method of variable-length coding the syntax elements using a context (probability model) appropriately selected according to the type of syntax element and the surrounding conditions; and a method of variable-length coding the syntax elements using a predetermined table or calculation formula. The former is represented by CABAC (Context Adaptive Binary Arithmetic Coding). The separated code contains prediction information for generating a predicted image and a prediction error for generating a differential image.

[0122] The entropy decoding unit 301 outputs a part of the separated code to the parameter decoding unit 302. The part of the separated code is, for example, a prediction mode predMode, a merge flag merge_flag, a merge index merge_idx, an inter-frame prediction identifier interpred_idc, a reference picture index refIdxLX, a prediction vector index mvp_LX_idx, and a difference vector mvdLX. Control of which code is decoded is performed based on the instruction of the parameter decoding unit 302. The entropy decoding unit 301 outputs the quantized transform coefficient to the inverse quantization / inverse transform unit 311. (Parameter decoding unit 302)

[0123] Fig.15 This is a diagram showing an example of syntax for performing notification to implement phased refresh. Fig.15 (a) indicates the syntax (phased refresh information) notified in the sequence parameter set (SPS). seq_refresh_enable_flag is a flag indicating whether phased refresh is used in subsequent pictures. The parameter decoding unit 302 decodes the phased refresh information. When the seq_refresh_enable_flag flag is 1, the motion picture decoding device uses phased refresh to decode the motion picture. When the seq_refresh_enable_flag flag is 0, the phased refresh is not used. When the seq_refresh_enable_flag is 1, the parameter decoding unit 302 decodes seq_refresh_mode, seq_refresh_direction, and seq_refresh_period. seq_refresh_mode is a flag indicating whether phased refresh is used in subsequent pictures. Fig.13 Flag indicating the type of phased refresh shown.

[0124] For a moving picture decoding device, if seq_refresh_mode is 0, then Fig.13 As shown in (a) and (b), the new refresh area (FRA) is inserted into the picture vertically. If seq_refresh_mode is 1, Fig.13 As shown in (c) and (d), FRA is inserted into the image horizontally.

[0125] seq_refresh_direction indicates Fig.13 The flag of the shift direction of the phased refresh as shown. If seq_refresh_direction is 0, then Fig.13As shown in (a) and (c) of FIG. 1 , the moving picture decoding apparatus shifts the FRA from the left to the right or from the top to the bottom of the picture. If seq_refresh_direction is 1, then Fig.13 As shown in (b) and (d), the FRA is shifted from the right to the left or from the bottom to the top of the picture. seq_refresh_period is the SDR period SDRperiod, which indicates the number of pictures between random access points.

[0126] Fig.15 (b) represents the syntax of the phased refresh information notified in the picture parameter set (PPS). When seq_refresh_enable_flag is 1, the parameter decoding unit 302 decodes seq_refresh_position, seq_refresh_size, and seq_refresh_overlap. seq_refresh_position is a syntax for specifying the upper left coordinates (xIR, yIR) of the new refresh area added to the restricted area. When seq_refresh_mode is 0 (vertical rectangular restricted area), the parameter decoding unit 302 sets seq_refresh_position to the x coordinate xIR and sets the y coordinate yIR to 0. When seq_refresh_mode is 1 (horizontal rectangular restricted area), the x coordinate xIR is set to 0 and the y coordinate yIR is set to seq_refresh_position. seq_refresh_size is the size of the FRA. When seq_refresh_mode is 0, it is set to the width wIR of the FRA. When seq_refresh_mode is 1, it is set to the height hIR of the FRA. seq_refresh_overlap is the size of the area where the position of the FRA of the previous picture overlaps the position of the FRA of the target picture in decoding order. When seq_refresh_mode is 0, it specifies the width wOVLP of the overlap. When seq_refresh_mode is 1, it specifies the height hOVLP of the overlap. It should be noted that seq_refresh_overlap can also be 0. (Overlapping new refresh area)

[0127] like Fig.16 As shown in FIG. 1 , the new refresh area (FRA) added to the restricted area in the target picture at time t can overlap (repeat) the FRA of the picture at time t-1. Fig.16In the example, the width wOVLP is an overlapping area. In this case, the end point of the restricted reference area when the picture at time t is referenced as a reference picture can be derived by subtracting the overlapping area as described below. Hereinafter, the restricted area obtained by subtracting the overlapping area may also be referred to as a filtered restricted area (or a filtered restricted area, an overlapped restricted area, or a reference restricted area). The derivation formula when seq_refresh_mode is 1 (horizontal rectangular restricted area) is as follows. (xRA_en[t], yRA_en[t])=(wPict, (hIR-hOVLP)*(t+1)-1) The derivation formula when seq_refresh_mode is 0 (vertical rectangular restricted area) is as follows. (xRA_en[t], yRA_en[t])=((wIR-wOVLP)*(t+1)-1,hPict)

[0128] When the new refresh area (eg, intra refresh area) is shifted from left to right, the derivation formula when the filter restriction area is 0 (vertical rectangular restriction area) in the case of seq_refresh_mode=0 and seq_refresh_direction=0 is as follows. (xRA_st[t], yRA_st[t])=(0,0) (xRA_en[t], yRA_en[t])=((wIR-wOVLP)*(t+1)-1, hPict-1)

[0129] When the new refresh area is shifted from right to left, the derivation formula of the filter restriction area in the case of (seq_refresh_mode=0 and seq_refresh_direction=1) is as follows. (xRA_st[t], yRA_st[t])=(wPict-(wIR-wOVLP)*(t+1)-1,0) (xRA_en[t], yRA_en[t])=(wPict-1, hPict-1)

[0130] When the new refresh area (eg, intra refresh area) is shifted from top to bottom, the derivation formula of the filter restriction area (seq_refresh_mode=1 and seq_refresh_direction=0) is as follows. (xRA_st[t], yRA_st[t])=(0,0) (xRA_en[t], yRA_en[t])=(wPict-1, (hIR-hOVLP)*(t+1)-1)

[0131] When the new refresh area is shifted from bottom to top, the derivation formula of the filter restriction area in the case of (seq_refresh_mode=1 and seq_refresh_direction=1) is as follows. (xRA_st[t], yRA_st[t])=(wPict-1, hPict-(hIR-hOVLP)*(t+1)-1) (xRA_en[t], yRA_en[t])=(wPict-1, hPict-1)

[0132] Thus, the video decoding device can obtain information about the stepwise refresh by decoding the syntax related to the stepwise refresh in the SPS or PPS. Therefore, the video decoding device can appropriately decode the coded data using the stepwise refresh when decoding the video at random access.

[0133] The loop filter 305 is a filter provided in the encoding loop, and is a filter for improving image quality by removing block distortion and ringing distortion. The loop filter 305 applies filters such as a deblocking filter 3051, a sample adaptive offset (SAO), and an adaptive loop filter (ALF) to the decoded image of the CU generated by the adder 312. (Deblocking filter 3051)

[0134] The deblocking filter 3051 determines that block distortion exists when the difference in pixel values ​​of pixels adjacent to each other via a block (CTU / CU / TU) boundary is within a preset range. Then, the image near the block boundary is smoothed by performing deblocking processing on the block boundary in the decoded image before deblocking filtering.

[0135] First, the deblocking filter 3051 performs a filter on / off determination in units of blocks. For example, when (Formula DB-1) is satisfied, filtering is performed on the target block. abs(P20-2*P10+P00)+abs(P23-2*P13+P03)+abs(Q20-2*Q10+QO0)+abs(Q23-2*Q13+Q03)<β(Formula DB-1)

[0136] Here, if Fig.33 (a) or Fig.33As shown in (b), P2k, P1k, P0k, Q0k, Q1k, Q2k (k=0, 3) are columns or rows whose distances from the block boundary are 2, 1, 0, 0, 1, and 2 pixels respectively. Fig.33 (a) is an example where block P and block Q are connected in the vertical direction. Fig.33 (b) is an example where block P and block Q are connected in the horizontal direction. P2k, P1k, and P0k are pixels included in block P, and Q0k, Q1k, and Q2k are pixels included in block Q. k represents the number of the pixel in the block boundary direction. β is a predetermined threshold.

[0137] When (Formula DB-1) is satisfied, the deblocking filter 3051 derives the strength of the filter by (Formula DB-2) with respect to the boundary between the block P and the block Q. When all of (Formula DB-2) are satisfied, the deblocking filter 3051 sets the filter strength to "strong", and when (Formula DB-2) is not satisfied, the filter strength is set to "weak". 2*(abs(P20-2*P10+P00)+abs(Q20-2*Q10+Q00))<(β>>2)(Formula DB-2) 2*(abs(P23-2*P13+P03)+abs(Q23-2*Q13+Q03))<(β>>2) abs(P30-P00)+abs(Q00-Q30))<(β>>3) abs(P33-P03)+abs(Q03-Q33))<(γ) abs(P00-P00)+abs(Q00-Q30))<(γ)

[0138] Here, γ is a threshold value depending on the quantization parameter.

[0139] Furthermore, when the filter strength is "strong", filtering processing is performed according to (Formula DB-3). P2k'=Clip3(P2k-δ, P2k+δ, (2*P3k+3*P2k+P1k+P0k+Q0k+4)>>3) (Formula DB-3) P1k'=CliP3(P1k-δ, P1k+δ, (P2k+P1k+P0k+Q0k+2)>>2) P0k'=CliP3(P0k-δ, P0k+δ, (P2k+2*P1k+2*P0k+2*Q0k+Q1k+4)>>3) Q0k'=CliP3(Q0k-δ, Q0k+δ, (P1k+2*P0k+2*Q0k+2*Q1k+Q2k+4)>>3) Q1k'=CliP3(Q1k-δ, Q1k+δ, (P0k+Q0k+Q1k+Q2k+2)>>2) Q2k'=CliP3(Q2k-δ, Q2k+δ, (P0k+Q0k+Q1k+3*Q2k+2*Q3k+4)>>3)

[0140] Here, P3k, P2k, P1k, P0k, Q0k, Q1k, Q2k, and Q3k are columns or rows whose distances from the block boundary are 3, 2, 1, 0, 0, 1, 2, and 3 pixels, respectively. δ is a variable used for suppression filtering.

[0141] In the above example, the deblocking filter 3051 refers to Fig.33 Every three pixels starting from the boundary shown by the horizontal line in (c) of FIG. 1 are used to determine the on / off of filtering and the strength of filtering in two blocks where the boundary is connected. In addition, referring to Fig.33 Each pair of four pixels starting from the boundary indicated by the horizontal line in (d) Fig.33 The values ​​of every three pixels starting from the boundary indicated by the vertical line in (e) are corrected to implement deblocking filtering. That is, the pixels of adjacent different blocks are required for the filtering process.

[0142] However, if Fig.34 As shown in (a), when block P is included in the restricted area and block Q is not included in the restricted area, the area of ​​block P after filtering with reference to block Q refers to the pixel value that is not determined during random access (non-restricted area), and therefore cannot be decoded normally.

[0143] In the present application, a method is provided for generating randomly accessible encoded data for normal decoding by the following method. (Deblocking filter restrictions in restricted areas)

[0144] Fig.35 This is a diagram showing the relationship between the restriction area and the deblocking filter.

[0145] When seq_refresh_enable_flag is 1, the deblocking filter 3051 of this embodiment turns off filtering in blocks adjacent to each other across the boundary between the restricted area and the non-restricted area.

[0146] The deblocking filter 3051 determines whether a block P and a block Q separated by a block boundary (CU boundary, PU boundary, TU boundary) are separated by a restricted area boundary.

[0147] For example, when the upper left coordinate of the restricted area is set to (xRA_st, yRA_st), the width of the restricted area is set to wRA, and the width of the CU of block P is set to wCU, the x coordinates xCUP and xCUQ of blocks P and Q adjacent to each other across the boundary of the restricted area can be expressed by (Formula DB-4). xCUP=xRA_st+wRA-wCU(Formula DB-4) xCUQ=xRA_st+wRA

[0148] The deblocking filter 3051 may use the following coordinates (xP, yP) of the pixels of block P and the coordinates (xQ, yQ) of the pixels of block Q, which may be the following determination formula. (IsRA(xP, yP)&&!IsRA(xQ, yQ))‖‖(!IsRA(xP, yP)&&IsRA(xQ, yQ))

[0149] Furthermore, when the shift method of the new refresh area is fixed, for example, when seq_refresh_direction=0, the following determination formula can be used. (IsRA(xP, yP)&&! IsRA(xQ, yQ))

[0150] Furthermore, for example, when seq_refresh_direction=1, the following determination formula can be used. (!IsRA(xP, yP)&&IsRA(xQ, yQ))

[0151] For example, when the upper left coordinate of the newly refreshed area is set to (xIR, yIR), the width of the newly refreshed area is set to wIR, and the width of the CU of block P is set to wCU, the x coordinates xCUP and xCUQ of blocks P and Q adjacent to each other across the boundary of the restricted area can be expressed by (Formula DB-4). xCUP=xIR+wIR-wCU(Formula DB-4) xCUQ=xIR+wIR

[0152] In this case, the deblocking filter 3051 may turn off filtering for blocks P and Q whose upper left x-coordinates satisfy (Formula DB-4).

[0153] Fig.36 (a) is a flowchart showing the flow of filtering processing by the deblocking filter 3051 according to this embodiment.

[0154] S2202: The deblocking filter 3051 checks whether the target blocks P and Q are adjacent to each other across the restriction region boundary based on the upper left coordinates of these blocks.

[0155] Specifically, when (IsRA(xP, yP)&&!IsRA(xQ, yQ))||(!IsRA(xP, yP)&&IsRA(xQ, yQ)) is true, it can be determined that they are adjacent across the restricted area boundary. In addition, the case where (IsRA(xP, yP)&&IsRA(xQ, yQ))||(!IsRA(xP, yP)&&!IsRA(xQ, yQ)) is true can also be determined as not adjacent across the restricted area boundary. In addition, this determination can also be performed on the filtered restricted area, which is the restricted area obtained by subtracting the overlapping area.

[0156] S2204: When block P and block Q are adjacent to each other across the restriction region boundary (Y in S2202), the deblocking filter 3051 sets filtering to off in both blocks.

[0157] S2206: When block P and block Q are not adjacent across the restricted area boundary (N in S2202), the deblocking filter 3051 sets the filter to on in both blocks. That is, when both block P and block Q are in the restricted area (IsRA(xP, yP) && IsRA(xQ, yQ)) or when both block P and block Q are in the unrestricted area (!IsRA(xP, yP) &&!IsRA(xQ, yQ)), the filter is turned on. Alternatively, when both block P and block Q are in the restricted area IsRA(xP, yP) && IsRA(xQ, yQ), or when the restricted area is off (seq_refresh_enable_flag = 0), the filter is turned on.

[0158] These processes are performed before the determination of (Formula DB-1). When the filter is set to be turned off in S2204, the deblocking filter 3051 does not perform the process after (Formula DB-1) and proceeds to the process of the next block. When the filter is set to be turned on in S2206, the deblocking filter 3051 performs the normal process after (Formula DB-1). At this time, if (Formula DB-1) is not satisfied, the deblocking filter 3051 turns off the filter.

[0159] In this way, the deblocking filter 3051 turns off the deblocking filter at the boundary of the restricted area, thereby, the pixel value of the area of ​​the restricted area included in the restricted area can be derived without using the pixel value of the non-restricted area. Therefore, the deblocking filter is turned off only for the boundary of the restricted area, thereby achieving normal decoding in random access and achieving low delay. (Another configuration example)

[0160] As another example, the deblocking filter 3051 turns off filtering only for the block P in the restricted area among the blocks adjacent to the boundary between the restricted area and the non-restricted area. That is, the filter is turned off when the upper left x coordinate of the block P satisfies xCUP of (Formula DF-4).

[0161] Fig.36 (b) is a flowchart showing the flow of filtering processing in this embodiment.

[0162] S2202 and S2206 are Fig.36 The same processing as (a) is performed and its description is omitted.

[0163] S2205: When block P and block Q are adjacent to each other with the restriction region boundary interposed therebetween (Y in S2202), the deblocking filter 3051 sets filtering in block P to off.

[0164] In this case, the deblocking filter 3051 makes an on / off decision based on (Formula DB-1) in block Q. When on, (Formula DB-1) is used to derive the strength of the filter, and (Formula DB-3) is used to filter only the pixels Q0k, Q1k, and Q2k of block Q.

[0165] On the other hand, in the newly refreshed area such as Fig.13 As shown in (c), in the case of vertical shift (seq_refresh_mode = 1 and seq_refresh_direction = 0), when the upper left coordinate of the restricted area is set to (xIR, yIR), the height of the restricted area is set to hIR, and the height of the CU is set to hCU, the y coordinates yCUP and yCUQ of the blocks P and Q adjacent to each other across the boundary of the restricted area can be expressed by (Formula DB-5). yCUP=yIR+hIR-hCU (calculation DB-5) yCUQ=yIR+hIR

[0166] In this way, the deblocking filter 3051 turns off the deblocking filter only for the blocks in the restricted area required for realizing the random access function at the boundary of the restricted area, and turns on the deblocking filter for the blocks not required for the random access function. As a result, the pixel values ​​of the area referred to as the restricted area in the subsequent encoding and decoding processing can be derived without using the pixel values ​​of the non-restricted area. Therefore, by turning off the deblocking filter only for the boundary of the restricted area, normal decoding can be realized in random access, and low delay can be realized.

[0167] With the above configuration, the deblocking filter 3051 forcibly turns off filtering in the filtering process between the restricted area (refresh area, first area) and the non-restricted area (unrefreshed area, second area).

[0168] With the above configuration, in the filtering process between the restricted area and the non-restricted area, only the filtering process on the restricted area side is forcibly turned off. (Overlap of refresh areas)

[0169] In this embodiment, a method is described in which the deblocking filter 3051 does not change the deblocking filter processing, but the block P is again encoded and decoded as a restricted area in the next encoded picture.

[0170] As described above, in a normal deblocking filter process, the deblocking filter 3051 performs filtering on / off derivation, filtering intensity derivation, and filtering, and the block P refers to the information of the block Q. For example, Fig.35 As shown, the blocks on both sides of the boundary of the restricted area refer to each other in the filtering process and correct the pixel values. Therefore, if the information of the block Q cannot be obtained, the decoding process of the block P cannot be performed normally.

[0171] In the deblocking filter 3051, when the pixels in the restricted area (block P side) are filtered using the pixels in the non-restricted area (block Q side), the pixels in the restricted area on the block P side are not restricted. Therefore, in subsequent pictures, they need to be removed from the restricted area. Fig.16 As shown, when a subsequent picture refers to a picture at t=k, the column of blocks P (the area represented by the wOVLP width) in the restricted area (e.g., t=k) of the target picture is not used as the restricted reference area. It should be noted that in the next picture to be encoded in time, the area removed from the restricted reference area may be encoded again as the restricted area.

[0172] That is, the video encoding device sets the width wOVL of the block P to overlap the restricted area at each time, sets the block P after the second encoding as the restricted area, and refers to it in the subsequent encoding process. Then, at time t=k+1, the deblocking filter 3051 of the video decoding device refers to the block in the restricted area adjacent to the left and the block in the same restricted area adjacent to the right, and performs filtering on the block P decoded as the restricted area for the second time. Therefore, the block P can be encoded and decoded using only the encoded data after the random access point.

[0173] Deblocking filtering is processed in units of blocks, so the overlapped width wOVLP can be the width wCU of the CU, but the pixels corrected by filtering are part of the block. Therefore, as shown in (Formula DB-6), the width wOVLP of the overlapped area can be derived from the number of pixels corrected by filtering numDFPEL and the minimum value minC of the width of the CU. wOVLP=ceil(numDFPEL / minCU)*minCU(Formula DB-6)

[0174] For example, when numDFPEL=3 and minCU is 4, wOVLP=4 is set. It should be noted that wIR>wOVLP. At time t=k, the upper left coordinate of the restricted area is set to (xRA_st[k], yRA_st[k]), and the width of the restricted area is set to wRA. The upper left coordinate of the restricted reference area at time t=k+1 (xRA_st[k+1], yRA_st[k+1]) can be derived by (Formula DB-7). xRA_st[k+1]=xRA_st[k]+wRA-wOVLP (calculation DB-7) yRA_st[k+1]=yRA_st[k]

[0175] On the other hand, in a new refresh area (eg, an intra refresh area) such as Fig.13 As shown in (c), in the case of vertical shift, the height hOVLP of the overlapping area and the upper left coordinates (xRA_st[k+1], yRA_st[k+1]) of the restricted reference area at time t=k+1 can be expressed by (Formula DB-8). hOVLP=ceil(numDFPEL / minCU)*minCU(calculation DB-8) xRA_st[k+1]=xRA_st[k] yRA_st[k+1]=yRA_st[k]+hRA-hOVLP

[0176] It should be noted that hRA>hOVLP.

[0177] However, when the deblocking filter of block P is turned off as described in (Deblocking Filter Off), wOVLP and hOVLP are set to 0 and no overlapping is performed.

[0178] The actions of the above-mentioned structure can be summarized as follows.

[0179] The phased refresh information is decoded to determine whether the target block is in the restricted area. When the phased refresh is turned on (seq_refresh_enable_flag = 1) and the target block is in the restricted area (IsRA (xPb, yPb) is true), as described above, only the prediction parameters of the restricted area are used to derive the prediction parameters for intra-frame prediction and inter-frame prediction. In addition, only the pixels in the restricted area are used for intra-frame prediction, and only the pixels in the restricted reference area of ​​the reference image are used for inter-frame prediction.

[0180] When the phased refresh is turned on (seq_refresh_enable_flag = 1) in the loop filter and the target block is in the restricted area (IsRA (xPb, yPb) is true), the deblocking filter is also applied to the restricted area. As a result, the area corresponding to the restricted area boundary (overlapping area) in the restricted area is removed from the basic operation of the restricted area, and thus the area is removed from the reference restricted area used for the next reference. It should be noted that in the loop filter, the range of the area corresponding to the overlapping area in the restricted area can also be used as a restricted reference area or filtered.

[0181] In this way, the motion picture encoding device encodes a part or all of the blocks connected to the boundary of the restricted area at time t=k and time t=k+1 as the restricted area. Then, the motion picture decoding device uses the pixel value of the block decoded at t=k+1 as the restricted area and refers to it in subsequent pictures. Therefore, normal decoding can be achieved in random access and low latency can be achieved.

[0182] Through the above-mentioned structure, for the non-restricted area (unrefreshed area) side or the restricted area side of the restricted area (refreshed area), the CU of the minimum CU size or the CU size including the number of pixels required for filtering (or the size specified by syntax) is encoded by the same method as the restricted area. (Variant 1)

[0183] In recent years, from the perspective of improving image quality, deblocking filters with a large number of taps are used for larger blocks. For example, in the case of blocks larger than 32×32, the 7 pixels starting from the block boundary are corrected by the filter. That is, numDFPEL=7. However, the corrected pixels must be encoded again by intra-frame prediction (as a restricted area) in the next picture. The more the overlapping area increases, the lower the coding efficiency. In this variant, the overlapping restricted area is reduced by shortening the deblocking filter of the block connected to the restricted area boundary, thereby suppressing the reduction in coding efficiency.

[0184] For example, when the minimum value of CU minCU=4 and the number of pixels corrected by deblocking filtering numDFPEL_L=7 (called long filtering), the overlap width wOVLP is 8 by (Formula DF-6). In contrast, when the number of pixels corrected by deblocking filtering is shortened to numDFPEL_S=3 (called short filtering), the overlap width wOVLP is 4.

[0185] The deblocking filter 3051 of the modification example 1 switches the type of filtering (long filtering, short filtering) according to whether the block is adjacent to the boundary of the restricted area. Whether the block is adjacent to the boundary of the restricted area is determined by (Formula DF-9). The upper left coordinate of the restricted area is set to (xRA_st[k], yRA_st[k]), the width of the restricted area is set to wRA, the upper left coordinate of the block is set to (xPb, yPb), and the width of the block is set to wPb. if (xPb==xRA_st[k]+wRA-wPb) (Formula DF-9)

[0186] When (Formula DF-9) is true, the block is adjacent to the boundary of the restricted area, and therefore the deblocking filter 3051 uses a short filter, otherwise, a long filter is used.

[0187] On the other hand, in the newly refreshed area such as Fig.13 In the case of vertical shift as shown in (c), whether or not the block is in contact with the boundary of the restricted area is determined by (Formula DF-10). if(yPb==yRA_st[k]+hRA-hPb) (calculation DF-10)

[0188] It should be noted that the number of pixels corrected by filtering in block Q has no relationship with the overlap width, so the process of switching the deblocking filter in block Q does not need to be performed.

[0189] In this way, in the block adjacent to the boundary of the restricted area, the short filter is applied to the deblocking filter process, thereby reducing the overlap width and suppressing the reduction in coding efficiency. In this case, it is ideal that the number of taps of the short filter is less than the minimum size of the CU. (Configuration of Inter-frame Prediction Parameter Decoding Unit)

[0190] The inter prediction parameter decoding unit 303 decodes the inter prediction parameter based on the code input from the entropy decoding unit 301, referring to the prediction parameter stored in the prediction parameter memory 307. In addition, the inter prediction parameter decoding unit 303 outputs the decoded inter prediction parameter to the prediction image generation unit 308, and stores it in the prediction parameter memory 307.

[0191] Fig.191 is a schematic diagram showing the structure of the inter-frame prediction parameter decoding unit 303 of the present embodiment. The inter-frame prediction parameter decoding unit 303 is configured to include: a parameter decoding control unit 3031, an AMVP prediction parameter derivation unit 3032, an addition unit 3038, a merged prediction parameter derivation unit 3036, and a sub-block prediction parameter derivation unit 3037. The AMVP prediction parameter derivation unit 3032, the merged prediction parameter derivation unit 3036, and the sub-block prediction parameter derivation unit 3037 may also be collectively referred to as a motion vector derivation unit (motion vector derivation device).

[0192] The parameter decoding control unit 3031 instructs the entropy decoding unit 301 to decode the syntax elements associated with inter-frame prediction, and extracts the syntax elements included in the encoded data, such as the merge flag merge_flag, the merge index merge_idx, the inter-frame prediction identifier inter_pred_idc, the reference picture index refIdxLX, the prediction vector index mvp_LX_idx and the differential vector mvdLX.

[0193] The parameter decoding control unit 3031 first extracts the merge flag merge_flag. The parameter decoding control unit 3031 indicates that when a certain syntax element is extracted, it indicates that the entropy decoding unit 301 is instructed to decode the certain syntax element, and reads the corresponding syntax element from the coded data.

[0194] When the merge flag merge_flag is 0, that is, indicating the AMVP prediction mode, the parameter decoding control unit 3031 uses the entropy decoding unit 301 to extract the AMVP prediction parameters from the encoded data. As AMVP prediction parameters, there are, for example, inter-frame prediction identifier inter_pred_idc, reference picture index refIdxLX, prediction vector index mvp_LX_idx, and differential vector mvdLX. The AMVP prediction parameter derivation unit 3032 derives the prediction vector mvpLX based on the prediction vector index mvp_LX_idx. The parameter decoding control unit 3031 outputs the differential vector mvdLX to the addition unit 3038. In the addition unit 3038, the prediction vector mvpLX and the differential vector mvdLX are added to derive the motion vector.

[0195] When the merge flag merge_flag is 1, that is, indicating the merge prediction mode, the parameter decoding control unit 3031 extracts the merge index merge_idx as the prediction parameter of the merge prediction. The parameter decoding control unit 3031 outputs the extracted merge index merge_idx to the merge prediction parameter derivation unit 3036.

[0196] Fig. 20(a) is a schematic diagram showing the structure of the merged prediction parameter derivation unit 3036 of the present embodiment. The merged prediction parameter derivation unit 3036 includes: a merged candidate derivation unit 30361, a merged candidate selection unit 30362 and a merged candidate storage unit 30363. The merged candidate storage unit 30363 stores the merged candidates input from the merged candidate derivation unit 30361. It should be noted that the merged candidate is composed of: a prediction list utilization flag predFlagLX, a motion vector mvLX and a reference picture index refIdxLX. In the merged candidate storage unit 30363, indexes are assigned to the stored merged candidates according to prescribed rules.

[0197] The merge candidate derivation unit 30361 directly uses the decoded motion vector of the adjacent block and the reference picture index refIdxLX to derive the merge candidate. In addition, the merge candidate derivation unit 30361 can apply affine prediction in the spatial merge candidate derivation process, the temporal merge candidate derivation process, the combined merge candidate derivation process, and the zero merge candidate derivation process described later. (Spatial Merge Candidate Derivation Process)

[0198] As a spatial merge candidate derivation process, the merge candidate derivation unit 30361 reads the prediction parameters (prediction list utilization flag predFlagLX, motion vector mvLX, reference picture index refIdxLX) stored in the prediction parameter memory 307 according to a predetermined rule, and sets them as merge candidates. In the method of specifying the reference picture, for example, the prediction parameters of each adjacent block (for example, blocks adjacent to the left end L, the lower left end BL, the upper left end AL, the upper end A, and the upper right end AR of the target block) within a predetermined range from the target block are used. Each merge candidate is referred to as L, BL, AL, A, and AR.

[0199] When the object block is in the restricted area (IsRA(xPb, yPb) is true) and the reference position (xNbX, yNbX) of the adjacent block of the object block is in the non-restricted area (IsRA(xNbX, yNbX) is false), the merge candidate derivation unit 30361 does not use the value of the adjacent block in the derivation of the merge candidate. That is, when the object block is in the restricted area (IsRA(xPb, yPb) is true) and the reference position (xNbX, yNbX) of the adjacent block of the object block is in the restricted area (IsRA(xNbX, yNbX) is true) and it is an inter-frame prediction, the merge candidate derivation unit 30361 uses the position (xNbX, yNbX) for derivation of the merge candidate.

[0200] For example, in the new refresh area such as Fig.13In the case of shifting from left to right as shown in (a) (seq_refresh_mode=0 and seq_refresh_direction=0), the object block is in the restricted area (IsRA(xPb, yPb) is true), and the right adjacent position B0 (xPb+wPb, yPb-1) of the object block is in the restricted area (IsRA(xPb+wPb, yPb-1) is true), in the merge candidate derivation unit 30361, B0 is set to be available (availableB0=1), and the value of the right adjacent block is used in the derivation of the merge candidate.

[0201] For example, in the new refresh area such as Fig.13 In the case of shifting from right to left as shown in (b) (seq_refresh_mode = 0 and seq_refresh_direction = 1), the object block is in the restricted area (IsRA(xPb, yPb) is true) and the left adjacent positions A0 (xPb-1, yPb+hPb), A1 (xPb-1, yPb+hPb-1), B2 (xPb-1, yPb-1) of the object block are in the restricted area, the merge candidate derivation unit 30361 uses the values ​​of the left adjacent positions A0, A1, B2 in the derivation of the merge candidate. (Temporal Merge Candidate Derivation Process)

[0202] As a temporal merge derivation process, the merge candidate derivation unit 30361 reads the prediction parameters of the block C in the reference image including the coordinates of the lower right CBR of the target block or the center from the prediction parameter memory 307, sets it as a merge candidate, and stores it in the merge candidate list mergeCandList[]. Generally, the block CBR is added to the merge candidate list mergeCandList[] with priority. When the block CBR does not have a motion vector (for example, an intra-frame prediction block) or the block CBR is located outside the picture, the motion vector of the block C is added to the prediction vector candidate. In the method of specifying the reference image, for example, the reference picture index refIdxLX specified in the slice header can be used, and the smallest reference picture index among the reference picture indexes refIdxLX of the adjacent blocks can be used for specification.

[0203] For example, the merge candidate derivation unit 30361 can derive the position of block C (xColCtr, yColCtr) and the position of block CBR (xColCBr, yColCBr) by (Formula MRG-1). xColCtr=xPb+(wPb>>1)(Formula MRG-1) yColCtr=yPb+(hPb>>1) xColCBr=xPb+wPb yColCBr=yPb+hPb

[0204] Here, (xPb, yPb) is the upper left coordinate of the object block, and (wPb, hPb) is the width and height of the object block. If block CBR is available, the motion vector of block CBR is used to derive the merge candidate COL. If block CBR is not available, COL is derived using block C.

[0205] The merge candidates derived by the merge candidate derivation unit 30361 are stored in the merge candidate storage unit 30363. The order stored in the merge candidate list mergeCandList[] is, for example, spatial merge candidates, temporal merge candidates, that is, {L, A, AR, BL, A, COL, COMB0, ..}. It should be noted that unusable reference blocks (blocks are intra-frame predictions, etc.) are not stored in the merge candidate list.

[0206] The merge candidate selection unit 30362 selects a merge candidate mergeCandList[merge_idx] assigned with the merge index merge_idx input from the parameter decoding control unit 3031 from among the merge candidates included in the merge candidate list stored in the merge candidate storage unit 30363 as the inter prediction parameter of the target block. The merge candidate selection unit 30362 stores the selected merge candidate in the prediction parameter memory 307 and outputs it to the prediction image generation unit 308.

[0207] Fig. 20 (b) is a schematic diagram showing the structure of the AMVP prediction parameter derivation unit 3032 of this embodiment. The AMVP prediction parameter derivation unit 3032 includes a vector candidate derivation unit 3033, a vector candidate selection unit 3034, and a vector candidate storage unit 3035. The vector candidate derivation unit 3033 derives prediction vector candidates from the decoded motion vector mvLX of the adjacent block stored in the prediction parameter memory 307 according to a predetermined rule, and stores the prediction vector candidate list mvpListLX[] in the vector candidate storage unit 3035.

[0208] The vector candidate selection unit 3034 selects the motion vector mvpListLX[mvp_LX_idx] indicated by the predictor index mvp_LX_idx from among the predictor candidates in the predictor candidate list mvpListLX[] as the predictor mvpLX. The vector candidate selection unit 3034 outputs the selected predictor mvpLX to the addition unit 3038 .

[0209] It should be noted that the prediction vector candidate can be derived by scaling the motion vectors of decoded adjacent blocks within a preset range from the object block by the method described below. In addition to blocks that are spatially adjacent to the object block, such as the left block and the upper block, adjacent blocks also include areas that are temporally adjacent to the object block, for example, areas obtained based on prediction parameters of blocks that have the same position as the object block but are displayed at different times.

[0210] The adder 3038 calculates a motion vector mvLX by adding the prediction vector mvpLX input from the AMVP prediction parameter derivation unit 3032 and the difference vector mvdLX input from the parameter decoding control unit 3031. The adder 3038 outputs the calculated motion vector mvLX to the prediction image generation unit 308 and the prediction parameter memory 307. mvLX[0]=mvpLX[0]+mvdLX[0] mvLX[1]=mvpLX[1]+mvdLX[1] (Motion Vector Scaling)

[0211] The method for deriving the scaled motion vector is described below. When a motion vector Mv (reference motion vector), a picture PicMv containing a block having Mv, a reference picture PicMvRef of Mv, a scaled motion vector sMv (scaledMV), a picture CurPic containing a block having sMv, and a reference picture CurPicRef referenced by sMv are used, sMv is derived by the derivation function MvScale (Mv, PicMv, PicMvRef, CurPic, CurPicRef) shown in (Formula INTERP-1). distScaleFactor=Clip3(-R2, R2-1, (tb*tx+round2)>>shift2) tx=(16384+abs(td)>>1) / td td=DiffPicOrderCnt(PicMv,PicMvRef) tb=DiffPicOrderCnt(CurPic, CurPicRef)

[0212] Here, round1, round2, shift1, and shift2 are round values ​​and shift values ​​for division using reciprocals, for example, round1=1<<(shift1-1), round2=1<<(shift2-1), shift1=8, shift2=6, etc. DiffPicOrderCnt(Pic1, Pic2) is a function that returns the difference between the time information (e.g., POC) of Pic1 and Pic2. In order to process with limited precision, the value range of R1 and R2 is limited, for example, R1=32768, R2=4096, etc.

[0213] In addition, the scaling function MvScale(Mv, PicMv, PicMvRef, CurPic, CurPicRef) can be the following formula. MvScale(Mv, PicMv, PicMvRef, CurPic, CurPicRef)= Mv*DiffPicOrderCnt(CurPic, CurPicRef) / DiffPicOrderCnt(PicMv, PicMvRef)

[0214] That is, Mv may be scaled according to the ratio of the difference between the time information of CurPic and CurPicRef and the difference between the time information of PicMv and PicMvRef. (Configuration of Intra-frame Prediction Parameter Decoding Unit 304)

[0215] The intra prediction parameter decoding unit 304 decodes the intra prediction parameters, for example, the intra prediction mode IntraPredMode, by referring to the prediction parameters stored in the prediction parameter memory 307 based on the code input from the entropy decoding unit 301. The intra prediction parameter decoding unit 304 outputs the decoded intra prediction parameters to the prediction image generation unit 308, and then stores them in the prediction parameter memory 307. The intra prediction parameter decoding unit 304 can also derive intra prediction modes that are different in brightness and color difference.

[0216] Fig.21 3 is a schematic diagram showing the configuration of the intra-frame prediction parameter decoding unit 304 of the parameter decoding unit 302. Fig.21 As shown, the intra prediction parameter decoding unit 304 is configured to include: a parameter decoding control unit 3041, a luma intra prediction parameter decoding unit 3042, and a chroma intra prediction parameter decoding unit 3043.

[0217] The parameter decoding control unit 3041 instructs the entropy decoding unit 301 to decode the syntax elements, and receives the syntax elements from the entropy decoding unit 301. When prev_intra_luma_pred_flag is 1, the parameter decoding control unit 3041 outputs mpm_idx to the MPM parameter decoding unit 30422 in the luma intra prediction parameter decoding unit 3042. When prev_intra_luma_pred_flag is 0, the parameter decoding control unit 3041 outputs rem_selected_mode_flag, rem_selected_mode, and rem_non_selected_mode to the non-MPM parameter decoding unit 30423 of the luma intra prediction parameter decoding unit 3042. When prev_intra_luma_pred_flag is 0, the parameter decoding control unit 3041 outputs the syntax elements of the chroma intra prediction parameters to the chroma intra prediction parameter decoding unit 3043.

[0218] The luma intra prediction parameter decoding unit 3042 is configured to include an MPM candidate list derivation unit 30421 , an MPM parameter decoding unit 30422 , and a non-MPM parameter decoding unit 30423 (decoding unit, derivation unit).

[0219] The MPM parameter decoding unit 30422 refers to the MPM candidate list mpmCandList[] and mpm_idx derived by the MPM candidate list derivation unit 30421 to derive the luma prediction mode IntraPredModeY, and outputs it to the intra-frame prediction image generation unit 310 .

[0220] The non-MPM parameter decoding unit 30423 derives RemIntraPredMode based on the MPM candidate list mpmCandList[] and rem_selected_mode_flag, rem_selected_mode, and rem_non_selected_mode, and outputs the brightness prediction mode IntraPredModeY to the intra-frame prediction image generation unit 310.

[0221] The chroma intra prediction parameter decoding unit 3043 derives the chroma prediction mode IntraPredModeC according to the syntax elements of the chroma intra prediction parameters, and outputs it to the intra prediction image generating unit 310 .

[0222] When using refresh to achieve random access, some restrictions are added to the above parameters. At the boundary between the restricted reference area and the unrestricted reference area, in the reference pixel filtering and predicted image generation process described later, pixels in the unrestricted reference area are sometimes referred to, and it is necessary to restrict the reference pixel filter, prediction mode, predicted pixel generation process, etc.

[0223] The reference picture memory 306 stores the CU decoded image generated by the adding unit 312 at a predetermined position for each target picture and target CU.

[0224] The prediction parameter memory 307 stores the prediction parameter at a predetermined position for each CTU or CU to be decoded. Specifically, the prediction parameter memory 307 stores the parameters decoded by the parameter decoding unit 302 and the prediction mode predMode separated by the entropy decoding unit 301 .

[0225] The prediction image generation unit 308 receives the prediction mode predMode, prediction parameters, and the like as input. In addition, the prediction image generation unit 308 reads a reference picture from the reference picture memory 306. The prediction image generation unit 308 generates a prediction image of a block or a sub-block using the prediction parameters and the read reference picture (reference picture block) in the prediction mode (intra-frame prediction, inter-frame prediction) indicated by the prediction mode predMode. Here, the reference picture block refers to a set of pixels on the reference picture (usually rectangular, so it is called a block), which is a region referenced to generate a prediction image. (Inter-frame prediction image generation unit 309)

[0226] When the prediction mode predMode indicates the inter prediction mode, the inter prediction image generator 309 generates a prediction image of a block or a subblock by inter prediction using the inter prediction parameters input from the inter prediction parameter decoder 303 and a reference picture.

[0227] Fig. 22 1 is a schematic diagram showing the configuration of the inter-prediction image generation unit 309 included in the prediction image generation unit 308 of the present embodiment. The inter-prediction image generation unit 309 is configured to include a motion compensation unit (prediction image generation device) 3091 and a weighted prediction unit 3094 . (Motion Compensation)

[0228] The motion compensation unit 3091 (interpolation image generation unit) generates an interpolation image (motion compensation image) by reading a block located in the reference picture RefLX specified by the reference picture index refIdxLX and shifted by the motion vector mvLX from the position of the target block from the reference picture RefLX, based on the inter prediction parameters (prediction list utilization flag predFlagLX, reference picture index refIdxLX, motion vector mvLX) input from the inter prediction parameter decoding unit 303. Here, when the accuracy of the motion vector mvLX is not integer accuracy, filtering for generating pixels at decimal positions, called motion compensation filtering, is performed to generate a motion compensation image.

[0229] The motion compensation unit 3091 first derives the integer position (xInt, yInt) and phase (xFrac, yFrac) corresponding to the coordinates (x, y) in the prediction block through the following formula. xInt=xPb+(mvLX[0]>>(log2(MVBIT)))+x xFrac=mvLX[0]&(MVBIT-1) yInt=yPb+(mvLX[1]>>(log2(MVBIT)))+y yFrac=mvLX[1]&(MVBIT-1)

[0230] Here, (xPb, yPb) is the upper left coordinate of a block of size wPb*hPb, x=0…wPb-1, y=0…hPb-1, and MVBIT represents the accuracy of the motion vector mvLX (1 / MVBIT pixel accuracy).

[0231] The motion compensation unit 3091 uses an interpolation filter to perform horizontal interpolation processing on the reference image refimg to thereby derive a temporary image temp[][]. Hereinafter, ∑ is a sum related to k of k=0..NTAP-1, shift1 is a normalization parameter of the adjustment value interval, and offset1=1<<(shift1-1). temp[x][y]=(∑mcFilter[xFrac][k]*refImg[xInt+k-NTAP / 2+1][yInt]+offset1)>>shift1

[0232] Next, the motion compensation unit 3091 performs vertical interpolation processing on the temporary image temp[][] to derive an interpolated image Pred[][]. The following ∑ is a sum related to k of k=0..NTAP-1, shift2 is a normalization parameter of the interval of the adjustment value, and offset2=1<<(shift2-1). Pred[x][y]=(∑mcFilter[yFrac][k]*temp[x][y+k-NTAP / 2+1]+offset2)>>shift2

[0233] It should be noted that, in the case of bidirectional prediction, the above-mentioned Pred[][] (called interpolated images PredL0[][] and PredL1[][]) is derived according to each L0 list and L1 list, and the interpolated image Pred[][] is generated based on the interpolated image PredL0[][] and the interpolated image PredL1[][]. (Weighted Forecast)

[0234] The weighted prediction unit 3094 generates a prediction image of the block by multiplying the motion compensation image PredLX by the weighting coefficient. When one of the prediction list utilization flags (predFlagL0 or predFlagL1) is 1 (unidirectional prediction) and weighted prediction is not used, the following calculation is performed to match the motion compensation image PredLX (LX is L0 or L1) with the pixel bit number bitDepth. Pred[x][y]=Clip3(0, (1<<bitDepth)-1, (PredLX[x][y]+offset1)>>shift1)

[0235] Here, shiftl=14-bitDepth, offsetl=1<<(shiftl-1).

[0236] When both reference list utilization flags (predFlagL0 and predFlagL1) are 1 (bidirectional prediction BiPred) and weighted prediction is not used, the motion compensated images PredL0 and PredL1 are averaged and the average is matched to the number of pixel bits using the following calculation. Pred[x][y]=Clip3(0, (1<<bitDepth)-1, (PredL0[x][y]+PredL1[x][y]+offset2)>>shift2)

[0237] Here, shift2=15-bitDepth, offset2=1<<(shift2-1).

[0238] Furthermore, when unidirectional prediction and weighted prediction are performed, the weighted prediction unit 3094 derives a weighted prediction coefficient w0 and an offset value o0 from the encoded data, and performs the following calculation processing. Pred[x][y]=Clip3(0, (1<<bitDepth)-1, ((PredLX[x][y]*w0+2^(log2WD-1))>>log2WD)+o0)

[0239] Here, log2WD is a variable representing a predetermined shift amount.

[0240] Furthermore, when bidirectional prediction BiPred and weighted prediction are performed, the weighted prediction unit 3094 derives weighted prediction coefficients w0, w1, o0, and o1 from the encoded data, and performs the following calculation processing. Pred[x][y]=Clip3(0, (1<<bitDepth)-1, (PredL0[x][y]*w0+PredL1[x][y]*w1+((o0+o1+1)<<log2WD))>>(log2WD+1))

[0241] The inter-frame prediction image generation unit 309 outputs the generated prediction image of the block to the addition unit 312 . (Refresh and inter prediction)

[0242] In refreshing, in order to realize normal decoding processing after the random access point at the time of random access, it is necessary to generate a predicted image by referring to only the restricted area (restricted reference area) of the reference picture even in the restricted area inter prediction. (limit the border filling of the reference area)

[0243] In the motion compensation unit 3091 of this embodiment, as Fig.23 As shown in (a), when the target block Pb is in the restricted area and seq_refresh_enable_flag is 1, when the pixel of the reference picture is in the unrestricted reference area, a padding process is performed to replace the pixel value of the unrestricted reference area (second reference area) in the reference picture with the pixel value of the restricted reference area. For example, when the upper left coordinate of the target block is (xPb, yPb), and the width and height are bW and bH, the motion compensation unit 3091 derives whether the target block (Pb) is in the restricted area through IsRA(Pb) described above.

[0244] Boundary filling is achieved by using the pixel value refImg[xRef+i][yRef+j] at the following position xRef+i, yRef+j as the pixel value at the position (xIntL+i, yIntL+j) of the reference pixel in the interpolation image generation (motion compensation) performed by the motion compensation unit 3091. That is, it is achieved by clipping the reference position of the pixel used for interpolation image generation to the position of the boundary pixel (for example, the left end, top end, bottom end, and right end position) of the limited reference area of ​​the reference picture.

[0245] For example, when the left end position of the restricted reference region of the reference picture is xRA_st, the right end position is xRA_en, the upper end position is yRA_st, and the lower end position is yRA_en, the motion compensation unit 3091 performs the following boundary filling processing. That is, when the X coordinate xIntL+i of the reference position is smaller than the left end of the restricted reference region, it is replaced with the position xRA_st of the left end pixel of the restricted reference region. Similarly, when the X coordinate xIntL+i of the reference position is larger than the right end of the restricted reference region, it is replaced with the position xRA_en of the right end pixel of the restricted reference region. When the Y coordinate yIntL+jj is smaller than the upper end of the restricted reference region, it is replaced with the upper end pixel position yRA_st, and when the Y coordinate yIntL+j is larger than the lower end of the restricted reference region, it is replaced with the position yRA_en of the lower end pixel. xRef+i=Clip3(xRA_st, xRA_en, xIntL+i) (calculation INTERP-2) yRef+j=Clip3(yRA_st, yRA_en, yIntL+j)

[0246] For example, when the phased refresh information is turned on (seq_refresh_enable_flag = 1), and seq_refresh_mode = 0, seq_refresh_direction = 0, that is, when the restricted reference area is moved from left to right, the right end position (width) of the restricted reference area at time t = k is set to wRA[k], xRA_st=0, xRA_en=wRA[k]-1 yRA_st=0,yRA_en=hPict-1

[0247] Therefore, the motion compensation unit 3091 uses (equation INTERP-3) to derive the coordinates of the reference pixel. xRef+i=Clip3(0, wRA[k]-1, xIntL+i) (calculation INTERP-3) yRef+j=Clip3(0, hPict-1, yIntL+j)

[0248] Here, hPict is the height of the picture.

[0249] It should be noted that for xIntL and yIntL, when the upper left coordinate of the object block based on the upper left coordinate of the image is set to (xPb, yPb) and the motion vector is set to (mvLX[0], mvLX[1]), the coordinates of the reference pixel are derived by (Formula INTERP-3'). xIntL=xPb+(mvLX[0]>>log2(MVBIT))(calculation INTERP-3') yIntL=yPb+(mvLX[1]>>log2(MVBIT))

[0250] Here, MVBIT indicates that the accuracy of the motion vector is 1 / MVBIT pixel (pel).

[0251] This can be achieved by reading the pixel value of the coordinates (xRef+i, yRef+j) Fig.23 (a) Filling.

[0252] It should be noted that when seq_efresh_mode=0 and seq_refresh_direction=1 in the step refresh information, that is, when the limited reference area is moved from right to left, the left end coordinate xRA_st is derived by yRA_en=wRA[k], and the coordinates of the reference pixel are derived using the following formula. xRef+i=Clip3(wRA[k], wPict-1, xIntL+i) (calculation INTERP-2') yRef+j=Clip3(0, hPict-1, yIntL+j)

[0253] Here, xRA_st=wRA[k], xRA_en=wPict-1 yRA_st=0,yRA_en=hPict-1

[0254] On the other hand, when refreshing Fig.13 As shown in (c), in the case of vertical shift (in the stage refresh information, seq_refresh_mode = 1, seq_refresh_direction = 0), when the height of the restricted reference area is set to hRA, the lower end coordinate is derived by yRA_en = hRA-1, and the coordinate of the reference pixel is derived by using (formula INTERP-4) in the boundary filling. xRef+i=Clip3(0, wPict-1, xIntL+i) (calculation INTERP-4) yRef+j=Clip3(0,yRA_en,yIntL+j)

[0255] Here, wPict is the width of the picture.

[0256] In the staged refresh, the boundary pixels of the restricted reference area are used for filling. Therefore, even if the motion vector in the inter-frame prediction within the restricted area points to outside the restricted reference area of ​​the reference picture (non-restricted reference area), the pixel value within the restricted reference area is used to replace the reference pixel. Therefore, normal decoding processing after the random access point can be achieved in the inter-frame prediction.

[0257] Through the above process, in the inter-frame prediction of the restricted area, in the image generation process using the motion vector pointing outside the restricted reference area, the motion compensation unit 3091 replaces (fills) the pixels of the non-restricted reference area with the pixels of the restricted reference area. (Motion vector restriction to limit reference area)

[0258] When referring to a reference picture, there is a boundary motion vector restriction as another method of generating a predicted image by referring only to pixels in a limited reference area. In this embodiment, in the motion compensation performed by the motion compensation unit 3091 described later, the motion vector is restricted (cropped) so that the position (xIntL+i, yIntL+j) of the reference pixel falls within the limited reference area.

[0259] In this embodiment, when the upper left coordinates of the target block at time t=k are (xPb, yPb), the size of the block is (wPb, hPb), the position of the restricted reference area of ​​the reference picture at time j is (xRA_st[j], xRA_st[j]), and the size is (wRA[j], hRA[j]), the motion vector mvLX of the block is input, and it is determined whether the tip of the motion vector is within the restricted range. This determination formula is expressed as (Formula INTERP-5). IsRA(Pb, mvLX)=(xRA_st[j]<=xPb+(mvLX[0]>>log2(MVBIT))-NTAP2+I)&& (xPb+wPb-1+(mvLX[0]>>log2(MVBIT))+NTAP2<xRA_st[j]+wRA[j])(Formula INTERP-5)

[0260] Alternatively, the following formula may be used. IsRA(Pb, mvLX)=(yRA_st[j]<=yPb+(mvLX[1]>>log2(MVBIT))-NTAP2+1)&& (yPb+hPb-1+(mvLX[1]>>log2(MVBIT))+NTAP2<yRA_st[j]+hRA[j])

[0261] In addition, in this embodiment, when the upper left coordinates of the target block at time t=k are (xPb, yPb), the size of the block is (wPb, hPb), the position of the restricted reference area of ​​the reference picture at time j is set to (xRA_st[j], xRA_st[j]), and the size is set to (wRA[j], hRA[j]), the motion vector mvLX of the block is set to input, and the restricted motion vector mvLX is output. The restriction of the motion vector mvLX is expressed as (Formula INTERP-6). mvLX[0]=ClipRA(mvLX[0])=Clip3(xRA_st[j]+(NTAP2-1-xPb)<<log2(MVBIT), (xRA_st[j]+wRA[j]-NTAP2-xPb-wPb)<<log2(MVBIT), mvLX[0]) (Formula INTERP-6) mvLX[1]=ClipRA(mvLX[1])=Clip3(yRA_st[j]+(NTAP2-1-yPb)<<log2(MVBIT), (yRA_st[j]+hRA[j]-NTAP2+1-yPb-hPb)<<log2(MVBIT), mvLX[1])

[0262] The above clipping process is expressed as ClipRA(mvLX) below.

[0263] Alternatively, when the restricted area is shifted from left to right (in the phased refresh information, seq_refresh_mode=0, seq_refresh_direction=0) and the width of the restricted reference area is wRA, xRA_st=0, xRA_en=wRA-1, yRA_st=0, yRA_en=hPict-1, so it can be derived by the following formula. mvLX[0]=Clip3((NTAP2-1-xPb)<<log2(MVBIT), (wRA[k]-NTAP2-xPb-wPb)<<log2(MVBIT), mvLX[0]) (Formula INTERP-6) mvLX[1]=Clip3((NTAP2-1-yPb)<<log2(MVBIT), (hPict-NTAP2+1-yPb-hPb)<<log2(MVBIT), mvLX[1])

[0264] Here, NTAP is the number of taps of the filter used for interpolation image generation, and NTAP2 = NTAP / 2. In addition, the width wRA[k] of the limited reference area at time t = k can be derived by the formula (Formula INTERP-7) using the width wIR of the limited area and the width wOVLP of the overlap. wRA[k]=(wIR-wOVLP)*(k+1)(Formula INTERP-7)

[0265] On the other hand, in restricted areas such as Fig.13 As shown in (c), in the case of vertical shift (in the staged refresh information, seq_refresh_mode = 1, seq_refresh_direction = 0), the restriction of the motion vector mvLX and the height hRA[k] of the restricted reference area at time t = k can be derived by (Formula INTERP-8). mvLX[0]=Clip3((NTAP2-1-xPb)<<log2(MVBIT), (wPict-NTAP2+1-xPb-wPb+1)<<log2(MVBIT), mvLX[0]) (calculation INTERP-8) mvLX[1]=Clip3((NTAP2-1-yPb)<<log2(MVBIT), (hRA[k]-NTAP2+1-yPb-hPb+1)<<log2(MVBIT), mvLX[1]) hRA[k]=(hIR-hOVLP)*(k+1)

[0266] In refreshing, the motion vector is limited in this way, so that the motion vector points to the limited area of ​​the reference picture, so that normal decoding processing after the random access point can be achieved in inter-frame prediction in the limited area.

[0267] Through the above process, when the inter-frame prediction parameter decoding unit 303 and the motion compensation unit 3091 perform inter-frame prediction, the vector pointing to the non-restricted reference area is clipped to the restricted reference boundary. (Merge prediction restriction of restricted reference area)

[0268] The restricted area (restricted reference area) that can be referenced in the inter-frame prediction of the restricted area in the subsequent picture is as follows Fig.10 As shown by the horizontal line in (b), the range varies depending on the reference picture (time). Therefore, when estimating the motion vector of the target block based on the motion vectors of the surrounding blocks (hereinafter, adjacent blocks) of the target block as shown in the merge prediction, additional processing is required.

[0269] Fig.23(b) is a diagram showing an example of this embodiment. Fig.23 In (b), the object block is in the restricted area of ​​the picture at time t=4. Moreover, the reference picture index of the adjacent block AL used in the spatial merge candidate represents the picture at time t=1 (reference picture PicMvRef), and the motion vector is mvLX_AL. In the case where the area at time t=1 referenced by the motion vector mvLX_AL of the adjacent block AL is included in the restricted reference area, when the motion vector mvLX_AL is applied as a merge candidate to the motion vector of the object block, it is also possible that it points outside the restricted reference area. The merge prediction parameter derivation unit 3036 of this embodiment derives the merge candidate in this way so that the merge candidate pointing outside the restricted reference area is not included in the merge candidate list.

[0270] Specifically, when seq_refresh_enable_flag is 1, in the merge candidate (e.g., merge candidate N) derived by the merge prediction parameter derivation unit 3036, the reference picture index and the motion vector are in the restricted reference area, that is, when IsRA(Pb, mvLX_N) is true, the availability N of the merge candidate N is set to 1, and when it is outside the restricted reference area, the availability N is set to 0. In addition, only the merge candidate with the availability N of 1 may be set in the merge candidate list.

[0271] In addition, when seq_refresh_enable_flag is 1, it is determined whether the reference picture index refIdxLXN and the motion vector mvLXN point to the restricted reference area in the L0 reference picture and the L1 reference picture of the merge candidate (e.g., merge candidate N) derived by the merge prediction parameter derivation unit 3036. In the case of pointing to the restricted reference area, if the flag predFlagLX=1 (X=0, 1) is used, the merge candidate for bidirectional prediction can be set in the merge candidate list. In the case where the reference picture of one of the reference picture lists L0 and L1 is not available (!IsRA(Pb, mvLXN)), for the reference picture list LX with predFlagLX=1, the merge candidate for single prediction is set in the merge candidate list. In addition, in the case of predFlagLX=0 where the reference pictures of both the reference picture lists L0 and L1 are not available, the availabilityN of the merge candidate N is set to 0.

[0272] It can be that when seq_refresh_enable_flag is 1, the merge prediction parameter derivation unit 3036 sequentially refers to the generated merge candidate list, and if the reference picture index and motion vector in the generated merge candidate list indicate outside the restricted reference area, it is removed from the merge candidate list.

[0273] In the phased refresh information, when seq_refresh_mode=1 and seq_refresh_direction=0, the time of the reference picture indicated by the reference picture index of the merge candidate is t=j, and when the motion vector of the merge candidate is mvLX_AL, the merge candidate having the reference picture index and motion vector satisfying (Formula INTERP-9) is set to be an available merge candidate. xPb+wPb-1+mvLX_AL[0]<=xRA_en[j]-1 (calculation INTERP-9)

[0274] Or it can be derived by the following formula. xPb+wPb-1+mvLX_AL[0]<xRA_st[j]+wRA[j]

[0275] Or it can be derived by the following formula. xPb+wPb-1+mvLX_AL[0]<=(wIR-wOVLP)*(j+1)

[0276] On the other hand, in the phased refresh information, when seq_refresh_mode = 1 and seq_refresh_direction = 0, that is, the restricted area is as follows Fig.13 When shifting from top to bottom in the vertical direction as shown in (c), the reference picture index and motion vector of the merge candidate are used to determine the available merge candidate using (Formula INTERP-10). yPb+hPb-1+mvLX_AL[1]<=yRA_en[j]-1 (calculation INTERP-10)

[0277] Or it can be derived by the following formula. yPb+hPb-1+mvLX_AL[1]<yRA_st[j]+wRA[j]

[0278] Or it can be derived by the following formula. yPb+hPb-1+mvLX_AL[1]<=(hIR-hOVLP)*(j+1)

[0279] As described above, when random access is performed using the restricted region, by limiting the merging candidates so that the region referenced by the target block is included in the restricted reference region, normal decoding processing can be performed after the random access point.

[0280] Alternatively, as another example, the merge candidate derivation unit 30361 of this embodiment may also use the following formula to clip the merge candidate N motion vector mvLX_N to a position within the restricted area when seq_refresh_enable_flag is 1 and the object block Pb is within the restricted area (IsRA(Pb) is true). mvLX[0]=ClipRA(mvLX_N[0]) mvLX[1]=ClipRA(mvLX_N[1])

[0281] Alternatively, as another example where seq_refresh_enable_flag is 1, a case where, when the reference picture index and motion vector of the merge candidate are applied to the object block, the area referenced by the object block is included in the unrestricted reference area is described. In this case, the reference picture index can be corrected and the motion vector can be scaled to determine whether the reference area of ​​the object block is included in the restricted reference area IsRA(Pb, scaledMV).

[0282] In the case of seq_refresh_mode=0 and seq_refresh_direction=0 in the periodic refresh information, the available merge candidates are determined using (Formula INTERP-11) with reference to the picture index and the motion vector. xPb+wPb-1+scaledMV[0]<=xRA_en[j]-1 (calculation INTERP-11)

[0283] Or it can be derived by the following formula. xPb+wPb-1+scaledMV[0]<xRA_st[j]+wRA[j]

[0284] Or it can be derived by the following formula. xPb+wPb-1+scaledMV[0]<=(wIR-wOVLP)*(j'+1) scaledMV=MvScale(mvLX_AL,k,j,k,j')

[0285] Here, k is the time of the target picture, j is the time of the reference picture of the adjacent block, but these can also be regarded as POC. scaledMV is the scaled motion vector of the adjacent block, which is derived by (formula INTERP-1). j' is the time (POC) of the reference picture indicated by the modified reference picture index.

[0286] On the other hand, in restricted areas such as Fig.13 As shown in (c) of FIG. 1 , in the case of vertical shift (in the case where seq_refresh_mode = 1 and seq_refresh_direction = 0 in the periodic refresh information), the reference picture index and the motion vector are derived to satisfy (Formula INTERP-12). yPb+hPb-1+scaledMV[1]<=(hIR-hOVLP)*(j'+1)(calculation INTERP-12) scaledMV=MvScale(mvLX_AL,k,j,k,j')

[0287] As described above, when random access is performed using the restricted area, the reference area of ​​the target block is included in the restricted reference area by correcting the reference picture index and motion vector of the merging candidate, so that normal decoding processing can be performed after the random access point. (Intra-frame prediction image generation unit 310)

[0288] When the prediction mode predMode indicates the intra prediction mode, the intra prediction image generation unit 310 performs intra prediction using the intra prediction parameters input from the intra prediction parameter decoding unit 304 and the reference pixels read from the reference picture memory 306 .

[0289] Specifically, the intra prediction image generator 310 reads neighboring blocks within a predetermined range from the target block on the target picture from the reference picture memory 306. The predetermined range is the neighboring blocks to the left, upper left, upper, and upper right of the target block, and the reference area varies depending on the intra prediction mode.

[0290] The intra prediction image generation unit 310 generates a prediction image of the target block by referring to the read decoded pixel value and the prediction mode indicated by the intra prediction mode IntraPredMode. The intra prediction image generation unit 310 outputs the generated prediction image to the addition unit 312.

[0291] The following describes the generation of a predicted image based on an intra-frame prediction mode. In planar prediction, DC prediction, and angular prediction, a decoded surrounding area adjacent to the target block is set as a reference area. Then, the pixels in the reference area are extrapolated in a specific direction to generate a predicted image. For example, the reference area can be set to an L-shaped area (e.g., a region including the left and top (or further including the top left, top right, and bottom left) of the target block. Fig.24 (a) The area shown by the pixels marked with diagonal circles). (Details of the predicted image generation unit)

[0292] Next, use Fig.25 The following describes the details of the structure of the intra-frame prediction image generation unit 310. The intra-frame prediction image generation unit 310 includes: a target block setting unit 3101, an unfiltered reference image setting unit 3102 (a first reference image setting unit), a filtered reference image setting unit 3103 (a second reference image setting unit), a prediction unit 3104, and a prediction image correction unit 3105 (a prediction image correction unit, a filter switching unit, and a weighting coefficient changing unit).

[0293] The prediction unit 3104 generates a temporary prediction image (pre-correction prediction image) of the target block based on each reference pixel (unfiltered reference image) on the reference area, the filtered reference image generated by applying the reference pixel filter (first filter), and the intra prediction mode, and outputs it to the prediction image correction unit 3105. The prediction image correction unit 3105 corrects the temporary prediction image according to the intra prediction mode, generates a prediction image (corrected prediction image), and outputs it.

[0294] Hereinafter, each unit included in the intra-prediction image generation unit 310 will be described. (Object Block Setting Unit 3101)

[0295] The target block setting unit 3101 sets the target CU as the target block and outputs information related to the target block (target block information). The target block information includes at least the size, position, and index indicating brightness or color difference of the target block. (Unfiltered Reference Image Setting Unit 3102)

[0296] The unfiltered reference image setting unit 3102 sets the adjacent area of ​​the target block as the reference area based on the size and position of the target block. Then, the decoded pixel values ​​of the corresponding positions on the reference picture memory 306 are set for each pixel value in the reference area (unfiltered reference image, boundary pixel). Fig.24The row r[x][-1] of decoded pixels adjacent to the upper side of the object block and the column r[-1][y] of decoded pixels adjacent to the left side of the object block shown in (a) are the unfiltered reference image. The unfiltered reference image r[x][y] is set by (Formula INTRAP-1) using the decoded pixel value u[][] of the object image represented based on the upper left coordinate of the object image. r[x][y]=u[xPb+x][yPb+y] (calculation INTRAP-1) x=-1, y=-1..(maxBS*2-1) and x=0..(maxBS*2-1), y=-1

[0297] Here, (xPb, yPb) represents the upper left coordinate of the target block, and maxBS represents the larger value of the width wPb or the height hPb of the target block.

[0298] In (formula INTRAP-1), if Fig.24 As shown in (a), the row r[x][-1] of the decoded pixels adjacent to the top of the object block and the column r[-1][y] of the decoded pixels adjacent to the left of the object block are the unfiltered reference image. It should be noted that when there is no or no reference to the decoded pixel value corresponding to the reference pixel position, a predetermined value (for example, when the pixel bit depth is bitDepth, 1<<(bitDepth-1)) can be set as the unfiltered reference image. Alternatively, the referenceable decoded pixel value existing in the vicinity of the corresponding decoded pixel can be set as the unfiltered reference image. In addition, "y=-1..(maxBS*2-1)" means that y can take (maxBS*2+1) values ​​from -1 to (maxBS*2-1), and "x=0..(maxBS*2-1)" means that x can take (maxBS*2) values ​​from 0 to (maxBS*2-1). (Pixel value setting 1 of reference area of ​​restricted area)

[0299] When seq_refresh_enable_flag is 1 and the target block Pb is within the restricted area (IsRA(Pb) is true), the intra-frame prediction image generation unit 310 of the present embodiment does not refer to the boundary between the restricted area (first area) and the non-restricted area (second area). Fig.26That is, when seq_refresh_enable_flag is 1, the intra-frame prediction image generation unit 310 sets the pixels in the non-restricted area as "not available" as reference pixels. In this case, the intra-frame prediction image generation unit 310 sets r[wCU-1][-1] in the restricted area to the pixel value of the reference pixel r[x][-1] (x>=wCU) outside the restricted area. r[x][-1]=r[wCU-1][-1](wCU<=x<2*maxBS)(Formula INTRAP-2)

[0300] On the other hand, when the restriction area is shifted in the vertical direction, r[-1][hCU-1] is set as the pixel value of the reference pixel r[-1][y] (y>=hCU). r[-1][y]=r[-1][hCU-1](hCU<=y<2*maxBS)(Formula INTRAP-3)

[0301] By setting the reference area in this way, in any intra prediction mode, the predicted image is generated without referring to the pixel values ​​of the non-restricted area. Therefore, there is no need to change the processing of the filtered reference image setting unit 3103, the intra prediction unit 3104, and the predicted image correction unit 3105 described later. (Filtered Reference Image Setting Unit 3103)

[0302] The filtered reference image setting unit 3103 applies a reference pixel filter (first filter) to the unfiltered reference image according to the intra prediction mode, and derives a filtered reference image s[x][y] at each position (x, y) on the reference area. Specifically, a low-pass filter is applied to the unfiltered reference image at the position (x, y) and its surroundings, and a filtered reference image ( Fig.24 (b)). It should be noted that the low-pass filter is not necessarily applied to all intra-frame prediction modes, and the low-pass filter may be applied to some intra-frame prediction modes.

[0303] Next, the case where the reference area is set by the usual method shown in (Formula INTRAP-1) instead of using the method (setting pixel value of reference area 1) at the boundary between the restricted area and the unrestricted area in the unfiltered reference image setting unit 3102 is described.

[0304] When the reference pixel filter is a smoothing filter with a tap number NTAP, the filtered reference image setting unit 3103 Fig.26As shown in (b), the reference pixel filter is not applied to the pixels that are filtered with reference to the pixels in the non-restricted area. That is, the pixels in the non-restricted area are set as "not available" as reference pixels. In this case, the reference pixel filter may not be applied to the pixels in the non-restricted area or all the reference areas above the target block.

[0305] Alternatively, when the reference pixel filter is a linear filter that refers to a pixel value of r[maxBS*2-1][-1], filtering is not performed on at least the reference area above the target block.

[0306] On the other hand, when the restricted area is shifted in the vertical direction, the reference area not filtered is the reference area on the left side of the target block. In addition, when the reference pixel filter is a linear filter that refers to the pixel value of r[-1][maxBS*2-1], at least the reference area on the left side of the target block is not filtered.

[0307] When the reference region is set by a normal method, it is also necessary to perform processing different from the normal method in the intra prediction unit 3104 described later.

[0308] It should be noted that the filter applied to the unfiltered reference image on the reference area in the filtered reference pixel setting unit 3103 is called a "reference pixel filter (first filter)", and in contrast, the filter used to correct the temporary predicted image in the predicted image correction unit 3105 described later is called a "boundary filter (second filter)". (Configuration of Intra-frame Prediction Unit 3104)

[0309] The intra prediction unit 3104 generates a temporary prediction image (temporary prediction pixel value, prediction image before correction) of the target block based on the intra prediction mode, the unfiltered reference image, and the filtered reference pixel value, and outputs it to the prediction image correction unit 3105. The prediction unit 3104 includes: a Planar prediction unit 31041, a DC prediction unit 31042, an Angular prediction unit 31043, and a LM prediction unit 31044. The prediction unit 3104 selects a specific prediction unit according to the intra prediction mode, and inputs the unfiltered reference image and the filtered reference image. The relationship between the intra prediction mode and the corresponding prediction unit is as follows. .Planar forecast…Planar forecast department 31041 .DC Prediction…DC Prediction Department 31042 .Angular prediction…Angular prediction department 31043 .LM Forecast…LM Forecast Department 31044

[0310] When random access is achieved using a restricted area, if reference pixels are not set at the boundary between the restricted area and the non-restricted area by the method of (setting the pixel value of the reference area to 1), the Planar prediction mode shall not be used unless the conditions described below are set. In addition, regarding the Angular intra-frame prediction mode, the Angular intra-frame prediction mode shall not be used. Figure 5 The vertical prediction mode (50) shown is larger than the intra prediction mode (IntraPredModeY>50). This is because these modes refer to pixels included in the non-restricted area. (Planar forecast)

[0311] The planar prediction unit 31041 linearly adds a plurality of filtered reference images according to the distance between the target pixel position and the reference pixel position to generate a temporary predicted image, and outputs it to the predicted image correction unit 3105. The pixel value q[x][y] of the temporary predicted image is derived by (Formula INTRAP-4) using the filtered reference pixel value s[x][y] and the width wPb and height hPb of the target block. q[x][y]=((wPb-1-x)*s[-1][y]+(x+1)*s[wPb][-1]+(hPb-1-y)*s[x][-1]+(y+1)*s[-1][hPb]+maxBS)>>(l0g2(maxBS)+1)(Formula INTRAP-4)

[0312] Here, x=0..wPb-1, y=0..hPb-1. (Restricted Area and Planar Forecast)

[0313] At the boundary between the restricted area and the non-restricted area, if s[wPb][-1] referenced in (Formula INTRAP-4) is a pixel of the non-restricted area, it cannot be referenced in random access using the restricted area. In this embodiment, s[wPb-1][-1] is used instead of s[wPb][-1] to calculate the pixel value q[x][y] of the temporary prediction image. q[x][y]=((wPb-1-x)*s[-1][y]+(x+1)*s[wPb-1][-1]+(hPb-1-y)*s[x][-1]+(y+1)*s[-1][hPb]+maxBS)>>(log2(maxBS)+1)(Formula INTRAP-5)

[0314] Furthermore, s[-1][hPb-1] may be used instead of s[-1][hPb] to calculate the pixel value q[x][y] of the temporary prediction image. q[x][y]=((wPb-1-x)*s[-1][y]+(x+1)*s[wPb-1][-1]+(hPb-1-y)*s[x][-1]+(y+1)*s[-1][hPb-1]+maxBS)>>(log2(maxBS)+1)(Formula INTRAP-6)

[0315] In this way, by calculating the pixel value q[x][y] of the temporary prediction image with reference to only the pixel value of the restricted area, Planar prediction can be used even when random access is achieved using the restricted area.

[0316] As described above, in the intra-frame CU within the restricted area, when the adjacent reference pixels used in the Planar prediction cross the non-restricted area (the second area), the intra-frame prediction unit 3104 replaces the adjacent reference pixels of the non-restricted area with pixels in the restricted area, and generates a predicted image implemented by the Planar prediction. (DC Prediction)

[0317] The DC prediction unit 31042 derives a DC prediction value equivalent to the average value of the filtered reference image s[x][y], and outputs a temporary prediction image q[x][y] having the DC prediction value as a pixel value. (Angular prediction)

[0318] The Angular prediction unit 31043 generates a temporary prediction image q[x][y] using the filtered reference image s[x][y] in the prediction direction (reference direction) indicated by the intra prediction mode, and outputs it to the prediction image correction unit 3105 . (Restricted Areas and MPM Forecast)

[0319] When the object block is in the restricted area (IsRA(xPb, yPb) is true) and the reference position (xNbX, yNbX) of the adjacent block of the object block is in the non-restricted area (IsRA(xNbX, yNbX) is false), the intra-frame prediction parameter decoding unit 304 (MPM candidate list derivation unit 30421) does not use the value of the adjacent block in the derivation of the MPM candidate. That is, when the object block is in the restricted area (IsRA(xPb, yPb) is true) and the reference position (xNbX, yNbX) of the adjacent block of the object block is in the restricted area (IsRA(xNbX, yNbX) is true), the intra-frame prediction parameter decoding unit 304 uses the position (xNbX, yNbX) for the MPM candidate candIntraPredModeX.

[0320] For example, in the new refresh area such as Fig.13In the case of shifting from left to right as shown in (a) (seq_refresh_mode = 0 and seq_refresh_direction = 0), the object block is in the restricted area (IsRA(xPb, yPb) is true), and the right adjacent position (xPb+wPb, yPb-1) of the object block is in the restricted area (IsRA(xPb+wPb, yPb-1) is true), the intra-frame prediction parameter decoding unit 304 uses the value of the right adjacent position in the derivation of the MPM candidate.

[0321] For example, in the new refresh area such as Fig.13 In the case of shifting from right to left as shown in (b) (seq_refresh_mode = 0 and seq_refresh_direction = 1), the object block is in the restricted area (IsRA(xPb, yPb) is true), and the left adjacent position (xPb-1, yPb) of the object block is in the restricted area (IsRA(xPb-1, yPb) is true), the intra-frame prediction parameter decoding unit 304 uses the value of the left adjacent position in the derivation of the MPM candidate. (Restricted Area and Angular Prediction)

[0322] In the case where the restricted area extends from left to right (seq_refresh_mode = 0 and seq_refresh_direction = 0), in Angular prediction, Figure 5 In the vertical prediction mode (IntraPredModeVer=50) shown in FIG. 1 , the pixels in the non-restricted area at the boundary between the restricted area and the non-restricted area are referenced. Fig. 27 As shown in (a) and (b), when the restricted area is extended from left to right (seq_refresh_mode = 0 and seq_refresh_direction = 0), the reference pixel adjacent to the upper right of the object block is included in the non-restricted area. That is, when the width of the object block is set to wPb, s[x][-1] (x>=wPb) is the pixel value of the non-restricted area. Therefore, when random access is achieved using the restricted area, the intra-frame prediction mode is limited to less than IntraPredModeVer (IntraPredModeY<=IntraPredModeVer).

[0323] Specifically, when the object block is in the restricted area (IsRA(xPb, yPb) is true) and the right area of ​​the object block is in the unrestricted area (IsRA(xPb+wPb, yPb-1) is false), the intra-frame prediction parameter decoding unit 304 (MPM candidate list derivation unit 30421) deduces a value greater than IntraPredModeVer as an MPM candidate, and sets the MPM candidate as non-available (usable) and does not insert it into the MPM candidate list mpmCandList[]. In addition, in this case, the intra-frame prediction parameter decoding unit 304 can be cropped to IntraPredModeVer when the derived intra-frame prediction mode is greater than IntraPredModeVer. IntraPredMode=Min(IntraPredMode, IntraPredModeVer)

[0324] On the contrary, the intra-frame prediction parameter decoding unit 304 Fig.13 In the case of shifting from right to left as shown in (b) (seq_refresh_mode = 0 and seq_refresh_direction = 1), that is, when the left area of ​​the object block is in the non-restricted area (IsRA(xPb-1, yPb) is false), when a value less than IntraPredModeVer is derived for the MPM candidate, the MPM candidate is set to non-available and is not inserted into the MPM candidate list mpmCandList[]. In addition, in this case, the intra-frame prediction parameter decoding unit 304 can crop to IntraPredModeVer when the derived intra-frame prediction mode is less than IntraPredModeVer. IntraPredMode=Max(IntraPredMode, IntraPredModeVer)

[0325] On the other hand, the intra-frame prediction parameter decoding unit 304 Fig.13 In the case of vertical displacement as shown in (c), for example, Fig. 27As shown in (c) and (d), when the restricted area is extended from top to bottom (seq_refresh_mode = 1 and seq_refresh_direction = 0), the reference pixel adjacent to the lower left of the target block is included in the non-restricted area. That is, when the height of the target block is set to hPb, s[-1][y] (y>=hPb) is the pixel value of the non-restricted area. Therefore, the intra-frame prediction mode is limited to IntraPredModeHor (=18) or more as the horizontal prediction mode (IntraPredModeY>=IntraPredModeHor).

[0326] Specifically, when the left adjacent area of ​​the object block is in the non-restricted area (IsRA(xPb, yPb-1) is false), the intra-frame prediction parameter decoding unit 304 derives a value less than IntraPredModeHor as an MPM candidate, sets the MPM candidate as non-available and does not insert it into the MPM candidate list mpmCandList[]. In addition, in this case, when the derived intra-frame prediction mode is less than IntraPredModeHor, the intra-frame prediction parameter decoding unit 304 can be cropped to IntraPredModeHor. IntraPredMode=Max(IntraPredMode, IntraPredModeHor)

[0327] On the contrary, the intra-frame prediction parameter decoding unit 304 Fig.13 In the case of shifting from bottom to top as shown in (d) of FIG. 1 (seq_refresh_mode = 1 and seq_refresh_direction = 1), that is, when the upper adjacent area of ​​the object block is in the non-restricted area (IsRA(xPb-1, yPb-1) is false), when a value greater than IntraPredModeHor is derived for the MPM candidate, the MPM candidate is set to non-available and is not inserted into the MPM candidate list mpmCandList[]. In addition, in this case, the intra-frame prediction parameter decoding unit 304 can be cropped to IntraPredModeHor when the derived intra-frame prediction mode is greater than IntraPredModeHor. IntraPredMode=Min(IntraPredMode, IntraPredModeHor)

[0328] In this way, by limiting the intra-frame prediction mode, Angular prediction can be used even when random access is achieved using a new refresh area.

[0329] Through the above process, the intra prediction parameter decoding unit 304 removes the Angular prediction mode that uses the restricted area when the adjacent reference pixels in the intra CU in the restricted area straddle the non-restricted area (second area).

[0330] Through the above process, when adjacent reference pixels in an intra CU within the restricted area span across the non-restricted area (second area), the intra prediction parameter decoding unit 304 regards the area as not available. (LM forecast)

[0331] The LM prediction unit 31044 predicts the pixel value of chrominance based on the pixel value of brightness. Specifically, it is a method of generating a predicted image of a chrominance image (Cb, Cr) using a linear model based on the decoded brightness image. LM prediction includes CCLM (Cross-Component Linear Model prediction) prediction and MMLM (Multiple Model ccLM) prediction. CCLM prediction is a prediction method that uses one linear model for predicting chrominance based on brightness for one block. MMLM prediction is a prediction method that uses two or more linear models for predicting chrominance based on brightness for one block. (Configuration of the Prediction Image Correction Unit 3105)

[0332] The predicted image correction unit 3105 corrects the temporary predicted image output from the prediction unit 3104 according to the intra-frame prediction mode. Specifically, the predicted image correction unit 3105 performs weighted addition (weighted average) of the unfiltered reference image and the temporary predicted image according to the distance between the reference area of ​​each pixel and the object prediction pixel, thereby deriving the predicted image (corrected predicted image) Pred of the corrected temporary predicted image. It should be noted that in some intra-frame prediction modes, the temporary predicted image is not corrected by the predicted image correction unit 3105, and the output of the prediction unit 3104 can also be directly used as the predicted image.

[0333] The inverse quantization / inverse transformation unit 311 inversely quantizes the quantized transformation coefficient input from the entropy decoding unit 301 to obtain a transformation coefficient. The quantized transformation coefficient is a coefficient obtained by performing a frequency transformation such as DCT (Discrete Cosine Transform), DST (Discrete Sine Transform), KLT (Karyhnen Loeve Transform) on the prediction error in the encoding process and quantizing it. The inverse quantization / inverse transformation unit 311 performs an inverse frequency transformation such as inverse DCT, inverse DST, inverse KLT on the obtained transformation coefficient to calculate the prediction error. The inverse quantization / inverse transformation unit 311 outputs the prediction error to the addition unit 312.

[0334] The adder 312 adds the predicted image of the block input from the predicted image generator 308 and the prediction error input from the inverse quantization / inverse transform unit 311 for each pixel to generate a decoded image of the block. The adder 312 stores the decoded image of the block in the reference picture memory 306 and outputs it to the loop filter 305. (Configuration of Moving Image Coding Device)

[0335] Next, the configuration of the moving picture encoding device 11 according to this embodiment will be described. Fig.18 1 is a block diagram showing the structure of the video encoding device 11 of the present embodiment. The video encoding device 11 includes a prediction image generation unit 101, a subtraction unit 102, a transformation / quantization unit 103, an inverse quantization / inverse transformation unit 105, an addition unit 106, a loop filter 107, a prediction parameter memory (prediction parameter storage unit, frame memory) 108, a reference picture memory (reference picture storage unit, frame memory) 109, a coding parameter determination unit 110, a parameter coding unit 111, and an entropy coding unit 104.

[0336] The predicted image generation unit 101 generates a predicted image for each CU which is a region obtained by dividing each picture of each image T. The predicted image generation unit 101 performs the same operation as the predicted image generation unit 308 described above, and the description thereof is omitted here.

[0337] The subtraction unit 102 generates a prediction error by subtracting the pixel value of the predicted image of the block input from the predicted image generation unit 101 from the pixel value of the image T. The subtraction unit 102 outputs the prediction error to the transformation / quantization unit 103 .

[0338] The transform / quantization unit 103 calculates transform coefficients by frequency transforming the prediction error input from the subtraction unit 102 , and derives quantized transform coefficients by quantization. The transform / quantization unit 103 outputs the quantized transform coefficients to the entropy coding unit 104 and the inverse quantization / inverse transform unit 105 .

[0339] The inverse quantization / inverse transformation unit 105 and the inverse quantization / inverse transformation unit 311 ( Fig.17 ) is the same as that of FIG. 106, and its description is omitted here. The calculated prediction error is input to the adding unit 106.

[0340] The parameter encoding unit 111 is composed of a restriction region control unit 120 , an inter-prediction parameter encoding unit 112 and an intra-prediction parameter encoding unit 113 (not shown).

[0341] The restricted area control unit 120 includes a header coding unit 1110 , a CT information coding unit 1111 , a CU coding unit 1112 (prediction mode coding unit), and an inter prediction parameter coding unit 112 and an intra prediction parameter coding unit 113 (not shown). The CU coding unit 1112 further includes a TU coding unit 1114 .

[0342] The following describes the schematic operation of each module. The parameter encoding unit 111 performs encoding processing of parameters such as header information, partition information, prediction information, and quantized transform coefficients.

[0343] The CT information encoding unit 1111 encodes QT, MT (BT, TT) division information and the like based on the encoded data.

[0344] The CU encoding unit 1112 encodes CU information, prediction information, TU split flag split_transform_flag, CU residual flags cbf_cb, cbf_cr, cbf_luma, and the like.

[0345] When a prediction error is included in a TU, the TU coding unit 1114 codes the QP update information (quantization correction value) and the quantized prediction error (residual_coding).

[0346] The entropy coding unit 104 converts the syntax elements supplied from the supplier into binary data, generates coded data by entropy coding methods such as CABAC, and outputs the coded data. The suppliers of the syntax elements are the CT information coding unit 1111 and the CU coding unit 1112. The syntax elements are inter-frame prediction parameters (prediction mode predMode, merge flag merge_flag, merge index merge_idx, inter-frame prediction identifier inter_pred_idc, reference picture index refIdxLX, prediction vector index mvp_LX_idx, difference vector mvdLX), intra-frame prediction parameters (prev_intra_luma_pred_flag, mpm_idx, rem_selected_mode_flag, rem_selected_mode, rem_non_selected_mode), quantized transform coefficients, etc.

[0347] The entropy coding unit 104 performs entropy coding on the partition information, prediction parameters, quantized transform coefficients, etc. to generate a coded stream Te and output it. (Configuration of Inter-frame Prediction Parameter Encoding Unit)

[0348] The inter prediction parameter encoding unit 112 derives an inter prediction parameter based on the prediction parameter input from the encoding parameter determination unit 110. The inter prediction parameter encoding unit 112 includes the same configuration as that of the inter prediction parameter decoding unit 303 for deriving the inter prediction parameter.

[0349] The structure of the inter-frame prediction parameter encoding unit 112 is described. Fig.28 As shown, the structure includes: a parameter coding control unit 1121, an AMVP prediction parameter derivation unit 1122, a subtraction unit 1123, a sub-block prediction parameter derivation unit 1125, etc. The parameter coding control unit 1121 includes a merge index derivation unit 11211 and a vector candidate index derivation unit 11212. The merge index derivation unit 11211, the vector candidate index derivation unit 11212, the AMVP prediction parameter derivation unit 1122, and the sub-block prediction parameter derivation unit 1125 can also be collectively referred to as a motion vector derivation unit (motion vector derivation device). The inter-frame prediction parameter coding unit 112 outputs the motion vector (mvLX, subMvLX), the reference picture index refIdxLX, the inter-frame prediction identifier inter_pred_idc or the information representing these to the predicted image generation unit 101. In addition, the inter-frame prediction parameter encoding unit 112 outputs the merge flag merge_flag, the merge index merge_idx, the inter-frame prediction identifier inter_pred_idc, the reference picture index refIdxLX, the prediction vector index mvp_1X_idx, the difference vector mvdLX, and the sub-block prediction mode flag subPbMotionFlag to the entropy encoding unit 104.

[0350] The merge index derivation unit 11211 compares the motion vector and reference picture index input from the coding parameter determination unit 110 with the motion vector and reference picture index of the merge candidate block read from the prediction parameter memory 108, derives the merge index merge_idx, and outputs it to the entropy coding unit 104. The merge candidate is a reference block within a predetermined range from the target CU (for example, a reference block adjacent to the left end, lower left end, upper left end, upper end, and upper right end of the target block), and is a block that has completed the coding process. The vector candidate index derivation unit 11212 derives the prediction vector index mvp_1X_idx.

[0351] The AMVP prediction parameter derivation unit 1122 derives a prediction vector mvpLX based on the motion vector mvLX. The AMVP prediction parameter derivation unit 1122 outputs the prediction vector mvpLX to the subtraction unit 1123. It should be noted that the reference picture index refIdxLX and the prediction vector index mvp_1X_idx are output to the entropy coding unit 104.

[0352] The subtraction unit 1123 subtracts the prediction vector mvpLX output from the AMVP prediction parameter derivation unit 1122 from the motion vector mvLX input from the encoding parameter determination unit 110 , thereby generating a difference vector mvdLX. The difference vector mvdLX is output to the entropy encoding unit 104 . (Configuration of Intra-frame Prediction Parameter Coding Unit 113)

[0353] The intra prediction parameter encoder 113 derives a format for encoding (eg, mpm_idx, remintra_luma_pred_mode, etc.) according to the intra prediction mode IntraPredMode input from the encoding parameter determiner 110. The intra prediction parameter encoder 113 includes the same configuration as that of the intra prediction parameter decoder 304 for deriving intra prediction parameters.

[0354] Fig.29 1 is a schematic diagram showing the structure of the intra prediction parameter coding unit 113 of the parameter coding unit 111. The intra prediction parameter coding unit 113 includes a parameter coding control unit 1131, a luma intra prediction parameter derivation unit 1132, and a chroma intra prediction parameter derivation unit 1133.

[0355] The luma prediction mode IntraPredModeY and the chroma prediction mode IntraPredModeC are input from the encoding parameter determination unit 110 to the parameter encoding control unit 1131. The parameter encoding control unit 1131 determines prev_intra_luma_pred_flag with reference to the MPM candidate list mpmCandList[] of the reference candidate list derivation unit 30421. Then, prev_intra_luma_pred_flag and the luma prediction mode IntraPredModeY are output to the luma intra prediction parameter derivation unit 1132. In addition, the chroma prediction mode IntraPredModeC is output to the chroma intra prediction parameter derivation unit 1133.

[0356] The luma intra prediction parameter derivation unit 1132 is configured to include an MPM candidate list derivation unit 30421 (candidate list derivation unit), an MPM parameter derivation unit 11322 (parameter derivation unit), and a non-MPM parameter derivation unit 11323 (encoding unit, derivation unit).

[0357] The MPM candidate list derivation unit 30421 refers to the intra prediction mode of the adjacent block stored in the prediction parameter memory 108 to derive the MPM candidate list mpmCandList[]. When prev_intra_luma_pred_flag is 1, the MPM parameter derivation unit 11322 derives mpm_idx according to the luma prediction mode IntraPredModeY and the MPM candidate list mpmCandList[], and outputs it to the entropy coding unit 104. When prev_intra_luma_pred_flag is 0, the non-MPM parameter derivation unit 11323 derives RemIntraPredMode according to the luma prediction mode IntraPredModeY and the MPM candidate list mpmCandList[], and outputs rem_selected_mode or rem_non_selected_mode to the entropy coding unit 104.

[0358] The chroma intra prediction parameter derivation unit 1133 derives not_dm_chroma_flag, not_lm_chroma_flag, and chroma_intra_mode_idx according to the luma prediction mode IntraPredModeY and the chroma prediction mode IntraPredModeC, and outputs them.

[0359] The adder 106 generates a decoded image by adding pixel values ​​of the block prediction image input from the prediction image generator 101 and the prediction error input from the inverse quantization / inverse transform unit 105 for each pixel. The adder 106 stores the generated decoded image in the reference picture memory 109.

[0360] The loop filter 107 performs deblocking filtering, SAO, and ALF on the decoded image generated by the adding unit 106. It should be noted that the loop filter 107 does not necessarily include the above three filters, and may include only a deblocking filter, for example.

[0361] The prediction parameter memory 108 stores the prediction parameter generated by the encoding parameter determination unit 110 in a predetermined position for each target picture and CU.

[0362] The reference picture memory 109 stores the decoded image generated by the loop filter 107 at a predetermined position for each target picture and CU.

[0363] The coding parameter determination unit 110 selects one of a plurality of sets of coding parameters. The coding parameters refer to the QT, BT or TT partition information, prediction parameters or parameters generated in association with them as the coding target. The predicted image generation unit 101 generates a predicted image using these coding parameters.

[0364] The coding parameter determination unit 110 calculates the RD cost value representing the size of the information amount and the coding error for each of the multiple sets. The RD cost value is, for example, the sum of the coding amount and the value obtained by multiplying the square error by the coefficient λ. The coding amount is the amount of information of the coding stream Te obtained by entropy encoding the quantization error and the coding parameter. The square error is the sum of the squares of the prediction error calculated in the subtraction unit 102. The coefficient λ is a real number greater than a preset zero. The coding parameter determination unit 110 selects a set of coding parameters with the smallest calculated cost value. Thus, the entropy coding unit 104 outputs the selected coding parameter set as the coding stream Te. The coding parameter determination unit 110 stores the determined coding parameters in the prediction parameter memory 108.

[0365] It should be noted that a part of the moving image encoding device 11 and the moving image decoding device 31 in the above-mentioned embodiment can be implemented by a computer, for example, the entropy decoding unit 301, the parameter decoding unit 302, the loop filter 305, the predicted image generation unit 308, the inverse quantization / inverse transformation unit 311, the addition unit 312, the predicted image generation unit 101, the subtraction unit 102, the transformation / quantization unit 103, the entropy encoding unit 104, the inverse quantization / inverse transformation unit 105, the loop filter 107, the encoding parameter determination unit 110 and the parameter encoding unit 111. In this case, the program for realizing the control function can be recorded in a computer-readable recording medium, and the computer system can read the program recorded in the recording medium and execute it. It should be noted that the "computer system" mentioned here refers to a computer system built into any one of the moving image encoding device 11 and the moving image decoding device 31, and a computer system including hardware such as an OS and peripheral devices is used. In addition, "computer-readable recording medium" refers to removable media such as floppy disks, magneto-optical disks, ROMs, CD-ROMs, and storage devices such as hard disks built into computer systems. Moreover, "computer-readable recording medium" may also include: recording media that dynamically store programs for a short period of time, such as communication lines when sending programs via networks such as the Internet or communication lines such as telephone lines; and recording media that store programs for a fixed period of time, such as volatile memories inside computer systems that serve as servers or clients in this case. In addition, the above-mentioned program may be a program for realizing a part of the above-mentioned functions, or a program that can realize the above-mentioned functions by combining with programs already recorded in the computer system.

[0366] A motion image decoding device of one embodiment of the present invention is characterized in that a picture is divided into a restricted area and a non-restricted area, and for blocks included in the restricted area, prediction is performed using intra-frame prediction with reference only to pixels in the restricted area of ​​the above picture or inter-frame prediction with reference to a restricted reference area of ​​a reference picture of the above picture, and for blocks included in the non-restricted area, prediction is performed using intra-frame prediction with reference to decoded pixels in the above picture or inter-frame prediction with reference to a reference picture of the above picture, and after the above picture is decoded, the above-mentioned restricted area of ​​the above-mentioned picture is set as the above-mentioned restricted reference area.

[0367] A motion image decoding device of one embodiment of the present invention is characterized in that a picture is divided into a new refresh area, a restricted area and a non-restricted area, and for blocks included in the new refresh area, intra-frame prediction is performed with reference only to pixels in the restricted area, and for blocks included in the restricted area, intra-frame prediction is performed with reference only to pixels in the new refresh area or the restricted area, or inter-frame prediction is performed with reference to the restricted reference area of ​​a reference picture, and for blocks included in the non-restricted area, prediction is performed using intra-frame prediction with reference to decoded pixels in the picture or inter-frame prediction with reference to a reference picture of the picture, and after decoding the picture, the restricted area to which the new refresh area is added is set as the restricted reference area.

[0368] A motion image decoding device of one embodiment of the present invention is characterized in that a picture is divided into a restricted area and a non-restricted area, and for blocks included in a new refresh area in the restricted area, intra-frame prediction is performed with reference only to pixels of the restricted area, and for blocks included in the restricted area, intra-frame prediction is performed with reference only to pixels of the new refresh area or the restricted area, or inter-frame prediction is performed with reference to a restricted reference area of ​​a reference picture, and for blocks included in the non-restricted area, prediction is performed using intra-frame prediction with reference to decoded pixels in the picture or inter-frame prediction with reference to a reference picture of the picture, and after the picture is decoded, the restricted area is set as the restricted reference area.

[0369] A moving picture decoding device according to one aspect of the present invention is characterized in that, in the newly updated area, areas decoded at time t and time t-1 overlap (overlap).

[0370] A motion image encoding device of one embodiment of the present invention is characterized in that a picture is divided into a restricted area and a non-restricted area, and for blocks included in the restricted area, prediction is performed using intra-frame prediction with reference only to pixels in the restricted area of ​​the above picture or inter-frame prediction with reference to a restricted reference area of ​​a reference picture of the above picture, and for blocks included in the non-restricted area, prediction is performed using intra-frame prediction with reference to decoded pixels in the above picture or inter-frame prediction with reference to a reference picture of the above picture, and after the above picture is encoded, the above-mentioned restricted area of ​​the above-mentioned picture is set as the above-mentioned restricted reference area.

[0371] A moving picture decoding device according to one aspect of the present invention is characterized by comprising a deblocking filter configured to disable filtering of a block in filtering of two blocks sandwiching the restricted area and the non-restricted area.

[0372] A moving picture decoding device according to one aspect of the present invention is characterized by comprising a deblocking filter for turning off filtering of a block included in the restricted area in filtering of two blocks sandwiching the restricted area and the non-restricted area.

[0373] In one aspect of the moving picture decoding device of the present invention, the width of the overlapping area is derived from a minimum value of the number of pixels corrected by filtering and a width of the block.

[0374] A motion image decoding device according to one embodiment of the present invention is characterized in that it comprises a prediction image generation unit, which sets pixels in a non-restricted area as "not usable as reference pixels" at a boundary between the restricted area and the non-restricted area, and sets pixel values ​​of the restricted area in a reference area of ​​an object block.

[0375] A moving picture decoding device according to one aspect of the present invention includes a predicted picture generation unit configured to turn off reference pixel filtering for pixels that refer to pixels in the non-restricted area at a boundary between the restricted area and the non-restricted area.

[0376] A moving picture decoding device according to one aspect of the present invention is characterized by comprising a predicted image generating unit which removes the intra prediction mode when reference pixels in a prediction direction indicated by the intra prediction mode in a target block in a restricted area include a non-restricted area.

[0377] A motion image decoding device according to one embodiment of the present invention is characterized in that it includes a prediction image generation unit, which sets the reference pixels in a target block within a restricted area as "unavailable" when the reference pixels in the prediction direction indicated by the intra-frame prediction mode include a non-restricted area.

[0378] A motion image decoding device according to one embodiment of the present invention is characterized in that it comprises a prediction image generating unit which replaces reference pixels of a non-restricted area with pixels of a restricted area when pixels referenced in planar prediction include pixels of a non-restricted area in an object block within a restricted area.

[0379] A moving picture decoding device according to one aspect of the present invention is characterized by comprising a predicted image generating unit configured to remove a planar prediction mode when pixels referenced in planar prediction in a target block in a restricted area include a non-restricted area.

[0380] A motion image decoding device according to one embodiment of the present invention is characterized in that it comprises a motion compensation unit which, when pixels of a picture referenced by an object block in a restricted area are in a non-restricted reference area, performs a filling process of replacing pixel values ​​of the non-restricted reference area in the reference picture with pixel values ​​of the restricted reference area.

[0381] A motion image decoding device according to one embodiment of the present invention is characterized in that it includes a motion compensation unit which limits (crops) a vector pointing to the non-restricted reference area to within a restricted reference boundary when pixels of a picture referenced by an object block in a restricted area are in a non-restricted reference area.

[0382] A motion image decoding device according to one embodiment of the present invention is characterized in that it comprises a motion compensation unit, which selects an adjacent block from a merge candidate list and sets it as a merge candidate, wherein the adjacent block has a combination of a reference picture index and a motion vector such that the area referred to by the object block of a restricted area is included in the restricted reference area.

[0383] A motion image decoding device according to one embodiment of the present invention is characterized in that it comprises a motion compensation unit, which is configured to correct the reference image index and scale the motion vector in a case where a reference area of ​​a reference image index and a motion vector based on a merge candidate is not included in the restricted reference area in an object block in a restricted area, so that the reference area of ​​the object block is included in the restricted reference area.

[0384] In addition, part or all of the moving picture encoding device 11 and the moving picture decoding device 31 in the above-mentioned embodiment can also be implemented as an integrated circuit such as LSI (Large Scale Integration). Each functional block of the moving picture encoding device 11 and the moving picture decoding device 31 can be individually processor-based, or part or all of them can be integrated to be processor-based. In addition, the method of integrated circuitization is not limited to LSI, and it can also be implemented by a dedicated circuit or a general-purpose processor. In addition, when a technology for integrated circuitization that replaces LSI appears with the advancement of semiconductor technology, an integrated circuit based on this technology can also be used.

[0385] As mentioned above, one embodiment of the present invention has been described in detail with reference to the drawings, but the specific configuration is not limited to the above embodiment, and various design changes and the like can be made within the scope not departing from the gist of the present invention. (Application example)

[0386] The above-mentioned moving picture encoding device 11 and moving picture decoding device 31 can be mounted on various devices for sending, receiving, recording, and reproducing moving pictures. It should be noted that the moving pictures can be natural moving pictures taken by a camera or the like, or artificial moving pictures (including CG and GUI) generated by a computer or the like.

[0387] First, refer to Fig.30 A case where the above-mentioned moving picture encoding device 11 and moving picture decoding device 31 can be used for transmission and reception of moving pictures will be described.

[0388] Fig.30 (a) is a block diagram showing the structure of a transmitting device PROD_A equipped with a motion picture encoding device 11. Fig.30 As shown in (a), the transmitting device PROD_A includes: an encoding unit PROD_A1 that obtains encoded data by encoding a moving image, a modulation unit PROD_A2 that obtains a modulated signal by modulating a carrier using the encoded data obtained by the encoding unit PROD_A1, and a transmitting unit PROD_A3 that transmits the modulated signal obtained by the modulation unit PROD_A2. The moving image encoding device 11 described above is used as the encoding unit PROD_A1.

[0389] As a source of motion images input to the encoding unit PROD_A1, the transmitting device PROD_A may further include: a camera PROD_A4 for shooting motion images, a recording medium PROD_A5 for recording motion images, an input terminal PROD_A6 for inputting motion images from the outside, and an image processing unit A7 for generating or processing images. Fig.30(a) shows an example in which the transmitting device PROD_A has all of these structures, but some of them may be omitted.

[0390] It should be noted that the recording medium PROD_A5 may be a medium on which an uncoded moving image is recorded, or may be a medium on which a moving image coded in a recording coding method different from a transmission coding method is recorded. In the latter case, it is preferred that a decoding unit (not shown) that decodes coded data read from the recording medium PROD_A5 in accordance with the recording coding method is interposed between the recording medium PROD_A5 and the coding unit PROD_A1.

[0391] Fig.30 (b) is a block diagram showing the structure of a receiving device PROD_B equipped with a motion picture decoding device 31. Fig.30 As shown in (b), the receiving device PROD_B includes: a receiving unit PROD_B1 that receives a modulated signal, a demodulating unit PROD_B2 that obtains coded data by demodulating the modulated signal received by the receiving unit PROD_B1, and a decoding unit PROD_B3 that obtains a moving image by decoding the coded data obtained by the demodulating unit PROD_B2. The moving image decoding device 31 described above is used as the decoding unit PROD_B3.

[0392] The receiving device PROD_B, as a supply destination of the moving image output by the decoding unit PROD_B3, may further include a display PROD_B4 for displaying the moving image, a recording medium PROD_B5 for recording the moving image, and an output terminal PROD_B6 for outputting the moving image to the outside. Fig.30 (b) shows an example in which the receiving device PROD_B has all of these structures, but some of them may be omitted.

[0393] It should be noted that the recording medium PROD_B5 may be a medium for recording uncoded moving images, or may be a medium coded in a recording coding method different from a transmission coding method. In the latter case, it is preferred that a coding unit (not shown) that codes the moving images obtained from the decoding unit PROD_B3 in a recording coding method is interposed between the decoding unit PROD_B3 and the recording medium PROD_B5.

[0394] It should be noted that the transmission medium for transmitting the modulated signal can be wireless or wired. In addition, the transmission scheme for transmitting the modulated signal can be broadcast (here, it refers to a transmission scheme in which the transmission destination is not predetermined) or communication (here, it refers to a transmission scheme in which the transmission destination is predetermined). That is, the transmission of the modulated signal can be achieved through any one of wireless broadcasting, wired broadcasting, wireless communication and wired communication.

[0395] For example, a broadcasting station (broadcasting equipment, etc.) / receiving station (television receiver, etc.) of terrestrial digital broadcasting is an example of a transmitting device PROD_A / receiving device PROD_B that transmits and receives modulated signals via wireless broadcasting. In addition, a broadcasting station (broadcasting equipment, etc.) / receiving station (television receiver, etc.) of cable television broadcasting is an example of a transmitting device PROD_A / receiving device PROD_B that transmits and receives modulated signals via wired broadcasting.

[0396] In addition, a server (workstation, etc.) / client (television receiver, personal computer, smart phone, etc.) of a VOD (Video On Demand) service or a motion picture sharing service using the Internet is an example of a transmitter PROD_A / receiver PROD_B that transmits and receives a modulated signal through communication (usually, a wireless or wired transmission medium is used as a transmission medium in a LAN, and a wired transmission medium is used as a transmission medium in a WAN). Here, a personal computer includes a desktop PC, a laptop PC, and a tablet PC. In addition, a smart phone also includes a multifunctional portable phone terminal.

[0397] It should be noted that the client of the moving image sharing service has the function of decoding the encoded data downloaded from the server and displaying it on the display, and also has the function of encoding the moving images captured by the camera and uploading them to the server. That is, the client of the moving image sharing service plays the role of both the sending device PROD_A and the receiving device PROD_B.

[0398] Next, refer to Fig.31 , a description is given of a case where the above-mentioned motion image encoding device 11 and motion image decoding device 31 can be used for recording and reproducing motion images.

[0399] Fig.31 (a) is a block diagram showing the structure of a recording device PROD_C equipped with the above-mentioned motion picture encoding device 11. Fig.31As shown in (a) of FIG. 1 , the recording device PROD_C includes a coding unit PROD_C1 for obtaining coded data by coding a moving picture, and a writing unit PROD_C2 for writing the coded data obtained by the coding unit PROD_C1 into the recording medium PROD_M. The moving picture coding device 11 described above is used as the coding unit PROD_C1.

[0400] It should be noted that the recording medium PROD_M can be (1) a type of recording medium built into the recording device PROD_C such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive), or (2) a type of recording medium connected to the recording device PROD_C such as an SD memory card or a USB (Universal Serial Bus) flash memory, or (3) a recording medium loaded into a drive device (not shown) built into the recording device PROD_C such as a DVD (Digital Versatile Disc, registered trademark) or a BD (Blu-ray Disc, registered trademark).

[0401] In addition, as a supply source of moving images input to the encoding unit PROD_C1, the recording device PROD_C may further include: a camera PROD_C3 for shooting moving images, an input terminal PROD_C4 for inputting moving images from the outside, a receiving unit PROD_C5 for receiving moving images, and an image processing unit PROD_C6 for generating or processing images. The figure shows an example of a recording device PROD_C having all of these components, but some of them may be omitted.

[0402] It should be noted that the receiving unit PROD_C5 can receive uncoded moving images, or can receive coded data coded in a transmission coding method different from the recording coding method. In the latter case, it is preferable to place a transmission decoding unit (not shown) that decodes the coded data coded in the transmission coding method between the receiving unit PROD_C5 and the coding unit PROD_C1.

[0403] Examples of such a recording device PROD_C include a DVD recorder, a BD recorder, and a HDD (Hard Disk Drive) recorder (in this case, the input terminal PROD_C4 or the receiving unit PROD_C5 is the main source of motion images). In addition, a portable camera (in this case, the camera PROD_C3 is the main source of motion images), a personal computer (in this case, the receiving unit PROD_C5 or the image processing unit C6 is the main source of motion images), and a smart phone (in this case, the camera PROD_C3 or the receiving unit PROD_C5 is the main source of motion images) are also examples of such a recording device PROD_C.

[0404] Fig.31 (b) is a block diagram showing the structure of a playback device PROD_D equipped with the above-mentioned motion picture decoding device 31. Fig.31 As shown in (b) of FIG. 1 , the playback device PROD_D includes a reading unit PROD_D1 for reading coded data written in the recording medium PROD_M and a decoding unit PROD_D2 for obtaining a moving image by decoding the coded data read by the reading unit PROD_D1. The moving image decoding device 31 described above is used as the decoding unit PROD_D2.

[0405] It should be noted that the recording medium PROD_M can be (1) a type of recording medium built into the reproduction device PROD_D such as an HDD, SSD, etc., or (2) a type of recording medium connected to the reproduction device PROD_D such as an SD memory card, USB flash memory, etc., or (3) a recording medium loaded into a drive device (not shown) built into the reproduction device PROD_D such as a DVD, BD, etc.

[0406] In addition, as a supply destination of the moving image output by the decoding unit PROD_D2, the reproduction device PROD_D may further include: a display PROD_D3 for displaying the moving image, an output terminal PROD_D4 for outputting the moving image to the outside, and a transmission unit PROD_D5 for transmitting the moving image. In the figure, the reproduction device PROD_D is shown as having all of these structures, but some of them may be omitted.

[0407] It should be noted that the sending unit PROD_D5 can send uncoded moving images or can send coded data coded in a transmission coding method different from the recording coding method. In the latter case, it is better to place a coding unit (not shown) that encodes the moving images in the transmission coding method between the decoding unit PROD_D2 and the sending unit PROD_D5.

[0408] Examples of such a reproduction device PROD_D include a DVD player, a BD player, and an HDD player (in this case, the output terminal PROD_D4 connected to a television receiver or the like is the main supply destination of the motion image). In addition, a television receiver (in this case, the display PROD_D3 is the main supply destination of the motion image), a digital signage (also called an electronic billboard, an electronic bulletin board, etc., the display PROD_D3 or the transmission unit PROD_D5 is the main supply destination of the motion image), a desktop PC (in this case, the output terminal PROD_D4 or the transmission unit PROD_D5 is the main supply destination of the motion image), a laptop or tablet PC (in this case, the display PROD_D3 or the transmission unit PROD_D5 is the main supply destination of the motion image), and a smartphone (in this case, the display PROD_D3 or the transmission unit PROD_D5 is the main supply destination of the motion image) are also examples of such a reproduction device PROD_D. (Hardware implementation and software implementation)

[0409] Furthermore, each block of the above-described moving picture decoding device 31 and moving picture encoding device 11 may be implemented in hardware by a logic circuit formed on an integrated circuit (IC chip), or may be implemented in software by using a CPU (Central Processing Unit).

[0410] In the latter case, each of the above-mentioned devices has: a CPU that executes commands of a program that realizes each function, a ROM (Read Only Memory) that stores the above-mentioned program, a RAM (Random Access Memory) that expands the above-mentioned program, and a storage device (recording medium) such as a memory that stores the above-mentioned program and various data. Then, the purpose of the embodiment of the present invention can also be achieved in the following manner: a recording medium that records the program code (executable form program, intermediate code program, source program) of the control program of the software that realizes the above-mentioned functions in a computer-readable manner is supplied to each of the above-mentioned devices, and the computer (or CPU, MPU) reads the program code recorded in the recording medium and executes it.

[0411] As the above-mentioned recording medium, for example, the following tapes can be used: tapes such as magnetic tapes and cassette tapes; magnetic disks including floppy disks (registered trademark) / hard disks, CD-ROMs (Compact Disc Read-Only Memory) / MO disks (Magneto-Optical Disc) / MD (Mini Disc) / DVD (Digital Versatile Disc) / CD-R (CD Recordable) / Blu-ray Discs (Blu-ray Disc) and the like; cards such as IC cards (including memory cards) / optical cards; semiconductor memories such as mask ROMs / EPROMs (Erasable Programmable Read-Only Memory) / EEPROMs (Electrically Erasable and Programmable Read-Only Memory) / flash ROMs; or PLDs (Programmable logic device), FPGAs (Field Programmable Gate Arrays) and the like. Array: Field Programmable Gate Array) and other logic circuits, etc.

[0412] In addition, each of the above-mentioned devices can also be configured to be connected to a communication network, and the above-mentioned program code can be supplied via the communication network. The communication network can transmit program codes without being particularly limited. For example, the Internet, intranet, extranet, LAN (Local Area Network), ISDN (Integrated Services Digital Network), VAN (Value-Added Network), CATV (Community R Antenna television / Cable Television) communication network, virtual private network (Virtual Private Network), telephone line network, mobile communication network, satellite communication network, etc. can be used. In addition, the transmission medium constituting the communication network can also be a medium capable of transmitting program codes, and is not limited to a specific structure or type. For example, it can be used in wired communication such as IEEE (Institute of Electrical and Electronic Engineers) 1394, USB, power line transmission, wired TV line, telephone line, ADSL (Asymmetric Digital Subscriber Line) line, or wireless communication such as IrDA (Infrared Data Association), infrared such as remote controller, BlueTooth (registered trademark), IEEE802.11 wireless, HDR (High Data Rate), NFC (Near Field Communication), DLNA (registered trademark) (Digital Living Network Alliance), mobile phone network, satellite line, terrestrial digital broadcasting network, etc. It should be noted that the embodiments of the present invention can be implemented in the form of a computer data signal embedded in a carrier wave that embodies the above-mentioned program code through electronic transmission.

[0413] The embodiments of the present invention are not limited to the above-mentioned embodiments, and various modifications can be made within the scope of the claims. That is, embodiments obtained by combining technical solutions appropriately modified within the scope of the claims are also included in the technical scope of the present invention. Industrial Applicability

[0414] The embodiments of the present invention can be preferably applied to a motion picture decoding device that decodes coded data obtained by encoding image data, and a motion picture encoding device that generates coded data obtained by encoding image data. In addition, the embodiments of the present invention can be preferably applied to a data structure of coded data generated by a motion picture encoding device and referred to by a motion picture decoding device. (Cross-reference of related applications)

[0415] This application claims the benefit of priority to Japanese Patent Application No. 2018-147658 filed on August 6, 2018, Japanese Patent Application No. 2018-148470 filed on August 7, 2018, and Japanese Patent Application No. 2018-148471 filed on August 7, 2018, and all of their contents are incorporated into this specification by reference thereto. Description of Reference Numerals 31 Image decoding device 301 Entropy Decoding Unit 302 parameter decoding unit 3020 Header decoding unit 303 Inter-frame prediction parameter decoding unit 304 Intra-frame prediction parameter decoding unit 308 prediction image generation unit 309 inter-frame prediction image generation unit 310 Intra-frame prediction image generation unit 311 Inverse quantization / inverse transformation unit 312 Addition Department 320 Restricted Area Control Department 11Image Coding Device 101 Prediction image generation unit 102 Subtraction Department 103 Transformation / Quantization Unit 104 Entropy Coding Unit 105 Inverse quantization / inverse transformation unit 107 loop filter 110 Coding parameter determination unit 111 parameter encoding unit 112 Inter-frame prediction parameter encoding unit 113 Intra-frame prediction parameter encoding unit 120 Restricted Area Control Department 1110 Header Encoding Unit 1111CT Information Coding Department 1112CU encoding unit (prediction mode encoding unit) 1114TU Coding Department

Claims

1. An image decoding device, for decoding a picture, The above-mentioned image decoding device comprises: a parameter decoder, which decodes a flag specifying whether a position syntax element is signaled, the flag being represented by a 1-bit unsigned integer, and, when the value of the flag is equal to 1, decodes the position syntax element associated with a position of a region boundary in a unit specifying a region; an image setting circuit, which sets the top boundary position to the position of the region boundary when the value of the flag is equal to 1 and the horizontal coordinate of the object block is less than 0; as well as A deblocking filter is used to perform a deblocking filter process on the region boundary when the value of the flag is equal to 0. Wherein, when the value of the flag is equal to 1, the deblocking filter processing is not applied to the region boundary. The distance between two region boundaries is equal to the coding block width.

2. An image encoding device for encoding an image, The above-mentioned image encoding device comprises: a parameter encoder for encoding a flag specifying whether a position syntax element is signaled, the flag being represented by a 1-bit unsigned integer, and, when the value of the flag is equal to 1, encoding the position syntax element associated with a position of a region boundary in a unit specifying a region; an image setting circuit, which sets the top boundary position to the position of the region boundary when the value of the flag is equal to 1 and the horizontal coordinate of the object block is less than 0; as well as A deblocking filter is used to perform a deblocking filter process on the region boundary when the value of the flag is equal to 0. Wherein, when the value of the flag is equal to 1, the deblocking filter processing is not applied to the region boundary. The distance between two region boundaries is equal to the coding block width.

3. A non-volatile computer-readable medium storing a bit stream generated by encoding an image, the bit stream being decoded by: decoding, from the bitstream, a flag specifying whether a position syntax element is signaled, the flag being represented by a 1-bit unsigned integer, and, when the value of the flag is equal to 1, decoding the position syntax element associated with the position of the region boundary in the unit specifying the region; When the value of the flag is equal to 1 and the horizontal coordinate of the object block is less than 0, the top boundary position is set to the position of the region boundary; and When the value of the flag is equal to 0, deblocking filtering is performed on the region boundary. in, When the value of the flag is equal to 1, the deblocking filter is not applied to the region boundary. The distance between two region boundaries is equal to the coding block width.

Images