Encoding method and decoding method

Abstract

The present disclosure relates to an encoding method and a decoding method. The encoding method performs a first or a second action based on a NAL unit type, the first action comprising: encoding a first picture; and encoding a second picture after encoding a parameter set of the second picture, the second picture following the first picture in encoding order, the second picture having a same temporal ID as the parameter set of the second picture, the temporal ID indicating a hierarchical level of temporal scalability, wherein the NAL unit type of the second picture is a NAL unit type of a step-wise temporal sub-layer access (STSA) picture, the second action comprising: encoding the first picture; and encoding a third picture after encoding the first picture without encoding a parameter set of the third picture, the third picture following the first picture in encoding order, the third picture having a different temporal ID from the second picture, wherein the NAL unit type of the third picture is not the NAL unit type of the STSA picture.

Description

[0001] This application is a divisional application of the invention patent application filed on October 3, 2018, with application number 201880064750.1 and entitled "Encoding device, decoding device, encoding method and decoding method". Technical Field

[0002] This invention relates to encoding and decoding methods. Background Technology

[0003] Previously, H.265 existed as a specification for encoding moving images. H.265 is also known as HEVC (High Efficiency Video Coding) (Non-Patent Document 1).

[0004] Existing technical documents

[0005] Non-patent literature

[0006] Non-patent literature 1: H.265 (ISO / IEC 23008-2 HEVC) / HEVC (High Efficiency Video Coding) Summary of the Invention

[0007] The problem that the invention aims to solve

[0008] However, motion picture encoding cannot be performed properly without properly setting the information related to motion picture encoding.

[0009] Therefore, the present invention provides an encoding device, etc., capable of appropriately setting information associated with the encoding of motion images.

[0010] Methods for solving problems

[0011] An encoding method for a technical solution of the present invention performs a first action or a second action based on the Network Abstraction Layer (NAL) unit type. The first action includes: encoding a first image; and encoding a second image after encoding the parameter set of a second image, wherein the second image follows the first image in the encoding order, and the parameter set of the second image and the second image have the same time ID, the time ID indicating the layer with respect to time scalability, wherein the NAL unit type of the second image is the NAL unit type of Step-Time Sublayer Access (STSA) image. The second action includes: encoding the first image; and encoding a third image without encoding the parameter set of the third image after encoding the first image, wherein the third image follows the first image in the encoding order, and the third image has a different time ID than the second image, wherein the NAL unit type of the third image is not the NAL unit type of the STSA image.

[0012] In addition, these inclusive or specific technical solutions can also be implemented by non-transitory recording media such as systems, devices, methods, integrated circuits, computer programs, or computer-readable CD-ROMs, or by any combination of systems, devices, methods, integrated circuits, computer programs, and recording media.

[0013] Invention Effects

[0014] The encoding device and the like of one of the technical solutions of the present invention can appropriately set information associated with the encoding of moving images. Attached Figure Description

[0015] Figure 1 This is a block diagram showing the functional structure of the encoding device according to Embodiment 1.

[0016] Figure 2 This is a diagram illustrating an example of block segmentation in Implementation Method 1.

[0017] Figure 3 It is a table representing the transformation basis functions corresponding to each transformation type.

[0018] Figure 4A This is a diagram showing an example of the shape of the filter used in ALF.

[0019] Figure 4B This is another example of the shape of the filter used in ALF.

[0020] Figure 4C This is another example of the shape of the filter used in ALF.

[0021] Figure 5A This is a diagram representing the 67 intra-prediction modes of intra-frame prediction.

[0022] Figure 5B This is a flowchart illustrating the outline of predictive image correction processing based on OBMC processing.

[0023] Figure 5C This is a conceptual diagram used to illustrate the outline of predictive image correction processing based on OBMC processing.

[0024] Figure 5D This is a diagram representing an example of FRUC.

[0025] Figure 6 It is a diagram used to illustrate pattern matching (bidirectional matching) between two blocks along a motion trajectory.

[0026] Figure 7 It is a diagram used to illustrate pattern matching (template matching) between a template in the current image and a block in a reference image.

[0027] Figure 8 It is a diagram used to illustrate a model that assumes uniform linear motion.

[0028] Figure 9A It is a diagram used to illustrate the derivation of the motion vectors of sub-block units based on the motion vectors of multiple adjacent blocks.

[0029] Figure 9B This is a diagram used to illustrate the overview of motion vector derivation processing based on the merging mode.

[0030] Figure 9C This is a conceptual diagram used to illustrate the outline of DMVR processing.

[0031] Figure 9D This is a diagram used to illustrate an overview of a predictive image generation method that employs LIC-based brightness correction processing.

[0032] Figure 10 This is a block diagram illustrating the functional structure of the decoding device according to Embodiment 1.

[0033] Figure 11 This is a block diagram showing the structure of the cyclic filtering section of the encoding device in Embodiment 1.

[0034] Figure 12A This is a flowchart illustrating a first specific example of the filter information management steps in Implementation 1.

[0035] Figure 12B This is a flowchart illustrating a first specific example of the filter information setting steps in Implementation Method 1.

[0036] Figure 13A This is a flowchart illustrating a second specific example of the filter information management steps in Implementation Method 1.

[0037] Figure 13B This is a flowchart illustrating a second specific example of the filter information setting steps in Implementation Method 1.

[0038] Figure 14A This is a conceptual diagram representing a first specific example of the reference limitation of filter information in Implementation 1.

[0039] Figure 14B This is a conceptual diagram representing a second specific example of the reference limitation of filter information in Implementation 1.

[0040] Figure 15 This is a block diagram showing the structure of the loop filter section of the decoding device in Embodiment 1.

[0041] Figure 16 This is a flowchart of the first specific example representing the processing order of filter information in the variant mode.

[0042] Figure 17 This is a flowchart of the second specific example of the filter information processing steps in the deformation mode.

[0043] Figure 18 This is a conceptual diagram representing the first specific instance of the PPS notification in the variant mode.

[0044] Figure 19 This is a conceptual diagram representing the second specific instance of the PPS notification in the variant mode.

[0045] Figure 20A This is a conceptual diagram representing the third specific instance of the PPS notification in the variant mode.

[0046] Figure 20B This is a conceptual diagram representing the fourth specific instance of the PPS notification in the variant mode.

[0047] Figure 21A This is a conceptual diagram representing the fifth specific instance of the PPS notification in the variant mode.

[0048] Figure 21B This is a conceptual diagram representing the sixth specific instance of the PPS notification in the variant mode.

[0049] Figure 22 This is a block diagram showing an example of the installation of an encoding device.

[0050] Figure 23 This is a flowchart representing the first action example of the encoding device.

[0051] Figure 24This is a flowchart illustrating the second operation example of the encoding device.

[0052] Figure 25 This is a flowchart representing the third operation example of the encoding device.

[0053] Figure 26 This is a flowchart representing the fourth operation example of the encoding device.

[0054] Figure 27 This is a block diagram illustrating an example of the installation of a decoding device.

[0055] Figure 28 This is a flowchart representing the first action example of the decoding device.

[0056] Figure 29 This is a flowchart illustrating the second action example of the decoding device.

[0057] Figure 30 This is a flowchart representing the third action example of the decoding device.

[0058] Figure 31 This is a flowchart representing the fourth action example of the decoding device.

[0059] Figure 32 This is a diagram showing the overall structure of a content supply system that enables content distribution services.

[0060] Figure 33 This is a diagram illustrating an example of encoding construction in the case of hierarchical encoding.

[0061] Figure 34 This is a diagram illustrating an example of encoding construction in the case of hierarchical encoding.

[0062] Figure 35 This is an example of a web page display.

[0063] Figure 36 This is an example of a web page display.

[0064] Figure 37 This is a diagram illustrating an example of a smartphone.

[0065] Figure 38 This is a block diagram representing a structural example of a smartphone. Detailed Implementation

[0066] (Basic insights of this invention)

[0067] For example, an encoding apparatus for a moving image comprising multiple images can encode an object image from the multiple images by referring to an already encoded image from the multiple images. Similarly, a decoding apparatus for a moving image comprising multiple images can decode an object image from the multiple images by referring to a already decoded image from the multiple images.

[0068] Sometimes, multiple images are assigned separate time IDs representing different levels related to temporal scalability. Time IDs correspond to integer values ​​greater than 0. For example, in encoding an object image, it is prohibited to reference already encoded images with time IDs greater than the object image. This appropriately restricts the already encoded images referenced in the encoding of the object image and reduces the complexity of processing during the encoding process.

[0069] Similarly, in decoding the target image, reference to previously decoded images with a time ID greater than the target image is prohibited. This appropriately limits the previously decoded images referenced in the decoding of the target image and reduces the complexity of processing during decoding. Furthermore, by decoding only a subset of images with time IDs below a specified value, the decoding device can perform interval filtering in motion image decoding, thereby reducing the processing load.

[0070] Furthermore, in the encoding and decoding of moving images, images contained within multiple images are sometimes used as TSA (Temporal Sub-layer Access) images. For example, in the encoding of images that are later than the TSA images in the encoding order, it is prohibited to refer to images that are earlier than the TSA images in the encoding order and have the same or greater time ID compared to the TSA images.

[0071] When a TSA image is decoded, the process can transition from decoding images with smaller time IDs compared to the TSA image to decoding images with the same or larger time IDs compared to the TSA image. This transition towards decoding images with larger time IDs is called "upward shift".

[0072] Images with large time IDs may not be decoded through interval culling. Furthermore, without the constraints of TSA image upshifting, it's possible to reference undecoded images. Therefore, proper upshifting is difficult without the constraints of TSA image upshifting.

[0073] Furthermore, in the encoding and decoding of moving images, there are cases where images contained within multiple images are used as STSA (Step-wise Temporal Sub-layer Access) images. For example, in the encoding of an image that follows the STSA image in the encoding order and has the same time ID as the STSA image, referencing an image that precedes the STSA image in the encoding order and has the same time ID as the STSA image is prohibited.

[0074] Therefore, it is possible to appropriately transition from the state where images with time IDs less than those of the STSA image are decoded to the state where images with time IDs the same as those of the STSA image are decoded.

[0075] That is, in TSA images, the state of decoding is guaranteed from the state where images with smaller time IDs compared to the TSA images are decoded to the state where images with the same or larger time IDs compared to the TSA images are decoded. In STSA images, the state of decoding is guaranteed from the state where images with smaller time IDs compared to the STSA images are decoded to the state where images with the same time IDs compared to the STSA images are decoded.

[0076] Furthermore, various types of information are used in the encoding and decoding of moving images. The filter information in an adaptive cyclic filter is one example of the information used in the encoding and decoding of moving images. An adaptive cyclic filter is a filter used to make the reconstructed image generated in the encoding or decoding of moving images approximate the original image; it is used for image processing such as smoothing or sharpening of the reconstructed image.

[0077] By appropriately setting the filter information, adaptive cyclic filters are applied appropriately to the reconstructed image, enabling proper encoding and decoding of moving images. Conversely, if the filter information is not appropriately set, proper encoding and decoding of moving images cannot be performed. That is, if the information used in the encoding and decoding of moving images is not properly set, proper encoding and decoding of moving images cannot be performed.

[0078] Therefore, for example, an encoding apparatus according to one aspect of the present invention applies an adaptive cyclic filter in the encoding of a moving image comprising multiple images each assigned a time ID representing a level related to time scalability. This apparatus includes circuitry and a memory. The circuitry uses the memory to perform: determining first filter information by referring to second filter information, which is used to apply the adaptive cyclic filter to a first image among the multiple images; the second filter information being associated with a second image among the multiple images that precedes the first image in the encoding order; and applying the adaptive cyclic filter to the first image using the determined first filter information. In the step of determining the first filter information, if the NAL (Network Abstraction Layer) unit type of the first image is a specified NAL unit type, the circuitry may also prevent the reference of third filter information as the second filter information. This third filter information is associated with a third image, which is the image among the multiple images that precedes the first image in the encoding order and has the same time ID as the first image.

[0079] Therefore, the encoding device can determine the first filter information of the first image by referring to the second filter information of the second image. At this time, the encoding device can prevent the third filter information of the third image, which is at the same level as the first image of the specified NAL unit type, from being referenced as the second filter information.

[0080] That is, the encoding device can reference and restrict the filter information of images at the same level as the first image of the specified NAL unit type in the same way that reference restrictions can be applied to images at the same level as those applied to images of the specified NAL unit type. Therefore, the encoding device can associate filter information with images and manage it appropriately, and can appropriately restrict and set the referenced filter information. Thus, the encoding device can appropriately set the information associated with the encoding of moving images.

[0081] Furthermore, for example, in the step of determining the first filter information, the circuit can prevent the fourth filter information from being used as a reference for the second filter information. The fourth filter information is associated with the fourth image, which is the image that comes before the first image in the encoding order among the plurality of images, and is the image with a time ID greater than the first image.

[0082] Therefore, when determining the first filter information of the first image by referring to the second filter information of the second image, it is prohibited to refer to the fourth filter information of the fourth image (whose time ID is greater than that of the first image) as the second filter information. Thus, in terms of temporal scalability, the encoding device can restrict the reference to filter information associated with images in the same way as it restricts reference to images. Therefore, the encoding device can appropriately restrict and set the referenced filter information.

[0083] Additionally, for example, the circuit described above also performs the following steps: determining the fifth filter information by referring to the sixth filter information, which is used to apply an adaptive cyclic filter to the fifth image in the plurality of images that is later in the encoding order than the first image, and the sixth filter information is associated with the sixth image in the plurality of images that is earlier in the encoding order than the fifth image; and applying the adaptive cyclic filter to the fifth image using the determined fifth filter information. In the step of determining the fifth filter information, if the NAL unit type of the first image is the specified NAL unit type, the circuit may also prevent the reference of the third filter information associated with the third image as the sixth filter information.

[0084] Therefore, the encoding device can refer to the sixth filter information of the sixth image to determine the fifth filter information of the fifth image, which is located after the first image in the encoding order. At this time, the encoding device can prevent the third filter information of the third image, which is at the same level as the first image of the specified NAL unit type, from being referenced as the sixth filter information.

[0085] That is, the encoding device can reference and restrict the filter information of images at the same level as the first image, located after the first image of the specified NAL unit type, in the same manner as the reference restrictions that might be applied to images at the same level as the first image. Therefore, the encoding device can associate filter information with images and manage it appropriately, and can appropriately restrict and set the referenced filter information. Thus, the encoding device can appropriately set the information associated with the encoding of moving images.

[0086] Additionally, for example, in the step of determining the fifth filter information, if the NAL unit type of the first image is the specified NAL unit type, the circuit may also prevent the fourth filter information from being referenced as the sixth filter information. The fourth filter information is associated with the fourth image, which is the image that comes before the first image in the encoding order among the plurality of images, and whose time ID is greater than that of the first image.

[0087] Therefore, when determining the fifth filter information of the fifth image by referring to the sixth filter information of the sixth image, it is prohibited to use the fourth filter information of the fourth image with a time ID greater than that of the first image as the reference for the fifth filter information.

[0088] That is, the encoding device can reference and restrict filter information at a location later than the first image of the specified NAL unit type, in the same way that reference restrictions can be applied to images with a time ID greater than the first image. Therefore, the encoding device can associate filter information with images and manage it appropriately, and can appropriately restrict and set the referenced filter information. Thus, the encoding device can appropriately set the information associated with the encoding of the moving image.

[0089] Furthermore, for example, in the step of determining the first filter information, if there is an eighth image between the first image and the seventh image, with the same time ID as the seventh image or with a time ID less than the seventh image, and the seventh image is an image that comes before the first image in the encoding order among the plurality of images and has a time ID greater than 0, and the NAL unit type of the eighth image is the specified NAL unit type, it is also possible to prohibit the seventh filter information associated with the seventh image from being used as a reference for the second filter information.

[0090] Therefore, the encoding device can prevent the 7th filter information of the 7th image, which has the same or larger time ID than the 8th image, from being used as the reference for the 2nd filter information, at a position after the 8th image of the specified NAL unit type.

[0091] That is, the encoding device can reference and restrict the seventh filter information of the seventh image at a point after the eighth image of the specified NAL unit type, in the same manner as the reference restriction that can be applied to the seventh image. Therefore, the encoding device can associate filter information with images and manage it appropriately, and can appropriately restrict and set the referenced filter information. Thus, the encoding device can appropriately set the information associated with the encoding of the moving image.

[0092] In addition, for example, the NAL unit type specified above can also be the NAL unit type of a TSA (Temporal Sub-layer Access) image.

[0093] Therefore, the encoding device can reference and restrict the filter information of images at the same level as TSA images in the same way that reference restrictions might be applied to images at the same level as TSA images. Thus, the encoding device can associate filter information with images and manage it appropriately, and can appropriately restrict and set the referenced filter information. Therefore, the encoding device can appropriately set the information associated with the encoding of moving images.

[0094] Additionally, for example, the circuit described above also performs the following steps: determining the fifth filter information by referring to the sixth filter information, which is used to apply an adaptive cyclic filter to the fifth image in the plurality of images that is later in the encoding order than the first image, and the sixth filter information is associated with the sixth image in the plurality of images that is earlier in the encoding order than the fifth image; and applying an adaptive cyclic filter to the fifth image using the determined fifth filter information. In the step of determining the fifth filter information, if the NAL unit type of the first image is the specified NAL unit type and the time ID of the fifth image is the same as the time ID of the first image, the circuit may also prevent the third filter information associated with the third image from being referenced as the sixth filter information.

[0095] Therefore, the encoding device can refer to the sixth filter information of the sixth image to determine the fifth filter information of the fifth image, which is located after the first image in the encoding order and at the same level as the first image. At this time, the encoding device can prevent the third filter information of the third image, which is at the same level as the first image of the specified NAL unit type, from being referenced as the sixth filter information.

[0096] That is, the encoding device can reference and restrict filter information at a later position than the first image of the specified NAL unit type, but at the same level as the first image, in the same way that reference restrictions can be applied to images preceding the first image. Therefore, the encoding device can associate filter information with images and manage it appropriately, and can appropriately restrict and set the referenced filter information. Thus, the encoding device can appropriately set the information associated with the encoding of moving images.

[0097] In addition, for example, the NAL unit type specified above can be the NAL unit type of an STSA (Step-wise Temporal Sub-layer Access) image.

[0098] Therefore, the encoding device can reference and restrict the filter information of images at the same level as STSA images in the same way that reference restrictions might be applied to images at the same level as STSA images. Thus, the encoding device can associate filter information with images and manage it appropriately, and can appropriately restrict and set the referenced filter information. Therefore, the encoding device can appropriately set the information associated with the encoding of moving images.

[0099] Additionally, for example, a decoding apparatus according to one aspect of the present invention, in decoding a moving image comprising multiple images each assigned a time ID representing a level related to time scalability, applies an adaptive cyclic filter, and includes circuitry and a memory. The circuitry uses the memory to perform the following steps: determining first filter information by referring to second filter information, which is used to apply the adaptive cyclic filter to a first image among the multiple images; the second filter information establishing an association with a second image among the multiple images that precedes the first image in the decoding order; and applying the adaptive cyclic filter to the first image using the determined first filter information. In the step of determining the first filter information, if the NAL (Network Abstraction Layer) cell type of the first image is a specified NAL cell type, the circuitry may also prevent the reference of third filter information as second filter information. This third filter information establishes an association with a third image, which is an image among the multiple images that precedes the first image in the decoding order and has the same time ID as the first image.

[0100] Therefore, the decoding device can determine the first filter information of the first image by referring to the second filter information of the second image. At this time, the decoding device can prevent the third filter information of the third image, which is at the same level as the first image of the specified NAL unit type, from being referenced as the second filter information.

[0101] That is, the decoding device can reference and restrict the filter information of images at the same level as the first image of the specified NAL unit type in the same way that reference restrictions can be applied to images at the same level as the first image of the specified NAL unit type. Therefore, the decoding device can associate filter information with images and manage it appropriately, and can appropriately restrict and set the referenced filter information. Thus, the decoding device can appropriately set the information associated with the decoding of moving images.

[0102] Furthermore, for example, in the step of determining the first filter information, the circuit can prevent the fourth filter information from being used as a reference for the second filter information. The fourth filter information is associated with the fourth image, which is the image that comes before the first image in the decoding order among the plurality of images, and is the image with a time ID greater than the first image.

[0103] Therefore, when determining the first filter information of the first image by referring to the second filter information of the second image, it is prohibited to refer to the fourth filter information of the fourth image (whose time ID is greater than that of the first image) as the second filter information. Thus, in terms of temporal scalability, the decoding device can restrict the reference to filter information associated with an image in the same way as it can restrict the reference to an image. Therefore, the decoding device can appropriately restrict and set the referenced filter information.

[0104] Additionally, for example, the circuit described above also performs the following steps: determining the fifth filter information by referring to the sixth filter information, which is used to apply an adaptive cyclic filter to the fifth image in the plurality of images that is later than the first image in the decoding order; the sixth filter information is associated with the sixth image in the plurality of images that is earlier than the fifth image in the decoding order; and applying an adaptive cyclic filter to the fifth image using the determined fifth filter information. In the step of determining the fifth filter information, if the NAL unit type of the first image is the specified NAL unit type, the circuit may also prevent the reference of the third filter information associated with the third image as the sixth filter information.

[0105] Therefore, the decoding device can refer to the sixth filter information of the sixth image to determine the fifth filter information of the fifth image, which is later than the first image in the decoding order. At this time, the decoding device can prevent the third filter information of the third image, which is at the same level as the first image of the specified NAL unit type, from being referenced as the sixth filter information.

[0106] That is, the decoding device can reference and restrict the filter information of images at the same level as the first image, located after the first image of the specified NAL unit type, in the same way as the reference restrictions that might be applied to images at the same level as the first image. Therefore, the decoding device can associate filter information with images and manage it appropriately, and can appropriately restrict and set the referenced filter information. Thus, the decoding device can appropriately set the information associated with the decoding of moving images.

[0107] Furthermore, for example, in the step of determining the fifth filter information, if the NAL unit type of the first image is the specified NAL unit type, the circuit may also prevent the fourth filter information from being referenced as the sixth filter information. The fourth filter information is associated with the fourth image, which is the image that comes before the first image in the encoding order among the plurality of images, and whose time ID is greater than that of the first image.

[0108] Therefore, when determining the fifth filter information of the fifth image by referring to the sixth filter information of the sixth image, it is prohibited to use the fourth filter information of the fourth image with a time ID greater than that of the first image as the reference for the fifth filter information.

[0109] That is, the decoding device can reference and restrict filter information in the same way that references images with time IDs greater than the first image are referenced at a location later than the first image of the specified NAL unit type. Therefore, the decoding device can associate filter information with images and manage it appropriately, and can appropriately restrict and set the referenced filter information. Thus, the decoding device can appropriately set information associated with the decoding of moving images.

[0110] Furthermore, for example, in the step of determining the first filter information, if there is an eighth image between the first image and the seventh image, with the same time ID as the seventh image or with a smaller time ID than the seventh image, and the NAL unit type of the eighth image is the specified NAL unit type, it is also possible to prohibit the reference of the seventh filter information associated with the seventh image as the second filter information. The seventh image is the image that comes before the first image in the decoding order among the multiple images, and it is an image with a time ID greater than 0.

[0111] Therefore, the decoding device can prevent the 7th filter information of the 7th image, which has the same or larger time ID than the 8th image, from being used as the reference for the 2nd filter information, at a position later than the 8th image of the specified NAL unit type.

[0112] That is, the decoding device can reference and restrict the seventh filter information of the seventh image, located after the eighth image of the specified NAL unit type, in the same manner as the reference restriction that can be applied to the seventh image. Therefore, the decoding device can associate and appropriately manage the filter information with the image, and can appropriately restrict and set the referenced filter information. Thus, the decoding device can appropriately set the information associated with the decoding of the moving image.

[0113] In addition, for example, the NAL unit type specified above can also be the NAL unit type of a TSA (Temporal Sub-layer Access) image.

[0114] Therefore, the decoding device can reference and restrict the filter information of images at the same level as TSA images in the same way that reference restrictions can be applied to images at the same level as TSA images. Thus, the decoding device can associate and appropriately manage filter information with images, and can appropriately restrict and set the referenced filter information. Therefore, the decoding device can appropriately set the information associated with the decoding of moving images.

[0115] Additionally, for example, the circuit described above also performs the following steps: determining the fifth filter information by referring to the sixth filter information, which is used to apply an adaptive cyclic filter to the fifth image in the plurality of images that is later than the first image in the decoding order; the sixth filter information is associated with the sixth image in the plurality of images that is earlier than the fifth image in the decoding order; and applying an adaptive cyclic filter to the fifth image using the determined fifth filter information. In the step of determining the fifth filter information, if the NAL unit type of the first image is the specified NAL unit type and the time ID of the fifth image is the same as the time ID of the first image, the circuit may also prevent the third filter information associated with the third image from being referenced as the sixth filter information.

[0116] Therefore, the decoding device can refer to the sixth filter information of the sixth image to determine the fifth filter information of the fifth image, which is located after the first image in the decoding order and at the same level as the first image. At this time, the decoding device can prevent the third filter information of the third image, which is at the same level as the first image of the specified NAL unit type, from being referenced as the sixth filter information.

[0117] That is, the decoding device can reference and restrict filter information at a location later than and at the same level as the first image of the specified NAL unit type, in the same way that reference restrictions can be applied to images preceding the first image. Therefore, the decoding device can associate filter information with images and manage it appropriately, and can appropriately restrict and set the referenced filter information. Thus, the decoding device can appropriately set information associated with the decoding of moving images.

[0118] In addition, for example, the NAL unit type specified above can be the NAL unit type of an STSA (Step-wise Temporal Sub-layer Access) image.

[0119] Therefore, the decoding device can apply the same reference restriction to the filter information of images at the same level as those at the same level as STSA images. Thus, the decoding device can associate and appropriately manage the filter information with the image, and can appropriately restrict and set the referenced filter information. Therefore, the decoding device can appropriately set the information associated with the decoding of the moving image.

[0120] Additionally, for example, one encoding method of the present invention is an encoding method that applies an adaptive cyclic filter in encoding a moving image comprising multiple images, each assigned a time ID representing a layer related to time scalability. This encoding method includes: a step of determining first filter information by referring to second filter information, the first filter information being used to apply an adaptive cyclic filter to a first image among the multiple images, the second filter information being associated with a second image among the multiple images that precedes the first image in the encoding order; and a step of applying an adaptive cyclic filter to the first image using the determined first filter information. In the step of determining the first filter information, if the NAL (Network Abstraction Layer) cell type of the first image is a specified NAL cell type, third filter information may also be prevented from being referenced as the second filter information. This third filter information is associated with a third image, the third image being the image among the multiple images that precedes the first image in the encoding order and has the same time ID as the first image.

[0121] Therefore, the first filter information of the first image can be determined by referring to the second filter information of the second image. At this time, it is possible to prevent the third filter information of the third image, which is at the same level as the first image layer of the specified NAL unit type, from being used as the reference for the second filter information.

[0122] That is, the filter information of images at the same level as the first image of the specified NAL unit type can be referenced and restricted in the same way as the reference restrictions that can be applied to images at the same level as the first image of the specified NAL unit type. Therefore, filter information can be associated with images and managed appropriately, and the referenced filter information can be appropriately restricted and set. Therefore, information related to the encoding of moving images can be appropriately set.

[0123] Additionally, for example, a decoding method related to one aspect of the present invention is a decoding method that applies an adaptive cyclic filter in the decoding of a moving image comprising multiple images, each assigned a time ID representing a layer related to time scalability. This decoding method includes: a step of determining first filter information by referring to second filter information, the first filter information being used to apply an adaptive cyclic filter to a first image among the multiple images, the second filter information being associated with a second image among the multiple images that precedes the first image in the decoding order; and a step of applying an adaptive cyclic filter to the first image using the determined first filter information. In the step of determining the first filter information, if the NAL (Network Abstraction Layer) cell type of the first image is a specified NAL cell type, it is also possible to prohibit the reference of third filter information as the second filter information. This third filter information is associated with a third image, the third image being the image among the multiple images that precedes the first image in the decoding order and has the same time ID as the first image.

[0124] Therefore, the first filter information of the first image can be determined by referring to the second filter information of the second image. At this time, it is possible to prevent the third filter information of the third image, which is at the same level as the first image layer of the specified NAL unit type, from being used as the reference for the second filter information.

[0125] That is, the filter information of an image at the same level as the first image of the specified NAL unit type can be referenced and restricted in the same way as the reference restrictions that can be applied to images at the same level as the first image of the specified NAL unit type. Therefore, filter information can be associated with and appropriately managed, and the referenced filter information can be appropriately restricted and set. Therefore, information related to the decoding of moving images can be appropriately set.

[0126] Additionally, for example, one encoding apparatus of the present invention is an encoding apparatus that applies an adaptive cyclic filter in encoding a moving image comprising multiple images each assigned a time ID representing a level related to time scalability. The apparatus includes circuitry and a memory. The circuitry uses the memory to perform: a step of encoding multiple parameter sets each assigned 0 as a time ID representing a level related to time scalability, the time ID representing the level related to time scalability; and a step of encoding the image that is first in the encoding order among the multiple images after encoding the multiple parameter sets. Each of the multiple parameter sets corresponds to a multiple level indicated by the multiple time IDs assigned to the multiple images. Each of the multiple parameter sets may also be a parameter set for one or more images among the multiple images assigned a time ID representing the level corresponding to that parameter set.

[0127] Therefore, the encoding device can first aggregate and encode multiple parameter sets corresponding to multiple levels. Furthermore, each parameter set is assigned 0 as a time ID. Thus, multiple parameter sets can be appropriately processed without discarding any. Therefore, the encoding device can appropriately set the information associated with the encoding of the moving image.

[0128] Additionally, for example, the aforementioned multiple images constitute a first image group, the aforementioned multiple parameter sets constitute a first parameter set group, and the aforementioned motion image also includes multiple images constituting a second image group. The circuit further performs the following steps: after encoding the images constituting the first image group, encoding the multiple parameter sets constituting the second parameter set group, wherein each of the multiple parameter sets constituting the second parameter set group is assigned 0 as a time ID representing a level related to the aforementioned time scalability; and after encoding the aforementioned multiple parameter sets constituting the second parameter set group, encoding the images constituting the second image group in the aforementioned multiple images in the encoding order as the first image, wherein each of the aforementioned multiple parameter sets constituting the second parameter set group corresponds to multiple levels indicated by multiple time IDs assigned to the aforementioned multiple images constituting the second image group. The aforementioned multiple parameter sets constituting the second parameter set group may also be parameter sets for one or more images constituting the second image group that are assigned a time ID representing the level corresponding to the parameter set constituting the second parameter set group.

[0129] Therefore, for each image group, the encoding device can first summarize and encode multiple parameter sets corresponding to multiple levels respectively. Thus, the encoding device can appropriately set information associated with the encoding of the moving image for each image group.

[0130] Additionally, for example, a decoding apparatus according to one aspect of the present invention is a decoding apparatus for decoding a moving image comprising multiple images each assigned a time ID representing a level related to time scalability. The apparatus includes a circuit and a memory. The circuit uses the memory to perform: a step of decoding multiple parameter sets each assigned a time ID of 0, where the time ID represents a level related to time scalability; and a step of decoding the image that is the first image in the decoding order among the multiple images after decoding the multiple parameter sets. Each of the multiple parameter sets corresponds to a multiple level indicated by the multiple time IDs assigned to the multiple images. Each of the multiple parameter sets may also be a parameter set for one or more images among the multiple images that are assigned a time ID representing the level corresponding to that parameter set.

[0131] Therefore, the decoding device can first summarize and decode multiple parameter sets corresponding to multiple levels. Furthermore, each parameter set is assigned 0 as a time ID. Thus, multiple parameter sets can be processed appropriately without discarding any. Therefore, the decoding device can appropriately set the information associated with the decoding of the moving image.

[0132] Additionally, for example, the aforementioned multiple images constitute a first image group, the aforementioned multiple parameter sets constitute a first parameter set group, and the aforementioned motion image also includes multiple images constituting a second image group. The circuit further performs: a step of decoding multiple parameter sets constituting the second parameter set group after decoding the images constituting the first image group, wherein each of the multiple parameter sets constituting the second parameter set group is assigned 0 as a time ID representing a level related to the aforementioned time scalability; and a step of decoding the image that is the first in the decoding order among the aforementioned multiple images constituting the second image group after decoding the aforementioned multiple parameter sets constituting the second parameter set group. The aforementioned multiple parameter sets constituting the second parameter set group correspond to multiple levels indicated by multiple time IDs assigned to the aforementioned multiple images constituting the second image group. The aforementioned multiple parameter sets constituting the second parameter set group may also be parameter sets for one or more images among the aforementioned multiple images constituting the second image group, each assigned a time ID representing the level corresponding to the parameter set constituting the second parameter set group.

[0133] Therefore, for each image group, the decoding device can first perform aggregated decoding on multiple parameter sets corresponding to multiple levels. Thus, the decoding device can appropriately set the information associated with the decoding of the moving image for each image group.

[0134] Additionally, for example, one encoding method of the present invention is an encoding method for encoding a moving image comprising multiple images, each assigned a time ID representing a level related to time scalability. This encoding method includes: a step of encoding multiple parameter sets, each assigned 0 as a time ID representing a level related to time scalability; and a step of encoding the first image among the multiple images in the encoding order after encoding the multiple parameter sets. The multiple parameter sets correspond to multiple levels indicated by the multiple time IDs assigned to the multiple images. Each of the multiple parameter sets may also be a parameter set for one or more images among the multiple images that are assigned a time ID representing the level corresponding to that parameter set.

[0135] Therefore, it is possible to first aggregate and encode multiple parameter sets corresponding to multiple levels. Furthermore, each parameter set is assigned 0 as a time ID. Thus, multiple parameter sets can be appropriately processed without discarding any. Therefore, information related to the encoding of motion images can be appropriately set.

[0136] Additionally, for example, one decoding method of the present invention is a decoding method for a motion image comprising multiple images, each assigned a time ID representing a level related to time scalability. This decoding method includes: a step of decoding multiple parameter sets, each assigned 0 as a time ID, where the time ID represents a level related to time scalability; and a step of decoding the image that is the first image in the decoding order among the multiple images after decoding the multiple parameter sets. The multiple parameter sets correspond to multiple levels indicated by the multiple time IDs assigned to the multiple images. Each of the multiple parameter sets may also be a parameter set for one or more images among the multiple images that are assigned a time ID representing the level corresponding to that parameter set.

[0137] Therefore, it is possible to first aggregate and decode multiple parameter sets corresponding to multiple levels. Furthermore, each parameter set is assigned 0 as a time ID. Thus, multiple parameter sets can be appropriately processed without discarding any. Therefore, information related to the decoding of moving images can be appropriately set.

[0138] Additionally, for example, one encoding device of the present invention is an encoding device for encoding a moving image containing multiple images, comprising a circuit and a memory. The circuit uses the memory to perform: a step of encoding a first image among the multiple images; and a step of (i) encoding a parameter set for a second image among the multiple images that is later in the encoding order than the first image after encoding the first image, and encoding the second image after encoding the parameter set, or (ii) encoding the second image without encoding the parameter set after encoding the first image. In the step of performing the first or second action, if the second image is a predetermined image, the circuit performs the first action.

[0139] Therefore, the encoding device can encode the parameter set of a specified image before the specified image is generated. Thus, in actions such as moving the specified image upwards, the parameter set of the specified image can be appropriately processed. Therefore, the encoding device can appropriately set the information associated with the encoding of the moving image.

[0140] In addition, for example, the images specified above can be TSA (Temporal Sub-layer Access) images.

[0141] Therefore, the encoding device can encode the parameter set for the TSA image before the TSA image is generated. Thus, in operations such as upshifting the TSA image, the parameter set for the TSA image can be appropriately processed. Therefore, the encoding device can appropriately set the information associated with the encoding of the moving image.

[0142] Additionally, for example, the aforementioned multiple images are images assigned time IDs representing levels related to time scalability. In the first operation, after encoding the first image, the circuit encodes multiple associated parameter sets, which are multiple parameter sets including the parameter sets for the second image. After encoding the multiple associated parameter sets, the second image is encoded. The multiple associated parameter sets correspond to multiple levels indicated by multiple image time IDs assigned to the second image or higher. The multiple associated parameter sets are parameter sets of one or more images among the multiple images that are assigned time IDs representing the level corresponding to the associated parameter set.

[0143] Therefore, the encoding device can encode multiple parameter sets for multiple images with the same or greater time ID than the specified image, before the specified image is generated. Thus, in actions such as moving images with time IDs greater than the specified image upwards, the parameter sets can be appropriately processed. Therefore, the encoding device can appropriately set information associated with the encoding of moving images.

[0144] In addition, for example, the image specified above can be an STSA (Step-wise Temporal Sub-layer Access) image.

[0145] Therefore, the encoding device can encode the parameter set for the STSA image before the STSA image is generated. Thus, during processes such as upshifting of the STSA image, the parameter set for the STSA image can be appropriately processed. Consequently, the encoding device can appropriately set the information associated with the encoding of the moving image.

[0146] Furthermore, for example, the second image mentioned above could be the image that is encoded following the first image among the plurality of images mentioned above.

[0147] Therefore, the encoding device can appropriately encode the parameter set of the specified image before encoding it. Thus, the parameter set for the specified image can be appropriately processed. Therefore, the encoding device can appropriately set the information associated with the encoding of the moving image.

[0148] Additionally, for example, the aforementioned multiple images are images assigned time IDs representing levels related to time scalability. Before encoding the image that is the first in the encoding order among the aforementioned multiple images, the circuit encodes multiple summary parameter sets that include the aforementioned parameter sets for the second image. The aforementioned multiple summary parameter sets correspond to multiple levels indicated by the multiple time IDs assigned to the aforementioned multiple images. The aforementioned multiple summary parameter sets are parameter sets of one or more images among the aforementioned multiple images that are assigned time IDs representing the levels corresponding to the summary parameter sets.

[0149] Therefore, even if multiple parameter sets containing a parameter set for a given image are encoded first, the encoding device can encode the parameter set for the given image again before the given image. Thus, the parameter set for the given image can be processed appropriately. Therefore, the encoding device can appropriately set the information associated with the encoding of the moving image.

[0150] Additionally, for example, a decoding apparatus according to one technical solution of the present invention is a decoding apparatus for decoding a moving image containing multiple images, comprising a circuit and a memory. The circuit utilizes the memory to perform: a step of decoding a first image among the multiple images; and a step of performing (i) after decoding the first image, decoding a parameter set for a second image among the multiple images that is later in the decoding order than the first image, and after decoding the parameter set, decoding the second image, or (ii) after decoding the first image, not decoding the parameter set, but decoding the second image. In the step of performing the first or second action, if the second image is a predetermined image, the circuit performs the first action.

[0151] Therefore, the decoding device can decode the parameter set of a specified image before the specified image is rendered. Thus, in actions such as moving the specified image upwards, the parameter set of the specified image can be appropriately processed. Therefore, the decoding device can appropriately set the information associated with the decoding of the moving image.

[0152] In addition, for example, the image specified above can be a TSA (Temporal Sub-layer Access) image.

[0153] Therefore, the decoding device can decode the parameter set for the TSA image before the TSA image is generated. Thus, in operations such as upshifting the TSA image, the parameter set for the TSA image can be appropriately processed. Therefore, the decoding device can appropriately set the information associated with the decoding of the moving image.

[0154] Additionally, for example, the aforementioned multiple images are images assigned time IDs representing levels related to time scalability. In the first operation, after decoding the first image, the circuit decodes multiple associated parameter sets, which include multiple parameter sets as parameter sets for the second image. After decoding the multiple associated parameter sets, the circuit decodes the second image. The multiple associated parameter sets correspond to multiple levels indicated by multiple time IDs assigned to the second image above the time ID. The multiple associated parameter sets are parameter sets of one or more images among the multiple images that are assigned time IDs representing the level corresponding to the associated parameter set.

[0155] Therefore, the decoding device can decode multiple parameter sets for multiple images with the same time ID as the specified image or with a time ID greater than the specified image, before the specified image is rendered. Thus, in tasks such as shifting images with time IDs greater than the specified image, the parameter sets can be appropriately processed. Consequently, the decoding device can appropriately set information associated with the decoding of moving images.

[0156] In addition, for example, the images specified above can be STSA (Step-wise Temporal Sub-layer Access) images.

[0157] Therefore, the decoding device can decode the parameter set for the STSA image before the STSA image is rendered. Thus, during processes such as upshifting of the STSA image, the parameter set for the STSA image can be appropriately processed. Consequently, the decoding device can appropriately set the information associated with the decoding of the moving image.

[0158] Furthermore, for example, the second image mentioned above could be the image that is decoded following the first image among the aforementioned multiple images.

[0159] Therefore, the decoding device can appropriately decode the parameter set of a given image before decoding it. Thus, the parameter set for the given image can be appropriately processed. Consequently, the decoding device can appropriately set the information associated with the decoding of the moving image.

[0160] Additionally, for example, the aforementioned multiple images are images assigned time IDs representing levels related to time scalability. Before decoding the image that is the first in the decoding order among the aforementioned multiple images, the circuit decodes multiple summary parameter sets that include the aforementioned parameter sets as the second image. The aforementioned multiple summary parameter sets correspond to multiple levels indicated by the multiple time IDs assigned to the aforementioned multiple images. The aforementioned multiple summary parameter sets are parameter sets of one or more images among the aforementioned multiple images that are assigned time IDs representing the levels corresponding to the summary parameter sets.

[0161] Therefore, even if multiple parameter sets containing the parameter set for a given image are decoded first, the decoding device can decode the parameter set for the given image again before the given image. Thus, the parameter set for the given image can be processed appropriately. Therefore, the decoding device can appropriately set the information associated with the decoding of the moving image.

[0162] Additionally, for example, one encoding method of the present invention is an encoding method for encoding a moving image containing multiple images. The encoding method includes: a step of encoding a first image among the multiple images; and a step of performing (i) encoding a parameter set for a second image among the multiple images that is later in the encoding order than the first image after encoding the first image, and encoding the second image after encoding the parameter set, or (ii) encoding the second image without encoding the parameter set after encoding the first image. In the step of performing the first action or the second action, if the second image is a predetermined image, the first action can be performed.

[0163] Therefore, the parameter set of a given image can be encoded before the given image is defined. Thus, in operations such as moving the given image upwards, the parameter set of the given image can be appropriately processed. Therefore, information related to the encoding of moving images can be appropriately set.

[0164] Furthermore, for example, a decoding method for a technical solution of the present invention is a decoding method for decoding a moving image containing multiple images. The decoding method includes: a step of decoding a first image among the multiple images; and a step of performing (i) after decoding the first image, decoding a parameter set for a second image among the multiple images that is later in the decoding order than the first image, and decoding the second image after decoding the parameter set, or (ii) after decoding the first image, decoding the second image without decoding the parameter set. In the step of performing the first or second action, if the second image is a predetermined image, the first action can be performed.

[0165] Therefore, the parameter set of a given image can be decoded before the given image is rendered. Thus, in operations such as moving the given image upwards, the parameter set of the given image can be appropriately processed. Therefore, information related to the decoding of moving images can be appropriately set.

[0166] Additionally, for example, one encoding apparatus of the present invention is an encoding apparatus that encodes a moving image comprising multiple images, each assigned a time ID representing a hierarchy related to time scalability, and includes circuitry and a memory. The circuitry uses the memory to perform: a step of encoding a first image among the multiple images; and a step of performing (i) encoding a parameter set for a second image among the multiple images that is later in the encoding order than the first image after encoding the first image, and then encoding the second image after encoding the parameter set, or (ii) encoding the second image without encoding the parameter set after encoding the first image. The circuitry can perform the first action if, during the first or second action, the time ID assigned to the second image is greater than the smallest time ID among the multiple time IDs assigned to the multiple images, but less than the largest time ID among the multiple time IDs.

[0167] Therefore, the encoding device can encode the parameter set for the intermediate layer image before the intermediate layer image. Thus, in processes such as moving the intermediate layer image upwards, the parameter set for the intermediate layer image can be appropriately processed. Therefore, the encoding device can appropriately set the information associated with the encoding of the moving image.

[0168] Furthermore, for example, in the step of performing the first action or the second action, the circuit can perform the first action even if the time ID assigned to the second image is the largest time ID.

[0169] Therefore, the encoding device can encode the parameter set for the topmost image before the topmost image. Thus, during processes such as moving the topmost image upwards, the parameter set for the topmost image can be appropriately processed. Therefore, the encoding device can appropriately set the information associated with the encoding of the moving image.

[0170] Additionally, for example, when the circuit performs the first operation if the time ID assigned to the second image is the second largest time ID among the plurality of time IDs, after encoding the first image, it encodes the parameter set and the topmost parameter set, and after encoding the parameter set and the topmost parameter set, it encodes the second image. The topmost parameter set may be a parameter set for one or more images assigned the largest time ID.

[0171] Thus, the encoding device can efficiently encode two parameter sets, which contain the parameter set for the topmost image.

[0172] Additionally, for example, in the step of performing the first action or the second action described above, if the time ID assigned to the second image is the maximum time ID, the circuit may also perform the second action described above.

[0173] Therefore, the encoding device can omit encoding the parameter set of the topmost image before the topmost image.

[0174] Additionally, for example, in the step of performing the first action or the second action described above, if the specified flag included in the sequence parameter set of the plurality of images is a specified value, and if the time ID assigned to the second image is greater than the minimum time ID and less than the maximum time ID, the circuit may also perform the first action described above.

[0175] Thus, the encoding device is able to appropriately encode the parameter set for the intermediate layer image in a prescribed sequence, prior to the intermediate layer image.

[0176] Furthermore, for example, if the specified flag is the specified value, and the time ID assigned to the image is not the minimum time ID, the multiple images may also be TSA (Temporal Sub-Layer Access) images.

[0177] Thus, the encoding device is able to appropriately encode the parameter set for the intermediate layer images in a prescribed sequence consisting of TSA images except for the lowest layer, before the intermediate layer images.

[0178] Additionally, for example, a decoding apparatus according to one aspect of the present invention is a decoding apparatus for decoding a moving image comprising multiple images, each assigned a time ID representing a level related to time scalability. It includes a circuit and a memory. The circuit uses the memory to perform: a step of decoding a first image among the multiple images; and a step of performing (i) after decoding the first image, decoding a parameter set for a second image among the multiple images that is later in the decoding order than the first image, and then decoding the second image after decoding the parameter set, or (ii) after decoding the first image, not decoding the parameter set but decoding the second image. In performing either the first or second step, the circuit can perform the first step if the time ID assigned to the second image is greater than the smallest time ID among the multiple time IDs assigned to the multiple images and less than the largest time ID among the multiple time IDs.

[0179] Therefore, the decoding device can decode the parameter set for the intermediate layer image before the intermediate layer image. Thus, in processes such as moving the intermediate layer image upwards, the parameter set for the intermediate layer image can be appropriately processed. Therefore, the decoding device can appropriately set the information associated with the decoding of the moving image.

[0180] Furthermore, for example, in the step of performing the first action or the second action, the circuit can perform the first action even if the time ID assigned to the second image is the largest time ID.

[0181] Therefore, the decoding device can decode the parameter set for the topmost image before the topmost image. Thus, in processes such as moving the topmost image upwards, the parameter set for the topmost image can be appropriately processed. Therefore, the decoding device can appropriately set the information associated with decoding the moving image.

[0182] Furthermore, for example, when the first operation is performed when the time ID assigned to the second image is the second largest time ID among the plurality of time IDs, the circuit decodes the parameter set and the topmost parameter set after decoding the first image, and decodes the second image after decoding the parameter set and the topmost parameter set. The topmost parameter set may be a parameter set for one or more images assigned the largest time ID.

[0183] Therefore, the decoding device can efficiently decode two parameter sets, which contain the parameter set for the topmost image.

[0184] Additionally, for example, in the step of performing the first action or the second action described above, if the time ID assigned to the second image is the maximum time ID, the circuit may also perform the second action described above.

[0185] Therefore, the decoding device can omit decoding the parameter set for the topmost image before the topmost image.

[0186] Additionally, for example, in the step of performing the first action or the second action described above, if the specified flag included in the sequence parameter set of the plurality of images is a specified value, and if the time ID assigned to the second image is greater than the minimum time ID and less than the maximum time ID, the circuit may also perform the first action described above.

[0187] Thus, the decoding device is able to appropriately decode the parameter set for the intermediate layer image in a specified sequence, before the intermediate layer image.

[0188] Furthermore, for example, if the specified flag is the specified value, and the time ID assigned to the image is not the minimum time ID, the multiple images may also be TSA (Temporal Sub-Layer Access) images.

[0189] Thus, the decoding device is able to appropriately decode the parameter set for the intermediate layer images in a prescribed sequence consisting of TSA images except for the lowest layer, before the intermediate layer images.

[0190] Additionally, for example, one encoding method of the present invention is an encoding method for encoding a motion image comprising multiple images, each assigned a time ID representing a hierarchy related to time scalability. This encoding method includes: a step of encoding a first image among the multiple images; and a step of performing (i) encoding a parameter set for a second image among the multiple images that is later in the encoding order than the first image after encoding the first image, and encoding the second image after encoding the parameter set, or (ii) encoding the second image after encoding the first image without encoding the parameter set. In performing either the first or second step, the first step may be performed if the time ID assigned to the second image is greater than the smallest time ID among the multiple time IDs assigned to the multiple images, and less than the largest time ID among the multiple time IDs.

[0191] Therefore, the parameter set of the intermediate layer image can be encoded before the intermediate layer image. Thus, in operations such as shifting the intermediate layer image upwards, the parameter set of the intermediate layer image can be appropriately processed. Therefore, information related to the encoding of moving images can be appropriately set.

[0192] Additionally, for example, a decoding method related to one technical solution of the present invention is a decoding method for decoding a motion image comprising multiple images, each assigned a time ID representing a hierarchy related to time scalability. This decoding method includes: a step of decoding a first image among the multiple images; and a first action of (i) decoding a parameter set for a second image among the multiple images that is later in the decoding order than the first image after decoding the first image, and then decoding the second image after decoding the parameter set, or (ii) decoding the second image without decoding the parameter set after decoding the first image. In the step of performing the first or second action, the first action may be performed if the time ID assigned to the second image is greater than the smallest time ID among the multiple time IDs assigned to the multiple images, and less than the largest time ID among the multiple time IDs.

[0193] Therefore, it is possible to decode the parameter set for the intermediate layer image before the intermediate layer image. Thus, in processes such as shifting the intermediate layer image upwards, the parameter set for the intermediate layer image can be appropriately processed. Therefore, information related to the decoding of moving images can be appropriately set.

[0194] Furthermore, for example, an encoding apparatus according to a technical solution of the present invention may include a segmentation unit, an intra-frame prediction unit, an inter-frame prediction unit, a transform unit, a quantization unit, an entropy coding unit, and a filter unit.

[0195] The segmentation unit described above can segment an image into multiple blocks. The intra-frame prediction unit described above can perform intra-frame prediction on blocks contained within the multiple blocks. The inter-frame prediction unit described above can perform inter-frame prediction on the blocks. The transform unit described above can transform the prediction error between the predicted image obtained through the intra-frame prediction or the inter-frame prediction and the original image, generating transform coefficients. The quantization unit described above can quantize the transform coefficients and generate quantization coefficients. The entropy coding unit described above can encode the quantization coefficients and generate a coded bitstream. The filter unit described above can also apply filters to the reconstructed image generated using the predicted image.

[0196] Alternatively, for example, the above-described encoding apparatus may also be an encoding apparatus that applies an adaptive cyclic filter in the encoding of a moving image comprising multiple images, each assigned a time ID representing a hierarchy related to time scalability.

[0197] Furthermore, the aforementioned filter unit performs the following steps: determining the first filter information by referring to the second filter information, the first filter information being used to apply an adaptive cyclic filter to the first image among the plurality of images, the second filter information being associated with the second image among the plurality of images that precedes the first image in the encoding order; and applying the adaptive cyclic filter to the first image using the determined first filter information.

[0198] Furthermore, in the step of determining the first filter information, if the NAL unit type of the first image is a specified NAL unit type, the filter unit may also prohibit the reference of the third filter information as the second filter information. The third filter information is associated with the third image, which is the image that comes before the first image in the encoding order among the plurality of images, and is the image with the same time ID as the first image.

[0199] Alternatively, for example, the encoding device described above can also be an encoding device that encodes a moving image containing multiple images, each assigned a time ID representing a hierarchy related to time scalability.

[0200] Furthermore, the entropy encoding unit described above can also perform the following steps: encoding multiple parameter sets, each assigned 0 as a time ID representing a level related to the aforementioned time scalability; and encoding the first image among the multiple images in the encoding order after encoding the multiple parameter sets.

[0201] Furthermore, the aforementioned multiple parameter sets correspond to multiple levels indicated by multiple time IDs assigned to the aforementioned multiple images. The aforementioned multiple parameter sets can also be parameter sets for one or more images among the multiple images that are assigned time IDs representing the levels corresponding to the aforementioned parameter sets.

[0202] Alternatively, for example, the encoding device described above can also be an encoding device for encoding a moving image containing multiple images.

[0203] Furthermore, the entropy encoding unit performs the following steps: encoding the first image among the plurality of images; and (i) encoding the parameter set of the second image among the plurality of images that is later in the encoding order than the first image after encoding the first image, and encoding the second image after encoding the parameter set, or (ii) encoding the second image without encoding the parameter set after encoding the first image.

[0204] Furthermore, the entropy encoding unit may perform the first action or the second action if the second image is a predetermined image.

[0205] Alternatively, for example, the encoding device described above can also be an encoding device that encodes a moving image containing multiple images, each assigned a time ID representing a hierarchy related to time scalability.

[0206] Furthermore, the entropy encoding unit performs the following steps: encoding the first image among the plurality of images; and (i) encoding the parameter set of the second image among the plurality of images that is later in the encoding order than the first image after encoding the first image, and encoding the second image after encoding the parameter set, or (ii) encoding the second image without encoding the parameter set after encoding the first image.

[0207] Furthermore, in the step of performing the first action or the second action, the entropy encoding unit may perform the first action if the time ID assigned to the second image is greater than the smallest time ID among the multiple time IDs assigned to the multiple images and less than the largest time ID among the multiple time IDs.

[0208] Furthermore, for example, a decoding apparatus according to a technical solution of the present invention may include an entropy decoding unit, an inverse quantization unit, an inverse transform unit, an intra-frame prediction unit, an inter-frame prediction unit, and a filter unit.

[0209] The entropy decoding unit described above can decode the quantization coefficients of blocks within an image from the encoded bitstream. The inverse quantization unit can perform inverse quantization on the quantization coefficients to obtain transform coefficients. The inverse transform unit can perform inverse transform on the transform coefficients to obtain prediction errors. The intra-frame prediction unit can perform intra-frame prediction on the blocks. The inter-frame prediction unit can perform inter-frame prediction on the blocks. The filter unit can apply filters to the reconstructed image generated using the prediction images obtained through the intra-frame prediction or inter-frame prediction and the prediction errors.

[0210] Alternatively, for example, the decoding device described above may also be a decoding device that applies an adaptive cyclic filter in the decoding of a moving image comprising multiple images, each assigned a time ID representing a hierarchy related to time scalability.

[0211] Furthermore, the aforementioned filter unit performs the following steps: determining the first filter information by referring to the second filter information, the first filter information being used to apply an adaptive cyclic filter to the first image among the plurality of images, the second filter information being associated with the second image among the plurality of images that precedes the first image in the decoding order; and applying the adaptive cyclic filter to the first image using the determined first filter information.

[0212] Furthermore, in the step of determining the first filter information, if the NAL unit type of the first image is a specified NAL unit type, the filter unit may also prohibit the reference of the third filter information as the second filter information. The third filter information is associated with the third image, which is the image that comes before the first image in the decoding order among the plurality of images, and is the image with the same time ID as the first image.

[0213] Furthermore, for example, the aforementioned decoding device may also be a decoding device for decoding a moving image containing multiple images, each assigned a time ID representing a hierarchy related to time scalability.

[0214] Furthermore, the aforementioned entropy decoding unit can also perform the following steps: decoding multiple parameter sets, each assigned 0 as a time ID, where the time ID represents a level related to the aforementioned time scalability; and after decoding the multiple parameter sets, decoding the first image among the multiple images in the decoding order.

[0215] Furthermore, the aforementioned parameter sets correspond to the multiple levels indicated by the multiple time IDs assigned to the aforementioned multiple images, and the aforementioned parameter sets can also be parameter sets for one or more images among the aforementioned multiple images that are assigned time IDs representing the levels corresponding to the parameter sets.

[0216] Furthermore, for example, the aforementioned decoding device can also be a decoding device for decoding moving images containing multiple images.

[0217] Furthermore, the entropy decoding unit described above can perform the following steps: decoding the first image among the plurality of images; and performing the following steps: (i) after decoding the first image, decoding the parameter set of the second image among the plurality of images that is later in the decoding order than the first image, and decoding the second image after decoding the parameter set; or (ii) after decoding the first image, decoding the second image without decoding the parameter set.

[0218] Furthermore, when the entropy decoding unit performs the first action or the second action, if the second image is a predetermined image, the first action can be performed.

[0219] Furthermore, for example, the aforementioned decoding device may also be a decoding device for decoding a moving image containing multiple images, each assigned a time ID representing a hierarchy related to time scalability.

[0220] Furthermore, the entropy decoding unit described above can perform the following steps: decoding the first image among the plurality of images; and performing the following steps: (i) after decoding the first image, decoding the parameter set of the second image among the plurality of images that is later in the decoding order than the first image, and decoding the second image after decoding the parameter set; or (ii) after decoding the first image, decoding the second image without decoding the parameter set.

[0221] Furthermore, in the step of performing the first action or the second action, the entropy decoding unit may perform the first action if the time ID assigned to the second image is greater than the smallest time ID among the multiple time IDs assigned to the multiple images and less than the largest time ID among the multiple time IDs.

[0222] Moreover, these inclusive or specific technical solutions can be implemented by systems, devices, methods, integrated circuits, computer programs, or recording media such as computer-readable CD-ROMs, or by any combination of systems, devices, methods, integrated circuits, computer programs, and recording media.

[0223] Hereinafter, the embodiments will be described in detail with reference to the accompanying drawings.

[0224] Furthermore, the embodiments described below are inclusive or specific examples. The numerical values, shapes, materials, constituent elements, arrangements and connection methods of constituent elements, steps, and order of steps shown in the following embodiments are examples and are not intended to limit the scope of the claims. In addition, any constituent elements in the following embodiments that are not described in the independent claim representing the highest-level concept are described as arbitrary constituent elements.

[0225] (Implementation Method 1)

[0226] First, as an example of an encoding and decoding apparatus for the processing and / or structure described in the various embodiments of the present invention described later, an outline of Embodiment 1 will be described. However, Embodiment 1 is merely an example of an encoding and decoding apparatus for the processing and / or structure described in the various embodiments of the present invention, and the processing and / or structure described in the various embodiments of the present invention can also be implemented in encoding and decoding apparatuses different from Embodiment 1.

[0227] When applying the processing and / or structure described in various aspects of the present invention to Embodiment 1, one of the following may also be performed, for example.

[0228] (1) For the encoding or decoding device of Embodiment 1, the constituent element that corresponds to the constituent element described in each aspect of the present invention is replaced with the constituent element described in each aspect of the present invention.

[0229] (2) For the encoding or decoding device of Embodiment 1, after any modification such as adding, replacing, or deleting any of the constituent elements of the plurality of constituent elements constituting the encoding or decoding device, the constituent elements corresponding to the constituent elements described in each aspect of the present invention are replaced with the constituent elements described in each aspect of the present invention.

[0230] (3) After adding processing to the method implemented by the encoding or decoding device of Embodiment 1, and / or replacing or deleting any of the processing among the multiple processing included in the method, the processing corresponding to the processing described in each aspect of the present invention is replaced with the processing described in each aspect of the present invention.

[0231] (4) A portion of the constituent elements constituting the encoding or decoding apparatus of Embodiment 1 are combined with constituent elements described in various aspects of the present invention, a portion of constituent elements having the functions of constituent elements described in various aspects of the present invention, or a portion of constituent elements implementing the processing performed by constituent elements described in various aspects of the present invention.

[0232] (5) A component having a portion of the functions of a portion of the components constituting the encoding or decoding apparatus of embodiment 1, or a component implementing a portion of the processing performed by a portion of the components constituting the encoding or decoding apparatus of embodiment 1, is combined with the components described in various aspects of the present invention, the components having a portion of the functions of the components described in various aspects of the present invention, or the components implementing a portion of the processing performed by the components described in various aspects of the present invention.

[0233] (6) For the method implemented by the encoding or decoding device of Embodiment 1, the processing that corresponds to the processing described in each aspect of the present invention among the multiple processing included in the method is replaced with the processing described in each aspect of the present invention.

[0234] (7) A portion of the processing included in the method implemented by the encoding or decoding apparatus of Embodiment 1 is combined with the processing described in the various aspects of the present invention.

[0235] Furthermore, the implementation of the processes and / or structures described in the various embodiments of the present invention is not limited to the examples described above. For example, it may be implemented in an apparatus used for a different purpose than the moving image / image encoding apparatus or moving image / image decoding apparatus disclosed in Embodiment 1, or the processes and / or structures described in each embodiment may be implemented individually. In addition, the processes and / or structures described in different embodiments may be combined and implemented.

[0236] [Overview of the encoding device]

[0237] First, an overview of the encoding device for Embodiment 1 will be provided. Figure 1 This is a block diagram illustrating the functional structure of the encoding apparatus 100 according to Embodiment 1. The encoding apparatus 100 is a motion picture / image encoding apparatus that encodes motion pictures / images in block units.

[0238] like Figure 1 As shown, the encoding device 100 is a device for encoding images in block units, and includes a segmentation unit 102, a subtraction unit 104, a transformation unit 106, a quantization unit 108, an entropy encoding unit 110, an inverse quantization unit 112, an inverse transformation unit 114, an addition unit 116, a block memory 118, a cyclic filtering unit 120, a frame memory 122, an intra-frame prediction unit 124, an inter-frame prediction unit 126, and a prediction control unit 128.

[0239] The encoding device 100 is implemented, for example, by a general-purpose processor and memory. In this case, when the software program stored in the memory is executed by the processor, the processor functions as the segmentation unit 102, subtraction unit 104, transform unit 106, quantization unit 108, entropy coding unit 110, inverse quantization unit 112, inverse transform unit 114, addition unit 116, cyclic filtering unit 120, intra-frame prediction unit 124, inter-frame prediction unit 126, and prediction control unit 128. Alternatively, the encoding device 100 may be implemented as one or more dedicated electronic circuits corresponding to the segmentation unit 102, subtraction unit 104, transform unit 106, quantization unit 108, entropy coding unit 110, inverse quantization unit 112, inverse transform unit 114, addition unit 116, cyclic filtering unit 120, intra-frame prediction unit 124, inter-frame prediction unit 126, and prediction control unit 128.

[0240] The following describes the constituent elements included in the encoding device 100.

[0241] [Divider]

[0242] The segmentation unit 102 divides each image contained in the input moving image into multiple blocks and outputs each block to the subtraction unit 104. For example, the segmentation unit 102 first segments the image into fixed-size blocks (e.g., 128×128). These fixed-size blocks may be called coding tree units (CTUs). Furthermore, the segmentation unit 102 divides each fixed-size block into variable-size blocks (e.g., 64×64 or less) based on recursive quadtree and / or binary tree block segmentation. These variable-size blocks may be called coding units (CUs), prediction units (PUs), or transform units (TUs). In addition, in this embodiment, it is not necessary to distinguish between CUs, PUs, and TUs, and some or all of the blocks in the image may be used as processing units of CUs, PUs, and TUs.

[0243] Figure 2 This is a diagram illustrating an example of block segmentation in Implementation Method 1. In Figure 2 In the diagram, solid lines represent block boundaries based on quadtree block partitioning, and dashed lines represent block boundaries based on binary tree block partitioning.

[0244] Here, block 10 is a square block of 128×128 pixels (128×128 block). This 128×128 block 10 is first divided into 4 square blocks of 64×64 (quadtree block partitioning).

[0245] The 64×64 block in the upper left corner is then vertically divided into two rectangular 32×64 blocks, and the 32×64 block on the left is then vertically divided into two rectangular 16×64 blocks (binary tree block partitioning). As a result, the 64×64 block in the upper left corner is divided into two 16×64 blocks (11 and 12) and a 32×64 block (13).

[0246] The 64×64 block in the upper right corner is horizontally divided into two rectangular 64×32 blocks, 14 and 15 (binary tree block division).

[0247] The 64×64 block in the lower left corner is divided into four 32×32 square blocks (quadtree block partitioning). The upper left and lower right blocks of these four 32×32 blocks are further partitioned. The upper left 32×32 block is vertically divided into two 16×32 rectangular blocks, and the right 16×32 block is horizontally divided into two 16×16 blocks (binary tree block partitioning). The lower right 32×32 block is horizontally divided into two 32×16 blocks (binary tree block partitioning). As a result, the lower left 64×64 block is divided into 16×32 block 16, two 16×16 blocks 17 and 18, two 32×32 blocks 19 and 20, and two 32×16 blocks 21 and 22.

[0248] The 64×64 block 23 in the lower right corner is not divided.

[0249] As described above, in Figure 2 In the example, block 10 is divided into 13 variable-size blocks 11 to 23 based on recursive quadtree and binary tree block partitioning. Such partitioning is sometimes referred to as QTBT (quadtree plus binary tree) partitioning.

[0250] In addition, Figure 2 In this context, a block can be divided into 2 or 4 blocks (quadtree or binary tree block partitioning), but the partitioning is not limited to these. For example, a block can also be divided into 3 blocks (ternary tree partitioning). Partitioning including such ternary tree partitioning is sometimes referred to as MBT (multi-type tree) partitioning.

[0251] [Subtraction Section]

[0252] The subtraction unit 104 subtracts the prediction signal (prediction sample) from the original signal (original sample) in block units divided by the segmentation unit 102. That is, the subtraction unit 104 calculates the prediction error (also called residual) of the encoded target block (hereinafter referred to as the current block). Furthermore, the subtraction unit 104 outputs the calculated prediction error to the transformation unit 106.

[0253] The original signal is the input signal of the encoding device 100, which is the signal representing the image of each picture that constitutes the moving image (e.g., luminance signal and two chroma signals). Hereinafter, the signal representing the image may also be referred to as a sample.

[0254] [Transformation Section]

[0255] The transformation unit 106 transforms the prediction error in the spatial domain into transformation coefficients in the frequency domain, and outputs the transformation coefficients vectorization unit 108. Specifically, the transformation unit 106 performs a preset discrete cosine transform (DCT) or discrete sine transform (DST) on the prediction error in the spatial domain, for example.

[0256] Alternatively, the transform unit 106 can adaptively select a transform type from multiple transform types and use the transform basis function corresponding to the selected transform type to transform the prediction error into transform coefficients. Such a transform is sometimes referred to as EMT (explicit multiple core transform) or AMT (adaptive multiple transform).

[0257] Several transformation types include, for example, DCT-II, DCT-V, DCT-VIII, DST-I, and DST-VII. Figure 3 This is a table representing the transformation basis functions corresponding to each transformation type. Figure 3 In this context, N represents the number of input pixels. The choice of transform type from these multiple transform types can depend on the type of prediction (intra-frame prediction and inter-frame prediction) or the intra-frame prediction mode.

[0258] Information indicating whether such EMT or AMT is applied (e.g., referred to as the AMT flag) and information indicating the selected transform type are signaled at the CU level. Furthermore, the signaling of this information is not limited to the CU level and can also be at other levels (e.g., sequence level, image level, slice level, tile level, or CTU level).

[0259] Furthermore, the transform unit 106 can also perform a re-transformation on the transform coefficients (transformation results). Such a re-transformation may be referred to as AST (adaptive secondary transform) or NSST (non-separable secondary transform). For example, the transform unit 106 performs a re-transformation on each sub-block (e.g., a 4×4 sub-block) contained in the block of transform coefficients corresponding to the intra-frame prediction error. Information indicating whether NSST is applied and information related to the transform matrix used in NSST are signaled at the CU level. In addition, the signaling of this information is not limited to the CU level and can also be at other levels (e.g., sequence level, picture level, slice level, tile level, or CTU level).

[0260] Here, a separable transformation refers to a method of performing multiple transformations in each direction, which is equivalent to the number of dimensions of the input. A non-separable transformation refers to a method of treating two or more dimensions together as one dimension and transforming them together when the input is multidimensional.

[0261] For example, as one example of a non-separable transformation, one can cite the way that when the input is a 4×4 block, it is treated as a permutation of 16 elements, and the permutation is transformed using a 16×16 transformation matrix.

[0262] Furthermore, the Hypercube Givens Transform, which treats a 4×4 input block as a permutation of 16 elements and then performs multiple Givens rotations on that permutation, is also an example of a non-separable transformation.

[0263] [Quantitative Department]

[0264] The quantization unit 108 quantizes the transform coefficients output from the transform unit 106. Specifically, the quantization unit 108 scans the transform coefficients of the current block in a predetermined scan order and quantizes the transform coefficients based on the quantization parameters (QP) corresponding to the scanned transform coefficients. Furthermore, the quantization unit 108 outputs the quantized transform coefficients (hereinafter referred to as quantized coefficients) of the current block to the entropy encoding unit 110 and the inverse quantization unit 112.

[0265] The specified order is the order in which the transform coefficients are quantized / dequantized. For example, the specified scan order is defined by ascending frequency (from low frequency to high frequency) or descending frequency (from high frequency to low frequency).

[0266] The quantization parameter is a parameter that defines the quantization step size (quantization width). For example, if the value of the quantization parameter increases, the quantization step size also increases. That is, if the value of the quantization parameter increases, the quantization error increases.

[0267] [Entropy Coding Department]

[0268] The entropy coding unit 110 generates a coded signal (coded bitstream) by performing variable-length coding on the quantization coefficients, which are input from the quantization unit 108. Specifically, the entropy coding unit 110 performs arithmetic coding on the binary signal, for example, by binarizing the quantization coefficients.

[0269] [De-quantization Department]

[0270] The inverse quantization unit 112 performs inverse quantization on the quantization coefficients that are input from the quantization unit 108. Specifically, the inverse quantization unit 112 performs inverse quantization on the quantization coefficients of the current block in a predetermined scan order. Furthermore, the inverse quantization unit 112 outputs the inverse quantized transform coefficients of the current block to the inverse transform unit 114.

[0271] [Inverse Transformation Section]

[0272] The inverse transform unit 114 restores the prediction error by performing an inverse transform on the transform coefficients, which are input from the inverse quantization unit 112. Specifically, the inverse transform unit 114 restores the prediction error of the current block by performing an inverse transform on the transform coefficients corresponding to the transform of the transform unit 106. Furthermore, the inverse transform unit 114 outputs the restored prediction error to the adder unit 116.

[0273] Furthermore, the restored prediction error differs from the prediction error calculated by the subtraction unit 104 because information was lost during quantization. In other words, the restored prediction error includes quantization error.

[0274] [Addition Department]

[0275] The addition unit 116 reconstructs the current block by adding the prediction error, which is input from the inverse transform unit 114, to the prediction sample, which is input from the prediction control unit 128. Furthermore, the addition unit 116 outputs the reconstructed block to the block memory 118 and the cyclic filtering unit 120. The reconstructed block may be referred to as a local decoding block.

[0276] [Block Memory]

[0277] Block memory 118 is a storage unit used to save blocks within the encoded object image (hereinafter referred to as the current image) referenced in intra-frame prediction. Specifically, block memory 118 saves the reconstructed blocks output from addition unit 116.

[0278] [Loop Filtering Section]

[0279] The cyclic filtering unit 120 applies cyclic filtering to the block reconstructed by the addition unit 116 and outputs the filtered reconstructed block to the frame memory 122. Cyclic filtering refers to filtering used within the encoding loop (in-loop filtering), such as deblocking filtering (DF), sample adaptive offset (SAO), and adaptive cyclic filtering (ALF).

[0280] In ALF, a least-squares error filter is used to remove coding distortion. For example, for each 2×2 sub-block within the current block, one filter is selected from multiple filters based on the direction of the gradient and the activity of the locality.

[0281] Specifically, sub-blocks (e.g., 2×2 sub-blocks) are first classified into multiple classes (e.g., 15 or 25 classes). The classification of sub-blocks is based on the direction and activity of the gradient. For example, using the gradient direction value D (e.g., 0–2 or 0–4) and the gradient activity value A (e.g., 0–4), a classification value C (e.g., C = 5D + A) is calculated. Then, based on the classification value C, the sub-blocks are classified into multiple classes (e.g., 15 or 25 classes).

[0282] The gradient direction value D is derived, for example, by comparing gradients in multiple directions (e.g., horizontal, vertical, and two diagonal directions). Furthermore, the gradient activity value A is derived, for example, by summing the gradients in multiple directions and quantizing the sum.

[0283] Based on the results of this classification, the filter used for the sub-block is determined from among multiple filters.

[0284] The shape of the filter used in ALF can be, for example, a circular symmetrical shape. Figures 4A to 4C This is a diagram showing several examples of the shapes of filters used in ALF. Figure 4A This indicates a 5×5 diamond-shaped filter. Figure 4B This indicates a 7×7 diamond-shaped filter. Figure 4C This represents a 9×9 diamond-shaped filter. Information representing the filter's shape is signaled at the image level. However, the signaling of the filter's shape information is not limited to the image level; it can also be at other levels (e.g., sequence level, slice level, tile level, CTU level, or CU level).

[0285] The on / off state of ALF is determined, for example, at the picture level or the CU level. For instance, regarding luminance, the decision to use ALF is made at the CU level, while regarding chromatic aberration, it is made at the picture level. Information indicating the on / off state of ALF is signaled at the picture level or the CU level. However, the signaling of information indicating the on / off state of ALF is not limited to the picture level or the CU level; it can also be at other levels (e.g., sequence level, slice level, tile level, or CTU level).

[0286] The coefficient set of a selectable set of filters (e.g., up to 15 or 25 filters) is signaled at the picture level. Furthermore, the signaling of the coefficient set is not limited to the picture level; it can also be at other levels (e.g., sequence level, slice level, tile level, CTU level, CU level, or sub-block level).

[0287] [Frame Memory]

[0288] The frame memory 122 is a storage unit used to store reference images used in inter-frame prediction, and is also sometimes referred to as a frame buffer. Specifically, the frame memory 122 stores the reconstructed blocks filtered by the cyclic filtering unit 120.

[0289] Intra-frame prediction unit

[0290] The intra-frame prediction unit 124 performs intra-frame prediction (also called intra-picture prediction) of the current block by referring to the blocks in the current image stored in the block memory 118, thereby generating a prediction signal (intra-frame prediction signal). Specifically, the intra-frame prediction unit 124 generates an intra-frame prediction signal by performing intra-frame prediction by referring to samples (e.g., luminance value, chrominance value) of blocks adjacent to the current block, and outputs the intra-frame prediction signal to the prediction control unit 128.

[0291] For example, the intra-prediction unit 124 performs intra-prediction using one of a plurality of predefined intra-prediction modes. The plurality of intra-prediction modes includes one or more non-directional prediction modes and a plurality of directional prediction modes.

[0292] One or more non-directional prediction modes include, for example, the Planar prediction mode and the DC prediction mode as specified by the H.265 / HEVC (High-Efficiency Video Coding) specification (Non-Patent Document 1).

[0293] Multiple directional prediction modes may include, for example, the 33 directional prediction modes specified in the H.265 / HEVC specification. Alternatively, multiple directional prediction modes may also include 32 additional directional prediction modes (a total of 65 directional prediction modes). Figure 5A This diagram represents the 67 intra-prediction modes (2 non-directional prediction modes and 65 directional prediction modes) in intra-frame prediction. Solid arrows indicate the 33 directions specified by the H.265 / HEVC specification, while dashed arrows indicate the additional 32 directions.

[0294] Additionally, in intra-frame prediction of chroma blocks, luma blocks can also be referenced. That is, the chroma components of the current block can be predicted based on the luma components of the current block. Such intra-frame prediction is sometimes referred to as CCLM (cross-component linear model) prediction. This intra-frame prediction mode of chroma blocks referencing luma blocks (e.g., called CCLM mode) can also be added as one of the intra-frame prediction modes for chroma blocks.

[0295] The intra-prediction unit 124 can also correct the intra-predicted pixel values ​​based on the gradient of the reference pixels in the horizontal / vertical directions. Intra-prediction accompanied by such correction is sometimes referred to as PDPC (position-dependent intraprediction combination). Information indicating whether PDPC has been used (e.g., a PDPC flag) is signaled, for example, at the CU level. Furthermore, the signaling of this information is not limited to the CU level; it can also be at other levels (e.g., sequence level, image level, slice level, tile level, or CTU level).

[0296] [Inter-frame prediction department]

[0297] The inter-frame prediction unit 126 performs inter-frame prediction (also called inter-picture prediction) for the current block by referring to a reference picture stored in the frame memory 122 that is different from the current picture, thereby generating a prediction signal (inter-frame prediction signal). Inter-frame prediction is performed in units of the current block or sub-blocks within the current block (e.g., 4×4 blocks). For example, the inter-frame prediction unit 126 performs motion estimation within the reference picture for the current block or sub-block. Furthermore, the inter-frame prediction unit 126 uses motion information (e.g., motion vectors) obtained through motion estimation to perform motion compensation, thereby generating the inter-frame prediction signal for the current block or sub-block. Finally, the inter-frame prediction unit 126 outputs the generated inter-frame prediction signal to the prediction control unit 128.

[0298] The motion information used in motion compensation is signaled. A motion vector predictor can also be used in the signaling of motion vectors. That is, the difference between the motion vector and the predicted motion vector can also be signaled.

[0299] Alternatively, the inter-frame prediction signal can be generated using not only the motion information of the current block obtained through motion estimation but also the motion information of neighboring blocks. Specifically, the prediction signal based on the motion information obtained through motion estimation can be weighted and added together with the prediction signal based on the motion information of neighboring blocks, thereby generating the inter-frame prediction signal in sub-block units within the current block. Such inter-frame prediction (motion compensation) is sometimes referred to as OBMC (overlapped block motion compensation).

[0300] In this OBMC mode, information indicating the size of the sub-block used for OBMC (e.g., OBMC block size) is signaled at the sequence level. Furthermore, information indicating whether the OBMC mode is used (e.g., OBMC flag) is signaled at the CU level. However, the signaling level for this information is not limited to the sequence and CU levels; it can also be other levels (e.g., image level, slice level, tile level, CTU level, or sub-block level).

[0301] The OBMC model will be explained in more detail. Figure 5B and Figure 5C This is a flowchart and concept diagram used to illustrate the outline of predictive image correction processing based on OBMC processing.

[0302] First, the predicted image (Pred) obtained through normal motion compensation is obtained using the motion vectors (MV) assigned to the encoded object block.

[0303] Next, the predicted image (Pred_L) is obtained by using the motion vector (MV_L) of the encoded left adjacent block for the encoded object block. The first correction of the predicted image is performed by weighted superposition of the predicted image and Pred_L.

[0304] Similarly, the predicted image (Pred_U) is obtained by using the motion vector (MV_U) of the upper adjacent block of the encoded object block. The predicted image is then corrected a second time by weighting and superimposing the predicted image after the first correction and Pred_U, and this is used as the final predicted image.

[0305] In addition, this describes a two-stage correction method using the left and top adjacent blocks, but it can also be configured to perform more corrections using the right and bottom adjacent blocks than the two-stage method.

[0306] In addition, the area to be overlaid may not be the entire pixel area of ​​the block, but only a part of the area near the block boundary.

[0307] Furthermore, the process of correcting the predicted image based on a single reference image is explained here. However, the same principle applies when correcting the predicted image based on multiple reference images. After obtaining the corrected predicted image based on each reference image, the resulting predicted images are further superimposed to obtain the final predicted image.

[0308] In addition, the processing target block mentioned above can be a prediction block unit or a sub-block unit that further divides the prediction block.

[0309] One method for determining whether to use OBMC processing is to use a signal called obmc_flag. Specifically, in an encoding device, it is determined whether the block to be encoded belongs to a motion-complex region. If it does, the obmc_flag is set to 1 and OBMC processing is performed for encoding. If it does not belong to a motion-complex region, the obmc_flag is set to 0, and OBMC processing is not performed for encoding. On the other hand, in a decoding device, decoding is performed by decoding the obmc_flag recorded in the stream and switching between using and not using OBMC processing based on its value.

[0310] Alternatively, motion information can be exported at the decoding device side without being signaled. For example, the merging mode specified by the H.265 / HEVC standard can be used. Furthermore, motion information can also be exported by performing motion estimation at the decoding device side. In this case, motion estimation is performed without using the pixel values ​​of the current block.

[0311] Here, we will explain the motion estimation mode performed on the decoding device side. This motion estimation mode on the decoding device side may be called PMMVD (pattern matched motion vector derivation) mode or FRUC (frame rate up-conversion) mode.

[0312] exist Figure 5D The diagram below illustrates an example of FRUC processing. First, referencing the motion vectors of coded blocks spatially or temporally adjacent to the current block, a list of multiple candidates, each with a predicted motion vector, is generated (this list can also be shared with a merge list). Next, the best candidate MV is selected from the multiple candidate MVs registered in the candidate list. For example, an evaluation value is calculated for each candidate included in the candidate list, and one candidate is selected based on the evaluation value.

[0313] Furthermore, based on the selected candidate motion vectors, motion vectors for the current block are derived. Specifically, for example, the selected candidate motion vector (best candidate MV) can be derived as is, using it as the motion vector for the current block. Alternatively, for example, motion vectors for the current block can be derived by performing pattern matching in the surrounding region of the position within the reference image corresponding to the selected candidate motion vector. That is, the surrounding region of the best candidate MV can be searched using the same method, and if an MV with a better evaluation value is found, the best candidate MV is updated to the aforementioned MV and used as the final MV for the current block. Alternatively, a structure that does not perform this processing can be implemented.

[0314] The exact same processing can also be performed when processing is done in sub-block units.

[0315] Furthermore, the evaluation value is calculated by obtaining the difference value of the reconstructed image through pattern matching between the region within the reference image corresponding to the motion vector and the specified region. Alternatively, information other than the difference value can be used to calculate the evaluation value.

[0316] As a pattern matching, either pattern matching 1 or pattern matching 2 is used. Pattern matching 1 and pattern matching 2 can be referred to as bilateral matching and template matching, respectively.

[0317] In the first pattern matching, pattern matching is performed between two blocks within two different reference images, along the motion trajectory of the current block. Therefore, in the first pattern matching, the regions within other reference images along the motion trajectory of the current block are used as the defined regions for calculating the candidate evaluation values ​​described above.

[0318] Figure 6 This diagram illustrates an example of pattern matching (bidirectional matching) between two blocks along a motion trajectory. For example... Figure 6 As shown, in the first pattern matching, two motion vectors (MV0, MV1) are derived by searching for the best matching pair among two blocks in two different reference images (Ref0, Ref1) along the motion trajectory of the current block. Specifically, for the current block, the difference between the reconstructed image at a specified position in the first encoded reference image (Ref0) specified by the candidate MV and the reconstructed image at a specified position in the second encoded reference image (Ref1) specified by the symmetrical MV scaled by the aforementioned candidate MV over the display time interval is derived, and the obtained difference value is used to calculate an evaluation value. The candidate MV with the best evaluation value can be selected as the final MV from among multiple candidate MVs.

[0319] Under the assumption of continuous motion trajectories, the motion vectors (MV0, MV1) indicating two reference blocks are proportional to the temporal distances (TD0, TD1) between the current image (Cur Pic) and the two reference images (Ref0, Ref1). For example, in the case where the current image is located between the two reference images in time and the temporal distances from the current image to the two reference images are equal, in the first pattern matching, mirror-symmetric bidirectional motion vectors are derived.

[0320] In the second pattern matching, pattern matching is performed between the template in the current image (the block adjacent to the current block in the current image (e.g., the upper and / or left adjacent block)) and the block in the reference image. Therefore, in the second pattern matching, the block adjacent to the current block in the current image is used as the defined area for calculating the candidate evaluation value as described above.

[0321] Figure 7 This is an example of pattern matching (template matching) between a template in the current image and a block in a reference image. For example... Figure 7 As shown, in the second pattern matching, the motion vector of the current block is derived by searching within the reference image (Ref0) for the block that best matches the block adjacent to the current block (Cur block) within the current image (Cur Pic). Specifically, for the current block, the difference between the reconstructed images of the encoded regions of the left and top adjacent regions or one of them and the reconstructed image at the same position within the encoded reference image (Ref0) specified by the candidate MV is derived. The obtained difference value is used to calculate the evaluation value, and the candidate MV with the best evaluation value among multiple candidate MVs is selected as the best candidate MV.

[0322] Information indicating whether FRUC mode is used (e.g., referred to as the FRUC flag) is signaled at the CU level. Furthermore, when FRUC mode is used (e.g., when the FRUC flag is true), information indicating the pattern matching method (first pattern matching or second pattern matching) (e.g., referred to as the FRUC mode flag) is signaled at the CU level. Additionally, the signaling of this information is not limited to the CU level and can also be at other levels (e.g., sequence level, picture level, slice level, tile level, CTU level, or sub-block level).

[0323] This section explains how to derive motion vector patterns based on a model that assumes uniform linear motion. This pattern can be termed BIO (bi-directional optical flow).

[0324] Figure 8 This diagram is used to illustrate a model that assumes uniform linear motion. In Figure 8 In the middle, (v x v y () represents the velocity vector, and τ0 and τ1 represent the time distance between the current image (Cur Pic) and the two reference images (Ref0, Ref1), respectively. (MVx0, MVy0) represents the motion vector corresponding to the reference image Ref0, and (MVx1, MVy1) represents the motion vector corresponding to the reference image Ref1.

[0325] At this time, in the velocity vector (v x v y Under the assumption of constant linear motion of MVx0, MVy0 and (MVx1, MVy1) are expressed as (vxτ0, vyτ0) and (-vxτ1, -vyτ1) respectively, and the following optical flow equation (1) holds.

[0326] [Formula 1]

[0327]

[0328] Here, I (k) This represents the luminance value of the reference image k (k = 0, 1) after motion compensation. The optical flow equation states that the sum of (i) the temporal derivative of the luminance value, (ii) the product of the horizontal velocity and the horizontal component of the spatial gradient of the reference image, and (iii) the product of the vertical velocity and the vertical component of the spatial gradient of the reference image is equal to zero. Based on this optical flow equation combined with Hermite interpolation, the block-unit motion vector obtained from merge lists, etc., is corrected in pixels.

[0329] Alternatively, motion vectors can be derived on the decoding device side using a different method than deriving motion vectors based on a model assuming constant linear motion. For example, motion vectors can be derived on a sub-block basis based on the motion vectors of multiple adjacent blocks.

[0330] Here, we will explain the mode of deriving motion vectors on a sub-block basis based on the motion vectors of multiple adjacent blocks. This mode is sometimes referred to as the affine motion compensation prediction mode.

[0331] Figure 9A This is a diagram used to illustrate the derivation of sub-block unit motion vectors based on the motion vectors of multiple adjacent blocks. Figure 9AIn this context, the current block comprises 16 4×4 sub-blocks. Here, based on the motion vectors of adjacent blocks, the motion vector v0 of the top-left control point of the current block is derived, and based on the motion vectors of adjacent sub-blocks, the motion vector v1 of the top-right control point of the current block is derived. Furthermore, using the two motion vectors v0 and v1, the motion vectors (v0, v1, v1) of each sub-block within the current block are derived using the following equation (2). x v y ).

[0332] [Formula 2]

[0333]

[0334] Here, x and y represent the horizontal and vertical positions of the sub-block, respectively, and w represents the pre-set weight coefficient.

[0335] Such an affine motion compensation prediction mode may also include several modes with different methods for deriving the motion vectors of the upper left and upper right control points. Information representing such an affine motion compensation prediction mode (e.g., affine flags) is signaled at the CU level. Furthermore, the signaling of information representing this affine motion compensation prediction mode is not limited to the CU level; it can also be at other levels (e.g., sequence level, image level, slice level, tile level, CTU level, or sub-block level).

[0336] [Forecasting and Control Department]

[0337] The prediction control unit 128 selects one of the intra-frame prediction signal and the inter-frame prediction signal, and outputs the selected signal as the prediction signal to the subtraction unit 104 and the addition unit 116.

[0338] This section illustrates an example of exporting motion vectors from an encoded object image using a merge mode. Figure 9B This is a diagram used to illustrate the overview of motion vector derivation processing based on the merging mode.

[0339] First, a list of candidate predicted MVs registered with the predicted MVs is generated. Candidate predicted MVs include: spatially adjacent predicted MVs (MVs) belonging to multiple coded blocks spatially surrounding the coded object block; temporally adjacent predicted MVs (MVs) belonging to blocks whose positions in the coded reference image are projected nearby; combined predicted MVs (MVs) generated by combining the MV values ​​of spatially adjacent and temporally adjacent predicted MVs; and zero predicted MVs (MVs with a value of zero).

[0340] Next, the MV for the encoded object block is determined by selecting one predicted MV from the multiple predicted MVs registered in the predicted MV list.

[0341] Furthermore, in the variable-length coding section, merge_idx, which represents the signal that selected which prediction MV was recorded in the stream and encoded.

[0342] In addition, Figure 9B The predicted MVs registered in the predicted MV list described in the figure are one example. They may also be a number different from the number shown in the figure, or a structure that does not include a part of the predicted MVs in the figure, or a structure that adds predicted MVs other than the predicted MVs in the figure.

[0343] Alternatively, the MV of the encoded object block exported through the merge mode can be used for the DMVR processing described later to determine the final MV.

[0344] Here, an example of using DMVR to determine MV is explained.

[0345] Figure 9C This is a conceptual diagram used to illustrate the outline of DMVR processing.

[0346] First, the optimal MVP set for the processing object block is taken as the candidate MV. According to the candidate MV, reference pixels are obtained from the first reference image of the processed image in the L0 direction and the second reference image of the processed image in the L1 direction, respectively. The template is generated by taking the average of each reference pixel.

[0347] Next, using the template described above, the surrounding areas of the candidate music videos (MVs) for the first and second reference images are searched, and the MV with the lowest cost is selected as the final MV. Furthermore, the cost value is calculated using the differences between the pixel values ​​of the template and the pixel values ​​of the search area, as well as the MV value.

[0348] Furthermore, the general outline of the processing described herein is essentially the same in both the encoding and decoding devices.

[0349] In addition, even if it is not the process described here, any other process that can search for the surrounding of candidate MVs and export the final MV can be used.

[0350] Here, the mode of generating predicted images using LIC processing is explained.

[0351] Figure 9D This is a diagram illustrating the outline of a predictive image generation method using LIC-based brightness correction processing.

[0352] First, export the MV used to obtain the reference image corresponding to the encoded object block from the reference image, which is an encoded image.

[0353] Next, for the encoded object block, using the brightness pixel values ​​of the left and top adjacent encoded surrounding reference areas and the brightness pixel values ​​at the same position in the reference image specified by MV, information indicating how the brightness values ​​change in the reference image and the encoded object image is extracted, and brightness correction parameters are calculated.

[0354] By using the aforementioned brightness correction parameters to perform brightness correction processing on the reference image within the reference image specified by MV, a predicted image for the coded object block is generated.

[0355] in addition, Figure 9D The shape of the surrounding reference area mentioned above is one example; other shapes may also be used.

[0356] Furthermore, the process of generating a prediction image based on a single reference image is described here, but the same applies when generating a prediction image based on multiple reference images. The prediction image is generated after performing brightness correction processing on the reference images obtained from each reference image in the same way.

[0357] One method for determining whether to use LIC processing is to use a lic_flag as a signal indicating whether LIC processing is used. Specifically, in an encoding device, it is determined whether the block to be encoded belongs to a region where a brightness change has occurred. If it does, the lic_flag is set to 1, and LIC processing is used for encoding. If it does not belong to a region where a brightness change has occurred, the lic_flag is set to 0, and LIC processing is not used for encoding. On the other hand, in a decoding device, decoding is performed by decoding the lic_flag recorded in the stream and switching between using and not using LIC processing based on its value.

[0358] Other methods for determining whether to use LIC processing include checking whether LIC processing was used in surrounding blocks. As a specific example, when the encoded target block is in merge mode, it is determined whether the surrounding encoded blocks selected during the export of the MV in merge mode processing have been encoded using LIC processing. Based on the result, encoding is switched between using LIC processing and other methods. Furthermore, in this example, the decoding process is exactly the same.

[0359] [Overview of the Decoding Device]

[0360] Next, an outline of a decoding apparatus capable of decoding the encoded signal (encoded bit stream) output from the encoding apparatus 100 will be described. Figure 10This is a block diagram illustrating the functional structure of the decoding device 200 according to Embodiment 1. The decoding device 200 is a motion image / image decoding device that decodes motion images / images in block units.

[0361] like Figure 10 As shown, the decoding device 200 includes an entropy decoding unit 202, an inverse quantization unit 204, an inverse transform unit 206, an adder unit 208, a block memory 210, a cyclic filtering unit 212, a frame memory 214, an intra-frame prediction unit 216, an inter-frame prediction unit 218, and a prediction control unit 220.

[0362] The decoding device 200 is implemented, for example, by a general-purpose processor and memory. In this case, when the processor executes the software program stored in the memory, the processor functions as the entropy decoding unit 202, the inverse quantization unit 204, the inverse transform unit 206, the adder 208, the cyclic filter unit 212, the intra-frame prediction unit 216, the inter-frame prediction unit 218, and the prediction control unit 220. Alternatively, the decoding device 200 can also be implemented as one or more dedicated electronic circuits corresponding to the entropy decoding unit 202, the inverse quantization unit 204, the inverse transform unit 206, the adder 208, the cyclic filter unit 212, the intra-frame prediction unit 216, the inter-frame prediction unit 218, and the prediction control unit 220.

[0363] The following describes the constituent elements included in the decoding device 200.

[0364] [Entropy Decoding Department]

[0365] The entropy decoding unit 202 performs entropy decoding on the encoded bitstream. Specifically, the entropy decoding unit 202, for example, arithmetically decodes the encoded bitstream into a binary signal. Then, the entropy decoding unit 202 debinarizes the binary signal. As a result, the entropy decoding unit 202 outputs the quantization coefficients to the inverse quantization unit 204 in block units.

[0366] [De-quantization Department]

[0367] The inverse quantization unit 204 performs inverse quantization on the quantization coefficients of the decoded target block (hereinafter referred to as the current block), which is input from the entropy decoding unit 202. Specifically, the inverse quantization unit 204 performs inverse quantization on each quantization coefficient of the current block based on the quantization parameter corresponding to that quantization coefficient. Furthermore, the inverse quantization unit 204 outputs the inverse quantization coefficients (i.e., transform coefficients) of the current block to the inverse transform unit 206.

[0368] [Inverse Transformation Section]

[0369] The inverse transform unit 206 restores the prediction error by performing an inverse transform on the transform coefficients, which are inputs from the inverse quantization unit 204.

[0370] For example, if the information read from the encoded bitstream represents EMT or AMT (e.g., the AMT flag is true), the inverse transform unit 206 performs an inverse transform on the transform coefficients of the current block based on the information representing the transform type read from the transducer.

[0371] Furthermore, for example, when the information read from the encoded bitstream is represented using NSST, the inverse transform unit 206 applies an inverse re-transformation to the transform coefficients.

[0372] [Addition Department]

[0373] The adder 208 reconstructs the current block by adding the prediction error, which is input from the inverse transform 206, to the prediction sample, which is input from the prediction control 220. The adder 208 then outputs the reconstructed block to the block memory 210 and the cyclic filtering 212.

[0374] [Block Memory]

[0375] Block memory 210 is a storage unit used to store blocks within the decoded target image (hereinafter referred to as the current image) that serve as a reference in intra-frame prediction. Specifically, block memory 210 stores the reconstructed blocks output from adder 208.

[0376] [Loop Filtering Section]

[0377] The cyclic filtering unit 212 applies cyclic filtering to the block reconstructed by the addition unit 208 and outputs the filtered reconstructed block to the frame memory 214 and the display device, etc.

[0378] Given that the information indicating the on / off state of the ALF is read from the encoded bitstream, and the ALF is on, one filter is selected from multiple filters based on the direction and activity of the gradient of locality, and the selected filter is applied to the reconstructed block.

[0379] [Frame Memory]

[0380] The frame memory 214 is a storage unit used to store reference images used in inter-frame prediction; it is also sometimes called a frame buffer. Specifically, the frame memory 214 stores the reconstructed blocks filtered by the cyclic filtering unit 212.

[0381] Intra-frame prediction unit

[0382] The intra-prediction unit 216 performs intra-prediction based on the intra-prediction pattern read from the encoded bitstream, referring to blocks within the current image stored in the block memory 210, thereby generating a prediction signal (intra-prediction signal). Specifically, the intra-prediction unit 216 performs intra-prediction by referring to samples (e.g., luminance values, chrominance values) of blocks adjacent to the current block, thereby generating an intra-prediction signal, and outputs the intra-prediction signal to the prediction control unit 220.

[0383] In addition, if the intra-prediction mode of the reference luma block is selected in the intra-prediction of the chromatic difference block, the intra-prediction unit 216 can also predict the chromatic difference component of the current block based on the luma component of the current block.

[0384] Furthermore, when the information read from the encoded bitstream represents PDPC, the intra-prediction unit 216 corrects the pixel values ​​after intra-prediction based on the gradient of the reference pixel in the horizontal / vertical direction.

[0385] [Inter-frame prediction department]

[0386] The inter-frame prediction unit 218 refers to a reference image stored in the frame memory 214 and predicts the current block. Prediction is performed in units of the current block or sub-blocks within the current block (e.g., 4×4 blocks). For example, the inter-frame prediction unit 218 uses motion information (e.g., motion vectors) read from the coded bitstream to perform motion compensation, thereby generating an inter-frame prediction signal for the current block or sub-block, and outputs the inter-frame prediction signal to the prediction control unit 220.

[0387] Furthermore, when the information read from the encoded bitstream is represented in OBMC mode, the inter-frame prediction unit 218 uses not only the motion information of the current block obtained through motion estimation, but also the motion information of adjacent blocks to generate the inter-frame prediction signal.

[0388] Furthermore, when the information read from the coded bitstream is in FRUC mode, the inter-frame prediction unit 218 performs motion estimation according to the pattern matching method (bidirectional matching or template matching) read from the coded stream, thereby deriving motion information. The inter-frame prediction unit 218 then uses the derived motion information to perform motion compensation.

[0389] Furthermore, when using BIO mode, the inter-frame prediction unit 218 derives motion vectors based on a model assuming constant-velocity linear motion. Additionally, when the information representation read from the encoded bitstream employs affine motion compensation prediction mode, the inter-frame prediction unit 218 derives motion vectors on a sub-block basis based on the motion vectors of multiple adjacent blocks.

[0390] [Forecasting and Control Department]

[0391] The prediction control unit 220 selects one of the intra-frame prediction signal and the inter-frame prediction signal, and outputs the selected signal as the prediction signal to the adder 208.

[0392] [Filter Information Settings]

[0393] Next, we will explain in detail the settings of the filter information for the Adaptive Loop Filter (ALF).

[0394] Figure 11 This is a block diagram showing the structure of the loop filtering unit 120 of the encoding device 100 in Embodiment 1. The loop filtering unit 120 includes a filter control unit 131, a current filter information storage unit 132, a reference filter information storage unit 133, and an adaptive filter unit 134.

[0395] The filter control unit 131 is a circuit for processing filter information. The current filter information storage unit 132 is a memory for storing filter information used in the adaptive loop filter. The reference filter information storage unit 133 is a memory for storing filter information used in the adaptive loop filter. The adaptive filter unit 134 is a circuit for applying the adaptive loop filter to each block using the filter information stored in the current filter information storage unit 132.

[0396] Filter information can be represented as a filter information set. For example, at the beginning of a slice, the filter control unit 131 refers to the filter information set stored in the reference filter information storage unit 133 and sets the filter information set in the current filter information storage unit 132.

[0397] The adaptive filter unit 134 applies an adaptive cyclic filter to each block using the filter information set set in the current filter information storage unit 132. For example, the adaptive filter unit 134 selects filter coefficients from a variety of filter coefficients included in the filter information set, and applies them to each block according to the filter coefficients used to make the reproduced image (reconstructed image) approximate the input image (original image). Then, the adaptive filter unit 134 applies the adaptive cyclic filter to each block using the selected filter coefficients.

[0398] Furthermore, the NAL (Network Abstraction Layer) cell type of the current slice sometimes corresponds to an image that can be used as a reference image for inter-frame prediction. In this case, the filter control unit 131 stores the filter information set for the image including the current slice, which has been subjected to an adaptive cyclic filter, in the reference filter information storage unit 133.

[0399] Then, the filter control unit 131 stores multiple filter information sets that have applied adaptive cyclic filters to multiple images in the reference filter information storage unit 133.

[0400] The filter control unit 131 associates and manages multiple filter information sets stored in the reference filter information storage unit 133 with reference images, i.e., multiple reference image buffers stored in the frame memory 122. At this time, the filter control unit 131 uses its filter information sets to associate and manage the multiple filter information sets with reference images to which adaptive cyclic filters have been applied.

[0401] Furthermore, for example, a reference image might be marked as "unused for reference". In this case, the filter control unit 131 marks the filter information set associated with the reference image marked as "unused for reference" as "unused for reference". That is, when the reference image is deleted, the filter control unit 131 deletes the filter information set associated with the reference image.

[0402] Here, data such as reference images or filter information sets are marked as "unused for reference," meaning that this data will not be referenced in the future.

[0403] Furthermore, the filter control unit 131 can also output filter control information related to the setting method for setting filter information in the current filter information storage unit 132. Then, the encoding device 100 can notify the decoding device 200 of the filter control information related to the setting method for setting filter information in the current filter information storage unit 132.

[0404] For example, filter control information is information used to specify the set of filter information used in an adaptive cyclic filter applied to an image. Specifically, the filter control information may represent the value of a reference image index of a reference image associated with the set of filter information used in the adaptive cyclic filter applied to the image. Then, the filter information set can be specified from multiple filter information sets in the reference filter information storage unit 133 using the value of the reference image index of the reference image.

[0405] Alternatively, the filter control information may not refer to the filter information set in the filter information storage unit 133, but may represent the filter coefficients of the filter information set generated based on the input image (original image) and the reconstructed image (reconstructed image). Thus, the filter information set used in the adaptive cyclic filter applied to the image can be specified.

[0406] Furthermore, the filter control unit 131 can restrict references, ensuring that only filter information sets associated with reference images that meet specified conditions are referenced. Specifically, the filter control unit 131 can allow references only to filter information sets associated with images that meet the restrictions regarding TSA images. Images that meet the restrictions regarding TSA images can be images with a time ID less than that of the TSA image.

[0407] For example, in the encoding of images following the TSA image in encoding order, reference to images that precede the TSA image in encoding order—that is, images with the same or larger time ID as the TSA image—can be prohibited. Reference to filter information sets associated with such images can also be prohibited.

[0408] Additionally, an image that meets the restrictions regarding TSA images can be a reference image whose ID is less than that of the TSA image at any time between the current image and the reference image, according to the encoding order.

[0409] For example, if, in the current image and a reference image, there exists a TSA image with the same time ID or a time ID less than that reference image, referencing that reference image is prohibited. Therefore, referencing filter information sets associated with that reference image can be prohibited.

[0410] Furthermore, the larger the time ID, the higher the level it represents. For example, in the encoding of other images included in a relatively lower level, references to images included in a relatively higher level are prohibited.

[0411] Additionally, the time ID is represented by an integer greater than or equal to 0. When the time ID is 0, the level indicated by that time ID is the lowest level. Essentially, in the encoding of other images included in the lowest level or other levels, references to images included in the lowest level are not prohibited.

[0412] Figure 12A This is a flowchart illustrating a first specific example of the filter information management steps in Implementation 1. Figure 1 The encoding device 100 shown, for example, performs... Figure 12A The actions shown.

[0413] First, the encoding device 100 performs slice header processing (S101). For example, the entropy encoding unit 110 generates the slice header of the current slice of the encoding object and encodes it.

[0414] Next, the encoding device 100 processes the filter control information (S102). For example, the filter control unit 131 generates and outputs filter control information. Furthermore, the entropy encoding unit 110 encodes the filter control information output from the filter control unit 131. The filter control information can be included in the chip header. Therefore, the processing of the filter control information (S102) can also be included in the processing of the chip header (S101).

[0415] Next, the encoding device 100 determines whether the current slice is the first slice of the image (S103). For example, the segmentation unit 102 determines whether the current slice is the first slice of the image. This determination can be made by other constituent elements, or multiple constituent elements can make this determination separately.

[0416] If the current slice is not the first slice of the image ("No" in S103), a loop of processing for CU (Coding Unit) is performed (S109). That is, the encoding device 100 performs encoding processing for each CU. At this time, the encoding device 100 applies an adaptive cyclic filter.

[0417] On the other hand, if the current slice is the first slice of an image ("Yes" in S103), the encoding device 100 updates the reference image buffer (S104). Specifically, the inter-frame prediction unit 126 updates the information of the reference images stored in the frame memory 122. For example, the inter-frame prediction unit 126 marks reference images that do not need to be referenced as "unused forreference". Thus, reference images that do not need to be referenced are essentially eliminated.

[0418] After updating the reference image buffer, the encoding device 100 deletes unnecessary filter information (S105). For example, the filter control unit 131 marks the filter information set associated with the reference image marked as "unused for reference" as "unused for reference". Thus, the filter information set that does not need to be referenced is essentially eliminated.

[0419] That is, when a reference image is removed from the reference image buffer, the filter information set associated with the reference image is also removed.

[0420] Furthermore, after deleting unnecessary filter information, the encoding device 100 determines whether the NAL cell type of the current slice corresponds to a reference or a non-reference (S106).

[0421] Here, the image corresponding to the reference is the image being referenced, i.e., the image that is allowed to be referenced, while the image corresponding to the non-reference is the image that is not referenced, i.e., the image that is prohibited from being referenced. For example, the filter control unit 131 determines whether the NAL cell type of the current slice corresponds to the type of the referenced image or the type of the non-referenced image.

[0422] In the non-referenced case (non-referenced in S106), a loop of processing for each CU is performed (S109). That is, the encoding device 100 performs encoding processing on each CU. At this time, the encoding device 100 applies an adaptive cyclic filter.

[0423] On the other hand, in the case of reference (reference in S106), the encoding device 100 establishes an association between the storage area and the current image (S107). For example, the filter control unit 131 establishes an association between the storage area in the reference filter information storage unit 133 that stores the filter information set and the current image including the current slice.

[0424] Then, the encoding device 100 saves the filter information in the storage area (S108). Specifically, the filter control unit 131 saves the set of filter information used in the adaptive cyclic filter for the current image in the storage area associated with the current image.

[0425] Then, a loop for processing the CU is performed (S109). That is, the encoding device 100 performs encoding processing on each CU. At this time, the encoding device 100 applies an adaptive cyclic filter.

[0426] By performing the above actions, the encoding device 100 can update the filter information in the reference filter information storage unit 133 at the beginning of the image according to the state of the reference image buffer. Furthermore, the encoding device 100 can store the filter information of the current image in the reference filter information storage unit 133.

[0427] Furthermore, by performing the aforementioned operations, the encoding device 100 can establish and manage the association between the reference image and the filter information set in the frame memory 122 and the reference filter information storage unit 133. Additionally, the filter information set associated with the reference image can also be managed using a reference image index used to specify the reference image.

[0428] Figure 12B This is a flowchart illustrating a first specific example of the filter information setting steps in Implementation Method 1. Figure 1 The encoding device 100 shown, for example, performs... Figure 12B The actions shown.

[0429] First, the encoding device 100 constructs a reference image list (S201). Specifically, the inter-frame prediction unit 126 constructs the reference image list. For example, in Figure 12A When updating the reference image buffer (S104), the process of constructing a reference image list can be performed.

[0430] Next, the encoding device 100 acquires filter control information (S202). For example, the filter control unit 131 acquires... Figure 12A The filter control information is processed in the filter control information processing (S102).

[0431] When using a filter information set associated with a reference image, the filter control information may include a reference image index for specifying the filter information set in the reference filter information storage unit 133.

[0432] Furthermore, when a filter information set associated with a reference image is not used, the filter control information may include the filter information set used in the adaptive cyclic filter instead of the reference image index. That is, in this case, the filter control information may also include the filter coefficients that constitute the filter information set.

[0433] Next, the encoding device 100 determines whether to refer to the filter information of the reference image (S203). Specifically, the filter control unit 131 determines whether to refer to the filter information set associated with the reference image in the setting of the filter information set for the current image.

[0434] When the encoding device 100 refers to the filter information of the reference image ("Yes" in S203), it sets the filter information based on the reference image index (S204). For example, the filter control unit 131 selects a filter information set that is associated with the reference image specified by the reference image index included in the filter control information from multiple filter information sets in the filter information storage unit 133. Furthermore, the filter control unit 131 stores the selected filter information set in the current filter information storage unit 132.

[0435] Furthermore, the encoding device 100 sets filter information based on the input image and the reproduced image without referring to the filter information of the reference image ("No" in S203) (S205). That is, the encoding device 100 stores the filter information set generated based on the input image and the reproduced image in the current filter information storage unit 132. For example, if the filter control information includes a filter information set generated based on the input image and the reproduced image, the filter control unit 131 stores the filter information set contained in the filter control information in the current filter information storage unit 132.

[0436] Subsequently, a loop of processing for each CU is performed (S206). That is, the encoding device 100 performs encoding processing on each CU. At this time, the encoding device 100 applies an adaptive cyclic filter. Specifically, the adaptive filter unit 134 applies an adaptive cyclic filter using the set filter information.

[0437] The encoding device 100 can save the filter information for the current image in the current filter information storage unit 132 according to the filter control information by performing the above-described actions.

[0438] Furthermore, the encoding device 100 may also perform encoding processing on each CU in part before generating the filter control information to generate the reproduced image. Therefore, the encoding device 100 can appropriately generate a filter information set based on the input image and the reproduced image, and can appropriately generate filter control information containing the filter information set based on the input image and the reproduced image.

[0439] Figure 13A This is a flowchart illustrating a second specific example of the filter information management steps in Implementation Method 1. Figure 1 The encoding device 100 shown can perform Figure 13A The actions shown.

[0440] In this example, firstly, the encoding device 100 and Figure 12A The processes shown (S101 and S102) similarly perform the processing of the chip header and filter control information (S301 and S302). Then, the encoding device 100 and Figure 12A The process shown (S103) similarly determines whether the current slice is the first slice of the image (S303).

[0441] If the current slice is the first slice of the image ("Yes" in S303), the encoding device 100 and Figure 12A The process shown (S104) similarly updates the reference image buffer (S304). After updating the reference image buffer, the encoding device 100 and... Figure 12A The process shown (S105) similarly removes unwanted filter information (S305).

[0442] Then, after deleting unnecessary filter information, the encoding device 100 determines, in the same manner as the process shown in FIG12 (S106), whether the NAL cell type of the current slice corresponds to a reference or a non-reference (S306).

[0443] In the case of reference (reference in S306), the encoding device 100 and Figure 12A The process shown (S107) similarly associates the saved area with the current image (S307).

[0444] If the current slice is not the first slice of the image ("No" in S303), the process from updating the reference image buffer (S304) to associating the saved region with the current image (S307) is skipped. Furthermore, if the NAL cell type of the current slice corresponds to a non-reference (non-reference in S306), the process of associating the saved region with the current image (S307) is skipped.

[0445] Then, a loop of processing for each CU is performed (S308). That is, the encoding device 100 performs encoding processing on each CU.

[0446] Next, the encoding device 100 sets the filter information used in the adaptive cyclic filter (S309). Specifically, the filter control unit 131 stores the set of filter information used in the adaptive cyclic filter in the current filter information storage unit 132.

[0447] Next, the encoding device 100 applies an adaptive cyclic filter (S310). Specifically, the adaptive filter unit 134 uses the filter information set stored in the current filter information storage unit 132 to apply an adaptive cyclic filter to the current slice.

[0448] Next, the encoding device 100 determines whether the NAL unit type of the current slice corresponds to a reference or a non-reference (S311). For example, the filter control unit 131 determines whether the NAL unit type of the current slice corresponds to the type of the referenced image or the type of the non-referenced image.

[0449] In the case of reference (reference in S311), the encoding device 100 saves the filter information in the storage area (S312). Specifically, the filter control unit 131 saves the set of filter information used in the adaptive cyclic filter for the current image in the storage area associated with the current image. In the case of no reference (no reference in S311), the saving process is skipped (S312).

[0450] By performing the aforementioned actions, the encoding device 100 can set filter information and apply an adaptive cyclic filter after processing by the CU. Therefore, the encoding device 100 can appropriately generate a reconstructed image before setting the filter information. Thus, the encoding device 100 can set the filter information based on the input image and the reconstructed image.

[0451] Furthermore, the entropy coding unit 110 can append filter information based on the input image and the reconstructed image to the slice data instead of the slice header and encode it. Moreover, the entropy coding unit 110 can encode the slice header containing filter control information indicating the use of the filter information appended to the slice data.

[0452] Figure 13B This is a flowchart illustrating a second specific example of the filter information setting steps in Implementation Method 1. Figure 1 The encoding device 100 shown can perform Figure 13B The actions shown.

[0453] In this example, firstly, the encoding device 100 and Figure 12B The process shown (S201) similarly constructs a list of reference images (S401). Then, the encoding device 100 and Figure 12B The process shown (S202) similarly obtains filter control information (S402).

[0454] Then, a loop of processing for each CU is performed (S403). That is, the encoding device 100 performs encoding processing on each CU.

[0455] Subsequently, the encoding device 100 and Figure 12B The process shown (S203) similarly determines whether to refer to the filter information of the reference image (S404).

[0456] When the encoding device 100 refers to the filter information of the reference image ("Yes" in S404), it... Figure 12B The processing shown (S204) similarly sets the filter information based on the reference image index (S405). The encoding device 100, without referring to the filter information of the reference image ("No" in S404), and... Figure 12B The processing shown (S205) similarly sets filter information based on the input image and the reproduced image (S406).

[0457] Then, the encoding device 100 applies an adaptive cyclic filter (S407). Specifically, the adaptive filter unit 134 applies an adaptive cyclic filter using the set filter information.

[0458] Figure 14A This is a conceptual diagram representing a first specific example of the reference limitation of filter information in Implementation Method 1. Figure 14A In this context, "reference" corresponds to permitted reference, and "non-reference" corresponds to prohibited reference. Furthermore, Figure 14A The images p0 to p8 shown are encoded in the order of p0, p1, p2, p3, p4, p5, p6, p7, and p8.

[0459] Furthermore, each of images p0 through p8 is assigned a time ID representing a hierarchy of time scalability. Specifically, 0 is assigned to images p0 and p1 as time IDs. 1 is assigned to image p2 as a time ID. 2 is assigned to images p3 and p6 as time IDs. 3 is assigned to images p4, p5, p7, and p8 as time IDs.

[0460] also, Figure 14A An example is shown where image p6 is the current image being encoded. Furthermore, the filter information settings for each slice of image p6 show the filter information that allows reference and the filter information that prohibits reference.

[0461] If image p6 is the current image being encoded, then images p0 through p5 are already encoded images. The time ID of each of images p4 and p5 is greater than the time ID of image p6. Therefore, in the encoding of image p6, referencing images p4 and p5 is prohibited. Therefore, in setting the filter information for image p6, referencing the filter information of each of images p4 and p5 can be prohibited.

[0462] Furthermore, if image p6 is a TSA image, referencing image p3, which has the same time ID as image p6, is prohibited in the encoding of image p6. Therefore, in the filter information settings for image p6, referencing the filter information of image p3 can be prohibited.

[0463] Therefore, in setting the filter information for image p6, it is permissible to refer to the filter information of images p0 to p2 in images p0 to p5, but it is prohibited to refer to the filter information of images p3 to p5 in images p0 to p5.

[0464] Figure 14B This is a conceptual diagram representing a second specific example of the reference limitation for filter information in Implementation Method 1. Figure 14B In, with Figure 14A Similarly, "referencing" corresponds to permitted referencing, and "non-referencing" corresponds to prohibited referencing. Furthermore, in Figure 14B In, with Figure 14A Similarly, images p0 to p8 are shown. Figure 14B The example shows the encoding order of images p0 to p8 and the time IDs assigned to images p0 to p8 respectively. Figure 14A The time IDs in the examples are the same.

[0465] and, Figure 14B An example is shown where image p7 is the current image being encoded. Furthermore, the filter information settings for each slice of image p7 show the filter information that allows reference and the filter information that prohibits reference.

[0466] If image p7 is the current image of the encoded object, then images p0 to p6 are already encoded images. It is permissible to refer to specific images from p0 to p6 whose encoded time ID is less than any image from the next image to p7, or to images with a time ID of 0.

[0467] Specifically, the time IDs of images p0 and p1 are each 0. Furthermore, the time ID of image p2 is less than any of images p3 through p7. Additionally, the time ID of image p6 is less than the time ID of image p7. Therefore, the filter information for image p7 can be set with reference to the filter information of images p0 through p2 and p6.

[0468] Furthermore, the time ID of image p3 is the same as the time ID of image p6 in images p4-p7. The time ID of image p4 is greater than the time ID of image p6 in images p5-p7. The time ID of image p5 is greater than the time ID of image p6 in both images p6 and p7. Therefore, in setting the filter information for image p7, referencing the filter information of images p3-p5 can be disabled.

[0469] The reference restrictions described above apply when each image with a time ID different from 0 is a TSA image. That is, the reference restrictions described above apply when images p2 to p8 are TSA images.

[0470] and, Figure 14B The reference restrictions shown also correspond to the reference restrictions when image p6 is a TSA image. For example, if there is a TSA image with the same or smaller time ID as the encoded image and the current image between an encoded image with a time ID greater than 0, then referencing the filter information of the encoded image can be prohibited. Figure 14B This reference limitation is also shown.

[0471] Figure 15 This is a block diagram showing the structure of the loop filter unit 212 of the decoding device 200 in Embodiment 1. The structure of the loop filter unit 212 of the decoding device 200 corresponds to the structure of the loop filter unit 120 of the encoding device 100. Specifically, the loop filter unit 212 includes a filter control unit 231, a current filter information storage unit 232, a reference filter information storage unit 233, and an adaptive filter unit 234.

[0472] The filter control unit 231 is a circuit for processing filter information. The current filter information storage unit 232 is a memory for storing filter information used in the adaptive cyclic filter. The reference filter information storage unit 233 is a memory for storing filter information used in the adaptive cyclic filter. The adaptive filter unit 234 is a circuit for applying the adaptive cyclic filter to each block using the filter information stored in the current filter information storage unit 232.

[0473] The operation of the loop filter unit 212 of the decoding device 200 corresponds to the operation of the loop filter unit 120 of the encoding device 100. For example, at the beginning of a slice, the filter control unit 231 refers to the filter information set stored in the reference filter information storage unit 233 and sets the filter information set in the current filter information storage unit 232.

[0474] The adaptive filter unit 234 applies an adaptive cyclic filter to each block using the filter information set stored in the current filter information storage unit 232. For example, according to the characteristics of the reproduced image, the adaptive filter unit 234 selects filter coefficients from a variety of filter coefficients included in the filter information set for each block to make the reproduced image (reconstructed image) approximate the input image (original image). Furthermore, the adaptive filter unit 234 applies the adaptive cyclic filter to each block using the selected filter coefficients.

[0475] The NAL unit type of the current slice may correspond to a reference image that can be used for inter-frame prediction. In this case, the filter control unit 231 stores the filter information set used in the adaptive cyclic filter for the image containing the current slice in the reference filter information storage unit 233.

[0476] Furthermore, the filter control unit 231 stores multiple filter information sets used in the adaptive cyclic filter for multiple images in the reference filter information storage unit 233.

[0477] The filter control unit 231 associates and manages multiple filter information sets stored in the reference filter information storage unit 233 with multiple reference images stored in the reference image buffer, i.e., the frame memory 214. At this time, the filter control unit 231 uses the filter information sets to associate and manage the multiple filter information sets with reference images to which adaptive cyclic filters have been applied.

[0478] Furthermore, for example, a reference image may be marked as "unused for reference". In this case, the filter control unit 231 marks the filter information set associated with the reference image marked as "unused for reference" as "unused for reference". That is, when the reference image is deleted, the filter control unit 231 deletes the filter information set associated with the reference image.

[0479] Furthermore, filter control information related to the setting method for setting filter information in the current filter information storage unit 232 can be notified from the encoding device 100 to the decoding device 200. Then, the filter control information related to the setting method for setting filter information in the current filter information storage unit 232 can be input to the filter control unit 231.

[0480] For example, filter control information is information used to specify the set of filter information used in an adaptive cyclic filter applied to an image. Specifically, the filter control information may represent the value of a reference image index of a reference image associated with the set of filter information used in the adaptive cyclic filter applied to the image. Then, the filter information set can be specified from multiple filter information sets in the reference filter information storage unit 233 using the value of the reference image index of the reference image.

[0481] Alternatively, the filter control information can represent filter coefficients from a filter information set generated based on the input image (original image) and the reconstructed image (reconstructed image), instead of the filter information set referenced in the filter information storage unit 233. This allows the filter information set used in the adaptive cyclic filter applied to the image to be specified.

[0482] Furthermore, the filter control unit 231 can restrict references so that only filter information sets associated with reference images that meet specified conditions are referenced. Specifically, the filter control unit 231 can only allow references to filter information sets associated with images that meet the restrictions regarding TSA images. Images that meet the restrictions regarding TSA images can be images with a time ID less than that of the TSA image.

[0483] For example, in the decoding of images following the TSA image in decoding order, referencing images that precede the TSA image in decoding order and have the same or larger time ID as the TSA image can be prohibited. Referencing filter information sets associated with such images can also be prohibited.

[0484] Additionally, an image that meets the restrictions regarding TSA images can be a reference image whose time ID between the current image and the reference image is less than that of any other TSA image, according to the decoding order.

[0485] For example, if, in the decoding order, there exists a TSA image between the current image and a reference image that has the same time ID or a smaller time ID than the reference image, referencing that reference image is prohibited. Therefore, referencing filter information sets associated with that reference image can be prohibited.

[0486] Regarding the encoding device 100, using Figures 12A to 14B The actions described can be explained by replacing the encoding with decoding, which relates to the decoding device 200.

[0487] For example, the decoding device 200 performs with Figure 12A The actions shown correspond to the actions described. This can be based on... Figure 12A Explanation and Figure 12A The actions shown correspond to the actions performed by the decoding device 200.

[0488] First, the decoding device 200 processes the slice header (S101). For example, the entropy decoding unit 202 analyzes the slice header of the current slice of the decoding object and performs decoding.

[0489] Next, the decoding device 200 processes the filter control information (S102). For example, the entropy decoding unit 202 analyzes the filter control information and decodes it. Furthermore, the filter control unit 231 acquires the decoded filter control information. The filter control information can be included in the chip header. Therefore, the processing of the filter control information (S102) can also be included in the processing of the chip header (S101).

[0490] Next, the decoding device 200 determines whether the current slice is the first slice of the image (S103). For example, the entropy decoding unit 202 determines whether the current slice is the first slice of the image. This determination can be performed by other components, or by multiple components separately.

[0491] If the current slice is not the first slice of the image ("No" in S103), a loop of processing for CU (Coding Unit) is performed (S109). That is, the decoding device 200 performs decoding processing for each CU. At this time, the decoding device 200 applies an adaptive cyclic filter.

[0492] On the other hand, if the current slice is the first slice of an image ("Yes" in S103), the decoding device 200 updates the reference image buffer (S104). Specifically, the inter-frame prediction unit 218 updates the information of the reference images stored in the frame memory 214. For example, the inter-frame prediction unit 218 marks reference images that do not need to be referenced as "unused forreference". Thus, reference images that do not need to be referenced are essentially eliminated.

[0493] After updating the reference image buffer, the decoding device 200 deletes unnecessary filter information (S105). For example, the filter control unit 231 marks the filter information set associated with the reference image marked as "unused for reference" as "unused for reference". Thus, the filter information set that does not need to be referenced is essentially eliminated.

[0494] That is, when a reference image is removed from the reference image buffer, the filter information set associated with the reference image is also removed.

[0495] Then, after deleting unnecessary filter information, the decoding device 200 determines whether the NAL unit type of the current slice corresponds to a reference or a non-reference (S106). For example, the filter control unit 231 determines whether the NAL unit type of the current slice corresponds to the type of the referenced image or the type of the non-referenced image.

[0496] In the non-referenced case (non-referenced in S106), a loop of processing for each CU is performed (S109). That is, the decoding device 200 performs encoding processing on each CU. At this time, the decoding device 200 applies an adaptive cyclic filter.

[0497] On the other hand, in the case of reference (reference in S106), the decoding device 200 establishes an association between the storage area and the current image (S107). For example, the filter control unit 231 establishes an association between the storage area in the reference filter information storage unit 233 that stores the filter information set and the current image containing the current slice.

[0498] Then, the decoding device 200 saves the filter information in the storage area (S108). Specifically, the filter control unit 231 saves the set of filter information used in the adaptive cyclic filter for the current image in the storage area associated with the current image.

[0499] Then, a loop of processing for each CU is performed (S109). That is, the decoding device 200 performs encoding processing on each CU. At this time, the decoding device 200 applies an adaptive cyclic filter.

[0500] By performing the aforementioned actions, the decoding device 200 can update the filter information in the reference filter information storage unit 233 at the beginning of an image, according to the state of the reference image buffer. Furthermore, the decoding device 200 can store the filter information of the current image in the reference filter information storage unit 233.

[0501] Furthermore, by performing the aforementioned actions, the decoding device 200 can associate and manage the reference image with the filter information set in the frame memory 214 and the reference filter information storage unit 233. Additionally, the filter information set associated with the reference image can be managed using a reference image index for specifying the reference image.

[0502] In addition, the decoding device 200 performs with Figure 12B The action shown corresponds to the action. It can be based on... Figure 12B Explanation and Figure 12B The actions shown correspond to the actions performed by the decoding device 200.

[0503] First, the decoding device 200 constructs a reference image list (S201). Specifically, the inter-frame prediction unit 218 constructs the reference image list. For example, in Figure 12A When updating the reference image buffer (S104), the process of constructing a reference image list can be performed.

[0504] Next, the decoding device 200 acquires filter control information (S202). For example, the filter control unit 231 acquires... Figure 12A The filter control information is processed in the filter control information processing (S102).

[0505] When using a filter information set associated with a reference image, the filter control information may include a reference image index for specifying the filter information set in the reference filter information storage unit 233.

[0506] Furthermore, in the absence of a filter information set associated with a reference image, the filter control information can include the filter information set used in the adaptive cyclic filter instead of the reference image index. That is, in this case, the filter control information can include the filter coefficients that constitute the filter information set.

[0507] Next, the decoding device 200 determines whether to refer to the filter information of the reference image (S203). Specifically, the filter control unit 231 determines whether to refer to the filter information set associated with the reference image in the setting of the filter information set for the current image.

[0508] When the decoding device 200 refers to the filter information of the reference image ("Yes" in S203), it sets filter information based on the reference image index (S204). For example, the filter control unit 231 selects a filter information set that is associated with the reference image specified by the reference image index included in the filter control information from multiple filter information sets in the reference filter information storage unit 233. Then, the filter control unit 231 saves the selected filter information set in the current filter information storage unit 232.

[0509] Furthermore, the decoding device 200 sets filter information based on the input image and the reproduced image without referring to the filter information of the reference image ("No" in S203) (S205). That is, the decoding device 200 stores the filter information set generated based on the input image and the reproduced image in the current filter information storage unit 232. For example, if the filter control information includes a filter information set generated based on the input image and the reproduced image, the filter control unit 231 stores the filter information set contained in the filter control information in the current filter information storage unit 232.

[0510] Then, a loop of processing for each CU is performed (S206). That is, the decoding device 200 performs decoding processing on each CU. At this time, the decoding device 200 applies an adaptive cyclic filter. Specifically, the adaptive filter unit 234 applies an adaptive cyclic filter using the set filter information.

[0511] By performing the above-described actions, the decoding device 200 can save the filter information for the current image in the current filter information storage unit 232 according to the filter control information.

[0512] Furthermore, the decoding device 200 can perform [interactions with / etc.]. Figure 13A The action shown corresponds to the action. It can be based on... Figure 13A Explanation of being able to Figure 13A The actions shown correspond to the actions performed by the decoding device 200.

[0513] First, the decoding device 200 and Figure 12A The processes shown (S101 and S102) similarly perform the processing of the chip header and filter control information (S301 and S302). Furthermore, the decoding device 200 and... Figure 12A The process shown (S103) similarly determines whether the current slice is the first slice of the image (S303).

[0514] If the current slice is the first slice of an image ("Yes" in S303), the decoding device 200 and Figure 12A The process shown (S104) updates the reference image buffer (S304) in the same way. After updating the reference image buffer, the decoding device 200 and Figure 12A The process shown (S105) similarly removes unwanted filter information (S305).

[0515] Then, after deleting unnecessary filter information, the decoding device 200... Figure 12A The process shown (S106) similarly determines whether the NAL cell type of the current slice corresponds to a reference or a non-reference (S306).

[0516] In the case of reference (reference in S306), the decoding device 200 and Figure 12A The process shown (S107) similarly associates the saved area with the current image (S307).

[0517] If the current slice is not the first slice of the image ("No" in S303), the process from updating the reference image buffer (S304) to associating the saved region with the current image (S307) is skipped. Furthermore, if the NAL cell type of the current slice corresponds to a non-referenced image ("Non-referenced" in S306), the process of associating the saved region with the current image (S307) is skipped.

[0518] Then, a loop of processing for each CU is performed (S308). That is, the decoding device 200 performs decoding processing for each CU.

[0519] Next, the decoding device 200 sets the filter information used in the adaptive loop filter (S309). Specifically, the filter control unit 231 stores the set of filter information used in the adaptive loop filter in the current filter information storage unit 232.

[0520] Next, the decoding device 200 applies an adaptive cyclic filter (S310). Specifically, the adaptive filter unit 234 uses the filter information set stored in the current filter information storage unit 232 to apply an adaptive cyclic filter to the current slice.

[0521] Next, the decoding device 200 determines whether the NAL unit type of the current slice corresponds to a reference or a non-reference (S311). For example, the filter control unit 231 determines whether the NAL unit type of the current slice corresponds to the type of the referenced image or the type of the non-referenced image.

[0522] In the case of reference (reference in S311), the decoding device 200 saves the filter information in the storage area (S312). Specifically, the filter control unit 231 saves the set of filter information used in the adaptive cyclic filter of the current image in the storage area associated with the current image. In the case of no reference (no reference in S311), the saving process is skipped (S312).

[0523] Furthermore, filter information set based on the input image and the reconstructed image can be appended to the slice data instead of to the slice header. Moreover, filter control information can indicate the use of the filter information appended to the slice data. The entropy decoding unit 202 can decode such filter information and filter control information.

[0524] Furthermore, the decoding device 200 can perform [interactions with / etc.]. Figure 13B The action shown corresponds to the action. It can be based on... Figure 13B Explanation and Figure 13B The actions shown correspond to actions that can be performed by the decoding device 200.

[0525] First, the decoding device 200 and Figure 12B The process shown (S201) similarly constructs a list of reference images (S401). Then, the decoding device 200 and... Figure 12B The process shown (S202) similarly obtains filter control information (S402).

[0526] Then, a loop of processing for each CU is performed (S403). That is, the decoding device 200 performs decoding processing for each CU.

[0527] Then, the decoding device 200 and Figure 12B The process shown (S203) similarly determines whether to refer to the filter information of the reference image (S404).

[0528] When the decoding device 200 refers to the filter information of the reference image ("Yes" in S404), it... Figure 12B The processing shown (S204) similarly sets the filter information based on the reference image index (S405). The decoding device 200, without referring to the filter information of the reference image ("No" in S404), works with... Figure 12B Similarly, the processing shown (S205) sets filter information based on the input image and the reproduced image (S406).

[0529] Then, the decoding device 200 applies an adaptive cyclic filter (S407). Specifically, the adaptive filter unit 234 applies an adaptive cyclic filter using the set filter information.

[0530] and Figure 14A and Figure 14B The relevant explanation can also be applied to the explanation related to the decoding device 200 by replacing encoding with decoding.

[0531] [Variations on filter information processing methods]

[0532] Next, variations of the filter information processing method will be explained. That is, regarding filter information, a description will be provided... Figures 12A to 14B The processing methods shown are different. This is because the basic constituent elements in the deformation method are different. Figure 1 , Figure 10 , Figure 11 as well as Figure 15 The constituent elements shown are the same, so the description is omitted.

[0533] In this modified embodiment, filter control information is communicated from the encoding device 100 to the decoding device 200 via a parameter set associated with the chip header. Specifically, the filter control information is contained within the parameter set communicated from the encoding device 100 to the decoding device 200. Here, the parameter set may be a PPS (Picture Parameter Set).

[0534] Figure 16 This is a flowchart of the first specific example of the filter information processing steps in the deformation mode. Figure 1 The encoding device 100 shown, for example, performs... Figure 16 The actions shown.

[0535] First, the encoding device 100 determines whether to perform PPS processing (S501). Specifically, PPS processing may correspond to the process of notifying the decoding device 200 of PPS.

[0536] For example, the entropy encoding unit 110 determines whether to encode PPS. The entropy encoding unit 110 can determine whether to encode PPS based on the encoding order, display order, or type of the images to be encoded. Specifically, the entropy encoding unit 110 can determine to encode PPS if the image to be encoded is an IDR (Instantaneous Decoder Refresh) image.

[0537] If the encoding device 100 determines that PPS processing should be performed ("Yes" in S501), it performs PPS processing (S502). For example, if the entropy encoding unit 110 determines that PPS should be encoded, it encodes PPS. Furthermore, PPS may include, for example, filter control information. The filter control information may include, for example, a filter information set. On the other hand, if the encoding device 100 determines that PPS processing should not be performed ("No" in S501), it skips PPS processing.

[0538] Next, the encoding device 100 obtains the NAL unit type (S503). For example, the entropy encoding unit 110 obtains the NAL unit type of the image to be encoded. More specifically, the entropy encoding unit 110 obtains the NAL unit type of the slice of the image to be encoded. The NAL unit type of the slice of the image to be encoded corresponds to the type of the image to be encoded.

[0539] Next, the encoding device 100 determines whether the image to be encoded is an IDR image (S504). For example, the filter control unit 131 determines whether the image to be encoded is an IDR image according to the NAL unit type of the image to be encoded.

[0540] If the image to be encoded is determined to be an IDR image ("Yes" in S504), the encoding device 100 deletes the stored filter information (S505). For example, the filter control unit 131 deletes multiple filter information sets stored in the reference filter information storage unit 133.

[0541] If the image to be encoded is determined to be not an IDR image ("No" in S504), the encoding device 100 determines whether PPS processing has been performed (S502) (S506). For example, the filter control unit 131 determines whether PPS has been encoded according to the encoding order, display order, or type of the image to be encoded.

[0542] Then, if it is determined that PPS processing (S502) has been performed ("Yes" in S506), the encoding device 100 saves the filter information based on the PPS in the PPS processing (S502) (S507). On the other hand, if it is determined that PPS processing (S502) has not been performed ("No" in S506), the encoding device 100 skips the process of saving the filter information (S507).

[0543] Furthermore, assuming that PPS processing is performed when the image to be encoded is determined to be an IDR image ("yes" in S504) (S502). Therefore, in this case, after the filter information is deleted, the encoding device 100 does not determine whether PPS processing has been performed, and saves the filter information based on the PPS in the PPS processing (S502).

[0544] For example, the filter control unit 131 specifies a filter information set based on the filter control information contained in the PPS encoded in the PPS processing (S502), and stores the specific filter information set in the reference filter information storage unit 133. Furthermore, the filter control unit 131 associates the filter information set specified based on the filter control information contained in the PPS with the image parameter set ID (PPS ID) of the PPS and manages it. That is, the filter information set is associated with the image parameter set ID and stored.

[0545] Furthermore, when processing multiple PPSs, for each of the multiple PPSs, a filter information set corresponding to that PPS is specified and stored based on the filter control information of that PPS. Thus, for example, multiple filter information sets corresponding to each of the consecutively inserted PPSs are specified and stored.

[0546] Next, the encoding device 100 obtains the image parameter set ID of the slice header (S508). That is, the filter control unit 131 obtains the image parameter set ID contained in the slice header of the encoded object slice.

[0547] Next, the encoding device 100 sets filter information based on the image parameter set ID (S509). For example, the filter control unit 131 selects a filter information set associated with the image parameter set ID of the title sequence from a plurality of filter information sets stored in the reference filter information storage unit 133. Then, the filter control unit 131 sets the filter information set by storing the selected filter information set in the current filter information storage unit 132.

[0548] Then, a loop of processing for each CU is performed (S510). That is, the encoding device 100 performs encoding processing on each CU. At this time, the encoding device 100 applies an adaptive cyclic filter.

[0549] By performing the aforementioned operations, the encoding device 100 can store the filter information set based on PPS filter control information in the reference filter information storage unit 133. Furthermore, the encoding device 100 can specifically select a filter information set based on the image parameter set ID of the title sequence from among the multiple filter information sets in the reference filter information storage unit 133. Then, the encoding device 100 can set the specific filter information set in the current filter information storage unit 132.

[0550] Figure 17 This is a flowchart of the second specific example of the filter information processing steps in the deformation mode. Figure 1 The encoding device 100 shown can perform Figure 17 The actions shown.

[0551] In this example, firstly, the encoding device 100 and Figure 16 The process shown (S501) similarly determines whether to perform PPS processing (S601). If it is determined that PPS processing should be performed ("Yes" in S601), the encoding device 100... Figure 16 The process shown (S502) also performs PPS processing (S602). On the other hand, if the encoding device 100 determines that PPS processing should not be performed ("No" in S601), it skips PPS processing.

[0552] Next, the encoding device 100 performs slice header processing (S603). For example, the entropy encoding unit 110 generates the slice header of the current slice of the encoding object and encodes it.

[0553] Next, the encoding device 100 processes the filter control information (S604). For example, the filter control unit 131 generates and outputs filter control information. Furthermore, the entropy encoding unit 110 encodes the filter control information output from the filter control unit 131.

[0554] Filter control information can be included in the PPS. Therefore, the processing of filter control information (S604) can be included in the PPS processing (S602). Furthermore, in the slice header processing (S603), a slice header containing an image parameter set ID can also be generated and encoded, where the image parameter set ID contains the PPS of filter control information corresponding to the current slice.

[0555] Then, a loop of processing for each CU is performed (S605). That is, the encoding device 100 performs encoding processing on each CU.

[0556] Next, the encoding device 100 and Figure 16 The process shown (S503) similarly obtains the NAL unit type (S606). Next, the encoding device 100 and... Figure 16 The process shown (S504) similarly determines whether the encoded object image is an IDR image (S607).

[0557] If the image to be encoded is determined to be an IDR image ("Yes" in S607), the encoding device 100 and Figure 16 The process shown (S505) similarly deletes the saved filter information (S608).

[0558] If it is determined that the image to be encoded is not an IDR image ("No" in S504), the encoding device 100 and Figure 16 The process shown (S506) similarly determines whether PPS processing has been performed (S602)(S609).

[0559] Then, if it is determined that PPS processing (S602) has been performed ("Yes" in S609), the encoding device 100 saves the PPS setting filter information based on the PPS processing (S602) (S610).

[0560] Furthermore, assuming that the image to be encoded is determined to be an IDR image ("Yes" in S607), PPS processing is performed (S602). Therefore, in this case, after the filter information is deleted, the encoding device 100 does not determine whether PPS processing has been performed, but sets and saves the filter information based on the PPS in the PPS processing (S602) (S610).

[0561] For example, the filter control unit 131 specifies a filter information set based on the filter control information contained in the PPS encoded in the PPS processing (S602). Then, the filter control unit 131 stores the specific filter information set in the current filter information storage unit 132 and the reference filter information storage unit 133, respectively.

[0562] Furthermore, in the reference filter information storage unit 133, the filter control unit 131 associates and manages a filter information set specific to the filter control information contained in the PPS with the image parameter set ID of the PPS. That is, the filter information set is saved in association with the image parameter set ID.

[0563] Furthermore, when processing multiple PPSs, for each of the multiple PPSs, a filter information set corresponding to that PPS is specified and stored based on the filter control information of that PPS. Thus, for example, multiple filter information sets corresponding to each of the consecutively inserted PPSs are specified and stored.

[0564] On the other hand, if it is determined that PPS processing has not been performed (S602) (No in S609), the encoding device 100 and Figure 16 The process shown (S508) similarly obtains the image parameter set ID of the title sequence (S611). Next, the encoding device 100 and Figure 16 The process shown (S509) similarly sets the filter information based on the image parameter set ID (S612).

[0565] Then, the encoding device 100 applies an adaptive cyclic filter (S613). Specifically, the adaptive filter unit 134 uses the filter information set stored in the current filter information storage unit 132 to apply an adaptive cyclic filter to the current slice.

[0566] By performing the aforementioned actions, the encoding device 100 can set filter information and apply an adaptive cyclic filter after processing by the CU. Therefore, the encoding device 100 can appropriately generate a reconstructed image before setting the filter information. Thus, the encoding device 100 can set the filter information based on the input image and the reconstructed image.

[0567] Furthermore, the entropy encoding unit 110 can append filter information based on the input image and the reconstructed image to the slice data instead of the PPS for encoding. Moreover, the entropy encoding unit 110 can encode the PPS containing filter control information indicating the use of filter information appended to the slice data.

[0568] Regarding the encoding device 100, using Figure 16 and Figure 17 The actions described can be explained as actions related to the decoding device 200 by replacing encoding with decoding.

[0569] For example, the decoding device 200 performs with Figure 16 The action shown corresponds to the action. It can be based on... Figure 16 Explanation of decoding device 200 and Figure 16 The actions shown correspond to the actions performed.

[0570] First, the decoding device 200 determines whether to perform PPS processing (S501). For example, the entropy decoding unit 202 determines whether to decode the PPS. The entropy decoding unit 202 can determine whether to decode the PPS according to the encoded stream. Specifically, if the PPS is encoded in the encoded stream, the entropy decoding unit 202 can determine that the PPS should be decoded.

[0571] If the decoding device 200 determines that PPS processing should be performed ("Yes" in S501), it performs PPS processing (S502). For example, if the entropy decoding unit 202 determines that PPS should be decoded, it decodes PPS. On the other hand, if the decoding device 200 determines that PPS processing should not be performed ("No" in S501), it skips PPS processing.

[0572] Next, the decoding device 200 obtains the NAL unit type (S503). For example, the entropy decoding unit 202 obtains the NAL unit type of the image to be decoded. More specifically, the entropy decoding unit 202 obtains the NAL unit type of the decoded object slice in the image to be decoded. The NAL unit type of the decoded object slice in the image to be decoded corresponds to the type of the image to be decoded.

[0573] Next, the decoding device 200 determines whether the image to be decoded is an IDR image (S504). For example, the filter control unit 231 determines whether the image to be decoded is an IDR image according to the NAL unit type of the image to be decoded.

[0574] If the image to be decoded is determined to be an IDR image ("Yes" in S504), the decoding device 200 deletes the stored filter information (S505). For example, the filter control unit 231 deletes multiple filter information sets stored in the reference filter information storage unit 233.

[0575] If the image to be decoded is determined to be not an IDR image ("No" in S504), the decoding device 200 determines whether PPS processing has been performed (S502) (S506). For example, the filter control unit 231 determines whether PPS has been decoded.

[0576] Then, if it is determined that PPS processing (S502) has been performed ("Yes" in S506), the decoding device 200 saves the filter information based on the PPS in the PPS processing (S502) (S507). On the other hand, if it is determined that PPS processing (S502) has not been performed ("No" in S506), the decoding device 200 skips the process of saving the filter information (S507).

[0577] Furthermore, assuming that the image to be decoded is determined to be an IDR image ("Yes" in S504), PPS processing is performed (S502). Therefore, in this case, after the filter information is deleted, the decoding device 200 does not determine whether PPS processing has been performed, and saves the filter information based on the PPS in the PPS processing (S502).

[0578] For example, the filter control unit 231 specifies a filter information set based on the filter control information contained in the PPS decoded in the PPS processing (S502), and stores the specific filter information set in the reference filter information storage unit 233. Furthermore, the filter control unit 231 associates the filter information set specified based on the filter control information contained in the PPS with the image parameter set ID of the PPS and manages it accordingly. That is, the filter information set is stored in association with the image parameter set ID.

[0579] Furthermore, when processing multiple PPSs, for each of the multiple PPSs, a filter information set corresponding to that PPS is specified and stored based on the filter control information of that PPS. Thus, for example, multiple filter information sets corresponding to each of the consecutively inserted PPSs are specified and stored.

[0580] Next, the decoding device 200 obtains the image parameter set ID of the slice header (S508). That is, the filter control unit 231 obtains the image parameter set ID contained in the slice header of the decoded target slice.

[0581] Next, the decoding device 200 sets the filter information based on the image parameter set ID (S509). For example, the filter control unit 231 selects a filter information set associated with the image parameter set ID of the title sequence from among multiple filter information sets stored in the reference filter information storage unit 233. Then, the filter control unit 231 sets the filter information set by storing the selected filter information set in the current filter information storage unit 232.

[0582] Then, a loop of processing for each CU is performed (S510). That is, the decoding device 200 performs decoding processing on each CU. At this time, the decoding device 200 applies an adaptive cyclic filter.

[0583] By performing the aforementioned operations, the decoding device 200 can store the filter information set based on PPS filter control information in the reference filter information storage unit 233. Furthermore, the decoding device 200 can select a specific filter information set based on the image parameter set ID of the title sequence from among multiple filter information sets in the reference filter information storage unit 233. Moreover, the decoding device 200 can set a specific filter information set in the current filter information storage unit 232.

[0584] In addition, the decoding device 200 can also perform with Figure 17 The action shown corresponds to the action. It can be based on... Figure 17 Explanation and Figure 17 The actions shown correspond to actions that can be performed by the decoding device 200.

[0585] First, the decoding device 200 and Figure 16 The process shown (S501) similarly determines whether to perform PPS processing (S601). If it is determined that PPS processing should be performed ("Yes" in S601), the decoding device 200, along with... Figure 16 The process shown (S502) also performs PPS processing (S602). On the other hand, if the decoding device 200 determines that PPS processing should not be performed ("No" in S601), it skips PPS processing.

[0586] Next, the decoding device 200 processes the slice header (S603). For example, the entropy decoding unit 202 analyzes the slice header of the current slice of the decoding object and performs decoding.

[0587] Next, the decoding device 200 processes the filter control information (S604). For example, the entropy decoding unit 202 decodes the filter control information, and the filter control unit 231 acquires the decoded filter control information.

[0588] Filter control information can be included in the PPS. Therefore, the processing of filter control information (S604) can be included in the PPS processing (S602). Furthermore, in the slice header processing (S603), the slice header containing the image parameter set ID, which contains the PPS of filter control information corresponding to the current slice, can also be decoded.

[0589] Then, a loop of processing for each CU is performed (S605). That is, the decoding device 200 performs decoding processing on each CU.

[0590] Next, the decoding device 200 and Figure 16 The process shown (S503) similarly obtains the NAL unit type (S606). Next, the decoding device 200 and... Figure 16 The process shown (S504) similarly determines whether the image to be decoded is an IDR image (S607).

[0591] If the image to be decoded is determined to be an IDR image ("Yes" in S607), the decoding device 200 and Figure 16 The process shown (S505) similarly deletes the saved filter information (S608).

[0592] If it is determined that the image to be decoded is not an IDR image ("No" in S607), the decoding device 200 and Figure 16 The process shown (S506) similarly determines whether PPS processing has been performed (S602)(S609).

[0593] Then, if it is determined that PPS processing (S602) has been performed ("Yes" in S609), the decoding device 200 saves the PPS setting filter information based on the PPS processing (S602) (S610).

[0594] Furthermore, assuming that the image to be decoded is determined to be an IDR image ("Yes" in S607), PPS processing is performed (S602). Therefore, in this case, after the filter information is deleted, the decoding device 200 does not determine whether PPS processing has been performed, but sets and saves the filter information based on the PPS in the PPS processing (S602) (S610).

[0595] For example, the filter control unit 231 selects a specific filter information set based on the filter control information contained in the PPS decoded in the PPS processing (S602). Then, the filter control unit 231 stores the specific filter information set in the current filter information storage unit 232 and the reference filter information storage unit 233, respectively.

[0596] Furthermore, in the reference filter information storage unit 233, the filter control unit 231 establishes and manages a specific filter information set based on the filter control information contained in the PPS with the image parameter set ID of the PPS. That is, the filter information set is saved in association with the image parameter set ID.

[0597] Furthermore, when processing multiple PPSs, for each of the multiple PPSs, a filter information set corresponding to that PPS is specified and stored based on the filter control information of that PPS. Thus, for example, multiple filter information sets corresponding to each of the consecutively inserted PPSs are specified and stored.

[0598] On the other hand, if it is determined that PPS processing has not been performed (S602) (No in S609), the decoding device 200 and Figure 16 The process shown (S508) similarly obtains the image parameter set ID of the title sequence (S611). Next, the decoding device 200 and... Figure 16 The process shown (S509) similarly sets the filter information based on the image parameter set ID (S612).

[0599] Then, the decoding device 200 applies an adaptive cyclic filter (S613). Specifically, the adaptive filter unit 234 uses the filter information set stored in the current filter information storage unit 232 to apply an adaptive cyclic filter to the current slice.

[0600] By performing the aforementioned actions, the decoding device 200 can set filter information and apply an adaptive cyclic filter after the CU processing. Therefore, the decoding device 200 can appropriately generate the reproduced image before setting the filter information. Thus, the decoding device 200 can set the filter information based on the input image and the reproduced image.

[0601] Furthermore, filter information set based on the input image and the reconstructed image can be appended to the slice data instead of to the PPS. Moreover, filter control information can indicate the use of filter information appended to the slice data. The entropy decoding unit 202 can decode such filter information and filter control information.

[0602] [PPS Notification]

[0603] For example, the PPS containing filter control information is notified from the encoding device 100 to the decoding device 200. That is, the entropy encoding unit 110 of the encoding device 100 encodes the PPS containing filter control information, and the entropy decoding unit 202 of the decoding device 200 decodes the PPS containing filter control information. Thus, the filter control information is shared between the encoding device 100 and the decoding device 200.

[0604] Additionally, time IDs representing hierarchical levels related to temporal scalability may be assigned to each of the multiple images contained within the motion image. Furthermore, time IDs representing hierarchical levels related to temporal scalability may also be assigned to the PPS.

[0605] Furthermore, for example, multiple PPSs can each correspond to multiple levels related to temporal scalability, and each of the multiple PPSs can be a PPS belonging to more than one image at the level corresponding to that PPS. In other words, for each of the multiple time IDs assigned to multiple images, the multiple PPSs can consist of PPSs assigned to more than one image at that time ID.

[0606] Here, PPS for more than one image means a public PPS applied to more than one image. Furthermore, PPS corresponding to a level means a PPS assigned to more than one image belonging to that level, that is, a PPS assigned to more than one image with a time ID representing that level. In other words, PPS corresponding to a level here means that the PPS is a public PPS applied to more than one image with a time ID representing that level.

[0607] Furthermore, the level corresponding to PPS and the level indicated by the time ID assigned to PPS can be different. These levels can be determined independently and unrelated to each other. Additionally, the level corresponding to PPS can also be a level related to PPS.

[0608] Furthermore, for example, the correspondence between a PPS and its corresponding layer is established through the image parameter set ID contained in the caption of a slice within an image belonging to that layer. Essentially, the caption of a slice within an image contains the image parameter set ID of the PPS corresponding to the layer to which that image belongs.

[0609] The following describes several specific examples related to PPS notification in variations of the filter information processing method.

[0610] Figure 18 This is a conceptual diagram representing the first specific instance of the PPS notification in the variant mode. Figure 18 In, with Figure 14A Similarly, images p0 to p8 are shown. Figure 18The example shows the encoding order of images p0 to p8 and the time IDs assigned to images p0 to p8 respectively. Figure 14A The examples are the same.

[0611] and, Figure 18 The diagram shows PPS0 to PPS3. Here, PPSx is the PPS referenced by time layer x. Furthermore, x is a time ID, and time layer x includes images assigned x as their time ID. Specifically, PPS0 is the PPS referenced in time layer 0. That is, PPS0 is the PPS for the image assigned 0 as its time ID.

[0612] Similarly, PPS1 is a PPS referenced in time layer 1. That is, PPS1 is the PPS for the image assigned time ID 1. Likewise, PPS2 is a PPS referenced in time layer 2. That is, PPS2 is the PPS for the image assigned time ID 2. Similarly, PPS3 is a PPS referenced in time layer 3. That is, PPS3 is the PPS for the image assigned time ID 3.

[0613] exist Figure 18 In this process, x is assigned to PPSx as a time ID. Specifically, 0 is assigned to PPS0 as a time ID. Similarly, 1 is assigned to PPS1 as a time ID, 2 to PPS2 as a time ID, and 3 to PPS3 as a time ID.

[0614] In addition, Figure 18 In this process, the encoding device 100 encodes PPS0 to PPS3 before encoding images p0 to p8. Then, the encoding device 100 encodes images p0 and p1 according to PPS0. Furthermore, the encoding device 100 encodes image p2 according to PPS1. Furthermore, the encoding device 100 encodes images p3 and p6 according to PPS2. Furthermore, the encoding device 100 encodes images p4, p5, p7, and p8 according to PPS3.

[0615] Similarly, the decoding device 200 decodes PPS 0 to PPS 3 before decoding images p0 to p8. Then, the decoding device 200 decodes images p0 and p1 according to PPS 0. Additionally, the decoding device 200 decodes image p2 according to PPS 1. Furthermore, the decoding device 200 decodes images p3 and p6 according to PPS 2. Moreover, the decoding device 200 decodes images p4, p5, p7, and p8 according to PPS 3.

[0616] Alternatively, the decoding device 200 may decode only a portion of the images p0 to p8 whose time ID is below a predetermined value. For example, when the predetermined value is 1, the decoding device 200 decodes images p0 and p1 according to PPS0, and decodes image p2 according to PPS1.

[0617] For example, when decoding only a subset of images p0 to p8 whose time IDs are below a predetermined value, the decoding device 200 may discard images p0 to p8 whose time IDs are greater than the predetermined value without decoding them. In this case, the decoding device 200 may also discard images whose time IDs are greater than the predetermined value without decoding them. Since such images with large time IDs may not be decoded and are discarded, it is difficult to appropriately increase the time ID.

[0618] For example, if the decoding device 200 decodes only images p0 to p2 (whose time ID is less than 1) from images p0 to p8, it decodes PPS0 and PPS1, but discards PP2 and PPS3. On the other hand, in this state, it is difficult to perform an upward shift, such as decoding image p6 after decoding image p2. That is, in Figure 18 In the example, proper upward movement is difficult.

[0619] Figure 19 This is a conceptual diagram representing the second specific instance of the PPS notification in the variant mode. Figure 19 In, with Figure 18 Similarly, images p0 to p8 are shown. Figure 19 The example shows the encoding order of images p0 to p8 and the time IDs assigned to images p0 to p8. Figure 18 The examples are the same. Moreover, Figure 19 PPS0 to PPS3 are shown. Here, PPSx is related to... Figure 18 Similarly, the example is the PPS referenced in time layer x.

[0620] exist Figure 19 In the example, 0 is assigned to each of PPS0 to PPS3 as a time ID.

[0621] In addition, Figure 19 In the example, with Figure 18 Similarly, the encoding device 100 encodes PPS0 to PPS3 before encoding images p0 to p8. Then, the encoding device 100 encodes images p0 and p1 according to PPS0, images p2 according to PPS1, images p3 and p6 according to PPS2, and images p4, p5, p7, and p8 according to PPS3.

[0622] Similarly, the decoding device 200 decodes PPS0 to PPS3 before decoding images p0 to p8. Then, the decoding device 200 decodes images p0 and p1 according to PPS0, images p2 according to PPS1, images p3 and p6 according to PPS2, and images p4, p5, p7, and p8 according to PPS3.

[0623] Alternatively, the decoding device 200 may decode only a portion of the images p0 to p8 whose time ID is below a predetermined value. For example, when the predetermined value is 1, the decoding device 200 decodes images p0 and p1 according to PPS0, and decodes image p2 according to PPS1.

[0624] Furthermore, for example, if the decoding device 200 only decodes a subset of images p0 to p8 whose time ID is below a predetermined value, it may discard images p0 to p8 whose time ID is greater than the predetermined value. Even in this case, the decoding device 200 decodes PPS0 to PPS3 whose time ID is 0. Therefore, appropriate upward shift can be achieved.

[0625] Specifically, for example, even when only images p0 to p2 with a time ID of 1 or less from images p0 to p8 are decoded, the decoding device 200 also decodes PPS0 to PPS3. Therefore, after decoding image p2, image p6 can be decoded according to PPS2. That is, in Figure 19 In the example, an appropriate upward shift can be achieved.

[0626] Figure 20A This is a conceptual diagram representing the third specific instance of the PPS notification in the variant mode. Figure 20A In, with Figure 18 Similarly, images p0 to p8 are shown. Figure 20A The example shows the encoding order of images p0 to p8 and the time IDs assigned to images p0 to p8 respectively. Figure 18 The examples are the same. Moreover, Figure 20A PPS0 to PPS3 are shown. Here, PPSx is related to... Figure 18 Similarly, the example is the PPS referenced in time layer x.

[0627] In addition, Figure 20A In the example, with Figure 18Similarly, before encoding images p0 to p8, the encoding device 100 encodes PPS0 to PPS3. x is assigned as a time ID to the PPSx being encoded at this time. Specifically, 0 is assigned as a time ID to PPS0. Likewise, 1 is assigned as a time ID to PPS1, 2 to PPS2, and 3 to PPS3.

[0628] Then, the encoding device 100 encodes images p0 and p1 according to PPS0, images p2 according to PPS1, images p3 and p6 according to PPS2, and images p4, p5, p7 and p8 according to PPS3.

[0629] In addition, Figure 20A In this example, before encoding the TSA image, the encoding device 100 encodes one or more PPSs referenced in one or more time layers above the time ID assigned to the TSA image. At this time, the one or more PPSs being encoded can be assigned the same time ID as the time ID assigned to the TSA image.

[0630] Specifically, in Figure 20A In the example, image p6, which is assigned a time ID of 2, is a TSA image. In this case, before encoding image p6, the encoding device 100 encodes PPS2 and PPS3, which are referenced in time layer 2 and time layer 3, respectively. At this time, the encoded PPS2 and PPS3 can be assigned a time ID of 2.

[0631] Similarly, before decoding images p0 to p8, the decoding device 200 decodes PPS0 to PPS3. At this time, x is assigned as a time ID to the PPSx being decoded. Then, the decoding device 200 decodes images p0 and p1 according to PPS0, images p2 according to PPS1, images p3 and p6 according to PPS2, and images p4, p5, p7, and p8 according to PPS3.

[0632] Furthermore, before decoding the TSA image, the decoding device 200 decodes one or more PPSs referenced in one or more time layers above the time ID assigned to the TSA image. At this time, the decoded PPSs may be assigned the same time ID as the time ID assigned to the TSA image.

[0633] Specifically, in Figure 20AIn the example, image p6, which is assigned a time ID of 2, is a TSA image. Therefore, before decoding image p6, the decoding device 200 decodes PPS2 and PPS3, which are referenced in time layer 2 and time layer 3, respectively. At this time, the decoded PPS2 and PPS3 can be assigned a time ID of 2.

[0634] For example, the decoding device 200 only decodes images p0 to p2 among images p0 to p8 whose time ID is less than 1. Therefore, before decoding image p0, it can decode only PPS0 and PPS1 among PPS0 to PPS3. Then, the decoding device 200 can discard PPS2 and PPS3 among PPS0 to PPS3 without decoding them before decoding image p0.

[0635] Then, the decoding device 200 decodes image p6, which is a TSA image in time layer 2, and therefore decodes PPS2, which is referenced in time layer 2. Furthermore, the decoding device 200 decodes PPS3, which is referenced in time layer 3.

[0636] Therefore, the decoding device 200 can decode image p6 according to PPS2 after image p2, and can decode images p7 and p8, which have a time ID of 3, according to PP3.

[0637] That is, when decoding a TSA image, the decoding device 200 can appropriately move upwards according to the allowable range of the TSA image.

[0638] Furthermore, during the upward shift from a lower to a higher time layer, the images and PPS corresponding to the higher time layer are more likely to be decoded. Therefore, the time ID of the time layer referenced by the PPS can be assigned to the PPS processed before the TSA image.

[0639] For example, in Figure 20A In the example, for PPS2 and PP3, which are processed before image p6, 2 can be assigned to PPS2 as a time ID, and 3 can be assigned to PPS3 as a time ID.

[0640] Figure 20B This is a conceptual diagram representing the fourth specific instance of the PPS notification in the variant mode. Figure 20B In, with Figure 20A Similarly, images p0 to p8 are shown. Figure 20B The example shows the encoding order of images p0 to p8 and the time IDs assigned to images p0 to p8 respectively. Figure 20A The examples are the same. Moreover, Figure 20B PPS0 to PPS3 are shown. Here, PPSx is related to... Figure 20ASimilarly, the example is the PPS referenced in time layer x.

[0641] In addition, Figure 20B In the example, with Figure 20A Similarly, before encoding images p0 to p8, the encoding device 100 encodes PPS0 to PPS3. At this time, x is assigned to the encoded PPSx as a time ID. Then, the encoding device 100 encodes images p0 and p1 according to PPS0, images p2 according to PPS1, images p3 and p6 according to PPS2, and images p4, p5, p7, and p8 according to PPS3.

[0642] In addition, Figure 20B In this example, before encoding the STSA image, the encoding device 100 encodes the PPS referenced in the time layer of the time ID assigned to the STSA image. At this time, the encoded PPS is assigned the same time ID as the time ID assigned to the STSA image.

[0643] Specifically, in Figure 20B In the example, image p6, which is assigned a time ID of 2, is an STSA image. In this case, the encoding device 100 encodes PPS2, which is referenced in time layer 2, before encoding image p6. At this time, the encoded PPS2 is also assigned a time ID of 2.

[0644] Similarly, before decoding images p0 to p8, the decoding device 200 decodes PPS0 to PPS3. At this time, x is assigned as a time ID to the PPSx being decoded. Then, the decoding device 200 decodes images p0 and p1 according to PPS0, images p2 according to PPS1, images p3 and p6 according to PPS2, and images p4, p5, p7, and p8 according to PPS3.

[0645] Furthermore, before decoding the STSA image, the decoding device 200 decodes the PPS referenced in the time layer of the time ID assigned to the STSA image. At this time, the decoded PPS is also assigned the same time ID as the time ID assigned to the STSA image.

[0646] Specifically, in Figure 20B In the example, image p6, which is assigned a time ID of 2, is an STSA image. Therefore, the decoding device 200 decodes PPS2, which is referenced in time layer 2, before decoding image p6. At this time, the encoded PPS2 is also assigned a time ID of 2.

[0647] For example, the decoding device 200 only decodes images p0 to p2, which have a time ID of 1 or less among images p0 to p8. Therefore, before decoding image p0, it can decode only PPS0 and PPS1 among PPS0 to PPS3. Then, the decoding device 200 can discard PPS2 and PPS3 among PPS0 to PPS3 without decoding them before decoding image p0.

[0648] Then, the decoding device 200 decodes image p6, which is an STSA image with time ID 2, and thus decodes PPS2, which is referenced in time layer 2.

[0649] Therefore, the decoding device 200 is able to decode image p6 according to PPS2 after image p2.

[0650] That is, when decoding an STSA image, the decoding device 200 can appropriately shift upwards according to the allowable range of the STSA image.

[0651] Figure 21A This is a conceptual diagram representing the fifth specific instance of the PPS notification in the variant mode. Figure 21A In, with Figure 18 Similarly, images p0 to p8 are shown. Figure 21A The example shows the encoding order of images p0 to p8 and the time IDs assigned to images p0 to p8 respectively. Figure 18 The examples are the same. Moreover, Figure 21A PPS0 to PPS3 are shown. Here, PPSx is related to... Figure 18 Similarly, the example is the PPS referenced in time layer x.

[0652] exist Figure 21A In the example, each of the images p2 to p8 with a time ID greater than 0 is a TSA image. Each of the images p2 to p8 with a time ID greater than 0 can be designated as a TSA image according to the flags included in the sequence parameter set. For example, in HEVC, when sps_temporal_id_nesting_flag is 1, each of the images p2 to p8 with a time ID greater than 0 is designated as a TSA image.

[0653] In addition, Figure 21A In the example, before encoding images p0 to p8, the encoding device 100 encodes PPS0, which is referenced in time layer 0. At this time, the encoded PPS0 is assigned 0 as a time ID. Then, the encoding device 100 encodes images p0 and p1 according to PPS0.

[0654] In addition, Figure 21AIn this example, the encoding device 100 encodes the PPS (Programmable Pixel Scale) for the TSA image before encoding the TSA image itself. Here, the PPS for the TSA image is assigned the same time ID as the time ID assigned to the TSA image. Then, after encoding the PPS for the TSA image, the encoding device 100 encodes the TSA image according to the PPS for the TSA image.

[0655] For example, the encoding device 100 encodes PPS1 for image p2 before encoding image p2. Here, PPS1 for image p2 is assigned the same time ID as the time ID assigned to image p2, which is 1. Then, after encoding PPS1 for image p2, the encoding device 100 encodes image p2 according to PPS1 for image p2. The processing for other images p3 to p8 is the same as the processing for image p2.

[0656] Similarly, before decoding images p0 to p8, the decoding device 200 decodes PPS0, which is referenced in time layer 0. At this time, the decoded PPS0 is assigned 0 as a time ID. Then, the decoding device 200 decodes images p0 and p1 according to PPS0.

[0657] Furthermore, the decoding device 200 decodes the PPS (Programmable Status Table) for the TSA image before decoding the TSA image itself. Here, the PPS for the TSA image is assigned the same time ID as the time ID assigned to the TSA image. Then, after decoding the PPS for the TSA image, the decoding device 200 decodes the TSA image according to the PPS for the TSA image.

[0658] For example, before decoding image p2, the decoding device 200 decodes PPS1 for image p2. Here, PPS1 for image p2 is assigned the same time ID as the time ID assigned to image p2, which is 1. Then, after decoding PPS1 for image p2, the decoding device 200 decodes image p2 according to PPS1 for image p2. The processing for other images p3 to p8 is the same as the processing for image p2.

[0659] Alternatively, the decoding device 200 may decode only a portion of the images p0 to p8 whose time ID is below a predetermined value. Then, the decoding device 200 may discard images p0 to p8 whose time ID is greater than the predetermined value without decoding. Similarly, the decoding device 200 may decode only a portion of the PPS images pS0 to PPS3 whose time ID is below the predetermined value. Then, the decoding device 200 may discard PPS images pS0 to PPS3 whose time ID is greater than the predetermined value without decoding.

[0660] For example, when the specified value is 1, the decoding device 200 decodes PPS0 and decodes images p0 and p1 according to PPS0. Furthermore, the decoding device 200 decodes PPS1 for image p2 before decoding image p2. Then, after decoding PPS1 for image p2, the decoding device 200 decodes image p2 according to PPS1 for image p2. Then, the decoding device 200 does not decode other images and PPSs with a time ID greater than 1 and discards them.

[0661] During the aforementioned actions, it is possible to move upwards. For example, the specified value may be changed to a value greater than 1.

[0662] Specifically, for example, after decoding image p2, it is possible to move the image upwards so that image p6 can be decoded. Even with such upward movement, the decoding device 200 can still decode PPS2 for image p6 before decoding image p6. Therefore, the decoding device 200 can appropriately decode image p6 according to PPS2 for image p6. Thus, the decoding device 200 is capable of such upward movement.

[0663] That is, even with this upward movement, the decoding device 200 can still decode the PPS for each TSA image before decoding each TSA image. Therefore, the decoding device 200 can appropriately decode the TSA image according to the PPS for that TSA image. Therefore, the decoding device 200 can appropriately move upward.

[0664] Figure 21B This is a conceptual diagram representing the sixth specific instance of the PPS notification in the variant mode. Figure 21B In, with Figure 21A Similarly, images p0 to p8 are shown. Figure 21B The example shows the encoding order of images p0 to p8 and the time IDs assigned to images p0 to p8 respectively. Figure 21A The examples are the same. Moreover, Figure 21B PPS0 to PPS3 are shown. Here, PPSx is related to... Figure 21A Similarly, the example is the PPS referenced in time layer x.

[0665] exist Figure 21B In the example, with Figure 21A Similarly, each of the images p2 to p8 with a time ID greater than 0 is a TSA image.

[0666] In addition, Figure 21B In the example, with Figure 21ASimilarly, before encoding images p0 to p8, the encoding device 100 encodes PPS0, which is referenced in time layer 0. At this time, the encoded PPS0 is assigned 0 as a time ID. Then, the encoding device 100 encodes images p0 and p1 according to PPS0.

[0667] In addition, Figure 21B In this example, the encoding device 100 encodes the PPS for the intermediate layer image before encoding the intermediate layer image. Here, the PPS for the intermediate layer image is assigned the same time ID as the time ID assigned to the intermediate layer image. Then, after encoding the PPS for the intermediate layer image, the encoding device 100 encodes the intermediate layer image according to the PPS for the intermediate layer image.

[0668] Here, the middle layer images are images assigned time IDs that are greater than the minimum time ID and less than the maximum time ID.

[0669] In addition, Figure 21B In this example, the encoding device 100 encodes the PPS for the top-layer image before encoding the specific intermediate layer image. Here, the top-layer image is the image assigned the largest time ID. The specific intermediate layer image is the intermediate layer image assigned a time ID that is second only to the largest time ID.

[0670] That is, the encoding device 100 encodes the PPS for the specific intermediate layer image and the PPS for the top layer image before encoding the specific intermediate layer image. Here, each of the PPS for the specific intermediate layer image and the PPS for the top layer image can be assigned the same time ID as the time ID assigned to the specific intermediate layer image.

[0671] Then, after encoding the PPS for a specific intermediate layer image, the encoding device 100 encodes the specific intermediate layer image according to the PPS for the specific intermediate layer image. Furthermore, after encoding the PPS for the top layer image, the encoding device 100 encodes the top layer image according to the PPS for the top layer image.

[0672] Specifically, images p2, p3, and p6 in images p0–p8 are intermediate layer images. Furthermore, images p3 and p6 in images p2, p3, and p6 are specific intermediate layer images. Additionally, images p4, p5, p7, and p8 in images p0–p8 are the top layer images.

[0673] For example, the encoding device 100 encodes PPS1 before encoding image p2. Here, PPS1 is assigned the same time ID as the time ID assigned to image p2, which is 1. Then, the encoding device 100 encodes image p2 according to PPS1 after encoding PPS1.

[0674] Furthermore, the encoding device 100 encodes PPS2 and PPS3 before encoding image p3. Here, PPS2 and PPS3 are each assigned a time ID of 2, which is the same as the time ID assigned to image p3. Then, after encoding PPS2, the encoding device 100 encodes image p3 according to PPS2. Additionally, after encoding PPS3, the encoding device 100 encodes images p4 and p5 according to PPS3.

[0675] Furthermore, the encoding device 100 encodes PPS2 and PPS3 before encoding image p6. Here, PPS2 and PPS3 are each assigned a time ID of 2, which is the same as the time ID assigned to image p6. Then, after encoding PPS2, the encoding device 100 encodes image p6 according to PPS2. Additionally, after encoding PPS3, the encoding device 100 encodes images p7 and p8 according to PPS3.

[0676] Similarly, before decoding images p0 to p8, the decoding device 200 decodes PPS0, which is referenced in time layer 0. At this time, the decoded PPS0 is assigned 0 as a time ID. Then, the decoding device 200 decodes images p0 and p1 according to PPS0.

[0677] Furthermore, the decoding device 200 decodes the PPS for the intermediate layer image before decoding the intermediate layer image. Here, the PPS for the intermediate layer image is assigned the same time ID as the time ID assigned to the intermediate layer image. Then, after decoding the PPS for the intermediate layer image, the decoding device 200 decodes the intermediate layer image according to the PPS for the intermediate layer image.

[0678] Here, the middle layer images are images assigned time IDs that are greater than the minimum time ID and less than the maximum time ID.

[0679] Furthermore, before decoding a specific intermediate layer image, the decoding device 200 decodes the PPS for the top layer image. Here, the top layer image is the image assigned the largest time ID. The specific intermediate layer image is the intermediate layer image assigned a time ID that is second only to the largest time ID.

[0680] That is, before decoding a specific intermediate layer image, the decoding device 200 decodes the PPS for the specific intermediate layer image and the PPS for the top layer image. Here, the PPS for the specific intermediate layer image and the PPS for the top layer image can each be assigned the same time ID as the time ID assigned to the specific intermediate layer image.

[0681] Then, after decoding the PPS for a specific intermediate layer image, the decoding device 200 decodes the specific intermediate layer image according to the PPS for that specific intermediate layer image. Furthermore, after decoding the PPS for the top layer image, the decoding device 200 decodes the top layer image according to the PPS for that top layer image.

[0682] Specifically, as mentioned above, images p2, p3, and p6 in images p0 to p8 are intermediate layer images. Furthermore, images p3 and p6 in images p2, p3, and p6 are specific intermediate layer images. Additionally, images p4, p5, p7, and p8 in images p0 to p8 are the topmost layer images.

[0683] For example, the decoding device 200 decodes PPS1 before decoding image p2. Here, PPS1 is assigned the same time ID as the time ID assigned to image p2, which is 1. Then, after decoding PPS1, the decoding device 200 decodes image p2 according to PPS1.

[0684] Furthermore, the decoding device 200 decodes PPS2 and PPS3 before decoding image p3. Here, PPS2 and PPS3 are each assigned a time ID of 2, which is the same as the time ID assigned to image p3. Then, after decoding PPS2, the decoding device 200 decodes image p3 according to PPS2. Furthermore, after decoding PPS3, the decoding device 200 decodes images p4 and p5 according to PPS3.

[0685] Furthermore, the decoding device 200 decodes PPS2 and PPS3 before decoding image p6. Here, PPS2 and PPS3 are each assigned a time ID of 2, which is the same as the time ID assigned to image p6. Then, after decoding PPS2, the decoding device 200 decodes image p6 according to PPS2. Furthermore, after decoding PPS3, the decoding device 200 decodes images p7 and p8 according to PPS3.

[0686] Furthermore, the decoding device 200 may also decode only a portion of the images p0 to p8 whose time ID is below a predetermined value. Moreover, the decoding device 200 may discard images p0 to p8 whose time ID is greater than the predetermined value without decoding. Additionally, the decoding device 200 may also decode only a portion of the PPS images pS0 to PPS3 whose time ID is below the predetermined value. Then, the decoding device 200 may discard PPS images pS0 to PPS3 whose time ID is greater than the predetermined value without decoding.

[0687] For example, when the specified value is 1, the decoding device 200 decodes PPS0 and decodes images p0 and p1 according to PPS0. Furthermore, the decoding device 200 decodes PPS1 for image p2 before decoding image p2. Then, after decoding PPS1 for image p2, the decoding device 200 decodes image p2 according to PPS1 for image p2. Then, the decoding device 200 does not decode other images and PPSs with a time ID greater than 1 and discards them.

[0688] During the aforementioned actions, it is possible to move upwards. For example, the specified value may be changed to a value greater than 1.

[0689] Specifically, for example, after decoding image p2, it is possible to move the image upwards so that image p6 can be decoded. Even with such upward movement, the decoding device 200 can still decode PPS2 for image p6 before decoding image p6. Therefore, the decoding device 200 can appropriately decode image p6 according to PPS2 for image p6. Thus, the decoding device 200 is capable of such upward movement.

[0690] Furthermore, after decoding image p6, it is possible to move the image upwards so that image p7 can be decoded. Even with such upward movement, the decoding device 200 can still decode PPS3 for image p7 before decoding image p6. Therefore, the decoding device 200 can appropriately decode image p7 according to PPS3 for image p7. Thus, the decoding device 200 is also capable of such upward movement.

[0691] That is, even if the decoding device 200 is moved upwards, it can still decode the image appropriately according to PPS. Therefore, the decoding device 200 can be moved upwards appropriately.

[0692] use Figures 18-21BThe PPS notification described is not limited to variations of the filter information processing method. For example, a PPS may not contain filter control information or be associated with an adaptive cyclic filter. A PPS may not contain information associated with an adaptive cyclic filter, but may contain other information used in the encoding or decoding of the image. For example, a PPS may contain information about a list of reference images or information about the quantization matrix.

[0693] In addition, with Figures 18-21B The associated actions are not limited to PPS notifications; they may also apply notifications with other parameter sets, such as sequence parameter sets or adaptive parameter sets. That is, notifications can be associated with... Figures 18-21B In the associated description, PPS is replaced with parameter set.

[0694] In addition, with Figure 14A , Figure 14B as well as Figures 18-21B The allocation of associated time IDs can be performed according to encoding order, decoding order, display order, or data type. Furthermore, the encoding device 100 can encode the time IDs, and the decoding device 200 can decode them. Alternatively, by allocating time IDs between the encoding device 100 and the decoding device 200 using the same reference, the encoding and decoding of time IDs can be omitted.

[0695] In the encoding device 100, the time ID can be assigned by the entropy encoding unit 110, or by other components. Similarly, in the decoding device 200, the time ID can be assigned by the entropy decoding unit 202, or by other components.

[0696] [Installation example of the encoding device]

[0697] Figure 22 This is a block diagram showing an example of the installation of the encoding device 100. The encoding device 100 includes circuitry 160 and a memory 162. For example, Figure 1 The multiple components of the encoding device 100 shown are composed of Figure 22 The circuit 160 and memory 162 shown are installed.

[0698] Circuit 160 is an electronic circuit capable of accessing memory 162 and performing information processing. For example, circuit 160 is a dedicated or general-purpose electronic circuit that uses memory 162 to encode moving images. Circuit 160 can be a processor like a CPU. Alternatively, circuit 160 can be an assembly of multiple electronic circuits.

[0699] Furthermore, for example, circuit 160 can serve as Figure 1The encoding device 100 shown includes multiple components, excluding those used for storing information. That is, the circuit 160 can perform the aforementioned actions as actions of these components.

[0700] Memory 162 is a dedicated or general-purpose memory used by storage circuit 160 to encode moving images. Memory 162 can be an electronic circuit that can be connected to circuit 160 or included in circuit 160.

[0701] Furthermore, the memory 162 can be an assembly of multiple electronic circuits, or it can be composed of multiple sub-memories. Furthermore, the memory 162 can be a disk or optical disk, or it can be a storage container or recording medium. Furthermore, the memory 162 can be either non-volatile or volatile memory.

[0702] For example, memory 162 can serve as Figure 1 The encoding device 100 shown is composed of multiple components for storing information. Specifically, the memory 162 can act as... Figure 1 The block memory 118, frame memory 122, current filter information storage unit 132, and reference filter information storage unit 133 are shown.

[0703] Additionally, the memory 162 may store the encoded motion image, or it may store the bit string corresponding to the encoded motion image. Furthermore, the memory 162 may also store a program for the circuit 160 to encode the motion image.

[0704] Alternatively, the encoding device 100 may not require installation of [something]. Figure 1 All of the multiple constituent elements shown can also be processed without the above-mentioned multiple processing steps. Figure 1 Some of the constituent elements shown may be included in other devices, and some of the aforementioned processes may be performed by other devices. Furthermore, in the encoding device 100, by installing... Figure 1 As part of the multiple components shown, and as part of the multiple processes described above, information associated with the encoding of the moving image can be appropriately set.

[0705] Figure 23 It means Figure 22 A flowchart illustrating the first operation example of the encoding device 100 shown. For example, Figure 22 The encoding device 100 shown applies an adaptive cyclic filter when encoding a moving image containing multiple images, each assigned a time ID representing a hierarchy related to time scalability. Figure 23 The actions shown are as follows. Specifically, circuit 160 uses memory 162 to perform the following actions.

[0706] First, circuit 160 determines first filter information by referring to second filter information associated with the second image. This first filter information applies an adaptive cyclic filter to the first image among the multiple images (S701). Here, the second image is the image that comes before the first image in the encoding order among the multiple images.

[0707] At this time, if the NAL unit type of the first image is a specified NAL unit type, circuit 160 prohibits referencing the third filter information associated with the third image as the second filter information. Here, the third image is the image that comes before the first image in the encoding order among multiple images, and it is the image with the same time ID as the first image. Then, circuit 160 applies an adaptive cyclic filter to the first image using the determined first filter information (S702).

[0708] Therefore, the encoding device 100 can determine the first filter information of the first image by referring to the second filter information of the second image. At this time, the encoding device 100 can prevent the third filter information of the third image, which is at the same level as the first image of the specified NAL unit type, from being referred to as the second filter information.

[0709] That is, the encoding device 100 can reference and restrict the filter information of images at the same level as the first image of the specified NAL unit type in the same way that reference restrictions can be applied to images at the same level as the first image of the specified NAL unit type. Therefore, the encoding device 100 can associate filter information with images and manage it appropriately, and can appropriately restrict and set the referenced filter information. Therefore, the encoding device 100 can appropriately set information associated with the encoding of moving images.

[0710] For example, in determining the first filter information, circuit 160 can prevent the fourth filter information, which is associated with the fourth image, from being used as reference for the second filter information. Here, the fourth image is an image that comes before the first image in the encoding order among multiple images, and is an image with a time ID greater than that of the first image.

[0711] Therefore, when determining the first filter information of the first image by referring to the second filter information of the second image, it is prohibited to use the fourth filter information of the fourth image with a time ID greater than that of the first image as the reference for the second filter information.

[0712] Therefore, in terms of temporal scalability, the encoding device 100 can reference and restrict the filter information associated with the image in the same way that reference restrictions can be applied to the image. Thus, the encoding device 100 can appropriately restrict and set the referenced filter information.

[0713] Furthermore, for example, circuit 160 can determine fifth filter information by referring to sixth filter information associated with the sixth image. This fifth filter information is used to apply an adaptive cyclic filter to the fifth image, which is later in the encoding order than the first image among the multiple images. Here, the sixth image is the image that is earlier in the encoding order than the fifth image among the multiple images.

[0714] At this point, if the NAL cell type of the first image is a specified NAL cell type, circuit 160 can prevent the third filter information associated with the third image from being referenced as the sixth filter information. Then, circuit 160 can apply an adaptive cyclic filter to the fifth image using the determined fifth filter information.

[0715] Therefore, the encoding device 100 can refer to the sixth filter information of the sixth image to determine the fifth filter information of the fifth image, which is located after the first image in the encoding order. At this time, the encoding device 100 can prevent the third filter information of the third image, which is at the same level as the first image of the specified NAL unit type, from being referenced as the sixth filter information.

[0716] That is, the encoding device 100 can reference and restrict the filter information of images at the same level as the first image, located after the first image of the specified NAL unit type, in the same manner as the reference restriction that can be applied to images at the same level as the first image. Therefore, the encoding device 100 can associate filter information with images and manage it appropriately, and can appropriately restrict and set the referenced filter information. Therefore, the encoding device 100 can appropriately set the information associated with the encoding of the moving image.

[0717] Furthermore, for example, in determining the fifth filter information, if the NAL unit type of the first image is a specified NAL unit type, circuit 160 can prevent the fourth filter information associated with the fourth image from being referenced as the sixth filter information. Here, the fourth image is the image that comes before the first image in the encoding order among multiple images, and is the image with a time ID greater than the first image.

[0718] Therefore, when determining the fifth filter information of the fifth image by referring to the sixth filter information of the sixth image, it is prohibited to use the fourth filter information of the fourth image with a time ID greater than that of the first image as the reference for the fifth filter information.

[0719] That is, the encoding device 100 can reference and restrict filter information in the same way that references images with a time ID greater than the first image are referenced, located after the first image of the specified NAL unit type. Therefore, the encoding device 100 can associate filter information with images and manage it appropriately, and can appropriately restrict and set the referenced filter information. Therefore, the encoding device 100 can appropriately set information associated with the encoding of moving images.

[0720] Furthermore, for example, in determining the first filter information, circuit 160 may, under specified circumstances, prohibit the reference of the seventh filter information associated with the seventh picture as the second filter information.

[0721] Here, the specified condition is that an eighth image exists between the first and seventh images, and the NAL unit type of the eighth image is the specified NAL unit type. Furthermore, the seventh image is the image that comes before the first image in the encoding order among multiple images, and it is an image with a time ID greater than 0. Additionally, the eighth image is an image whose time ID is the same as or less than that of the seventh image.

[0722] Therefore, the encoding device 100 can prevent the seventh filter information of the seventh picture, which has the same or larger time ID than the eighth picture, from being used as the reference for the second filter information, at a point after the eighth picture of the specified NAL unit type.

[0723] That is, the encoding device 100 can reference and restrict the seventh filter information of the seventh image at a location later than the eighth image of the specified NAL unit type, in the same manner as the reference restriction that can be applied to the seventh image. Therefore, the encoding device 100 can associate filter information with images and manage it appropriately, and can appropriately restrict and set the referenced filter information. Thus, the encoding device 100 can appropriately set information associated with the encoding of the moving image.

[0724] In addition, for example, it can be specified that the NAL unit type can be the NAL unit type of a TSA image.

[0725] Therefore, the encoding device 100 can reference and restrict the filter information of images at the same level as TSA images in the same way that reference restrictions can be applied to images at the same level as TSA images. Thus, the encoding device 100 can associate filter information with images and manage it appropriately, and can appropriately restrict and set the referenced filter information. Therefore, the encoding device 100 can appropriately set information associated with the encoding of moving images.

[0726] Furthermore, for example, circuit 160 can determine fifth filter information by referring to sixth filter information associated with the sixth image. This fifth filter information is used to apply an adaptive cyclic filter to the fifth image, which is later in the encoding order than the first image among the multiple images. Here, the sixth image is the image that is earlier in the encoding order than the fifth image among the multiple images.

[0727] At this point, circuit 160 can, under specified conditions, prevent the third filter information associated with the third image from being referenced as the sixth filter information. Here, the specified conditions are that the NAL unit type of the first image is a specified NAL unit type, and the time ID of the fifth image is the same as the time ID of the first image. Then, circuit 160 can apply an adaptive cyclic filter to the fifth image using the determined fifth filter information.

[0728] Therefore, the encoding device 100 can refer to the sixth filter information of the sixth image to determine the fifth filter information of the fifth image, which is located after the first image in the encoding order and at the same level as the first image. At this time, the encoding device 100 can prevent the third filter information of the third image, which is at the same level as the first image of the specified NAL unit type, from being referenced as the sixth filter information.

[0729] That is, the encoding device 100 can reference and restrict filter information in the same way as it does for images preceding the first image, but at the same level as the first image, even though the first image is located after the first image of the specified NAL unit type. Therefore, the encoding device 100 can associate filter information with images and manage it appropriately, and can appropriately restrict and set the referenced filter information. Thus, the encoding device 100 can appropriately set information associated with the encoding of moving images.

[0730] In addition, for example, it can be specified that the NAL unit type can be the NAL unit type of an STSA image.

[0731] Therefore, the encoding device 100 can reference and restrict the filter information of images at the same level as STSA images in the same way that reference restrictions can be applied to images at the same level as STSA images. Thus, the encoding device 100 can associate filter information with images and manage it appropriately, and can appropriately restrict and set the referenced filter information. Therefore, the encoding device 100 can appropriately set the information associated with the encoding of moving images.

[0732] Figure 24 It means Figure 22 A flowchart illustrating a second operation example of the encoding device 100 shown. For example, Figure 22The encoding device 100 shown performs the following when encoding a moving image containing multiple images, each assigned a time ID representing a hierarchy related to time scalability: Figure 24 The actions shown are as follows. Specifically, circuit 160 uses memory 162 to perform the following actions.

[0733] First, circuit 160 encodes multiple parameter sets, each assigned 0 as a time ID representing a level related to time scalability (S801). Then, after encoding the multiple parameter sets, circuit 160 encodes the first image in the multiple images according to the encoding order (S802).

[0734] Here, multiple parameters correspond to multiple levels represented by multiple time IDs assigned to multiple images. Additionally, multiple parameter sets are parameter sets for one or more images among the multiple images, each assigned a time ID representing the level corresponding to that parameter set.

[0735] Therefore, the encoding device 100 can first summarize and encode multiple parameter sets corresponding to multiple levels respectively. Furthermore, each of the multiple parameter sets is assigned 0 as a time ID. Therefore, multiple parameter sets can be appropriately processed without discarding any. Thus, the encoding device 100 can appropriately set information associated with the encoding of the moving image.

[0736] For example, multiple images can constitute a first image group. Furthermore, multiple parameter sets can constitute a first parameter set group. Additionally, the moving image can contain multiple images constituting a second image group. Then, circuit 160 can encode the multiple parameter sets constituting the second parameter set group after encoding the images constituting the first image group.

[0737] Here, the multiple parameter sets constituting the second parameter set group can be multiple parameter sets each assigned 0 as a time ID representing a level related to time scalability. Furthermore, the multiple parameter sets constituting the second parameter set group can each correspond to multiple levels indicated by multiple time IDs assigned to multiple images constituting the second image group.

[0738] Furthermore, each parameter set constituting the second parameter set group can be a parameter set for one or more images in the second image group, each image having a time ID representing the level corresponding to that parameter set.

[0739] In addition, circuit 160 can encode the first image in the encoding order among the multiple images constituting the second image group after encoding the multiple parameter sets constituting the second parameter set group.

[0740] Therefore, for each image group, the encoding device 100 can first summarize and encode multiple parameter sets corresponding to multiple levels respectively. Thus, the encoding device 100 can appropriately set information associated with the encoding of the moving image for each image group.

[0741] Figure 25 It means Figure 22 A flowchart illustrating the third operation example of the encoding device 100 shown. For example, Figure 22 The encoding device 100 shown performs the following when encoding a moving image containing multiple images: Figure 25 The actions shown are as follows. Specifically, circuit 160 uses memory 162 to perform the following actions.

[0742] First, circuit 160 encodes the first image among multiple images (S901). Then, circuit 160 performs either the first action or the second action (S902). At this time, if the second image is a specified image, circuit 160 performs the first action.

[0743] Here, the first action is to encode the parameter set for the second image after encoding the first image, and then encode the second image after encoding the parameter set for the second image. Conversely, the second action is to encode the second image without encoding the parameter set for the second image, after encoding the first image. Furthermore, the second image is the image that comes after the first image in the encoding order.

[0744] Therefore, the encoding device 100 can encode the parameter set of the specified image before the specified image is generated. Thus, in actions such as moving the specified image upwards, the parameter set of the specified image can be appropriately processed. Therefore, the encoding device 100 can appropriately set information associated with the encoding of the moving image.

[0745] For example, the image can be a TSA image. Therefore, the encoding device 100 can encode the parameter set of the TSA image before the TSA image is generated. Thus, in operations such as upshifting the TSA image, the parameter set of the TSA image can be appropriately processed. Therefore, the encoding device 100 can appropriately set information associated with the encoding of the moving image.

[0746] Furthermore, for example, the multiple images can each be an image assigned a time ID representing a hierarchy related to time scalability. Then, in the first operation, after encoding the first image, circuit 160 encodes multiple associated parameter sets, which are multiple parameter sets containing parameter sets for the second image, and after encoding the multiple associated parameter sets, it encodes the second image.

[0747] Here, the multiple association parameter sets correspond to multiple levels indicated by multiple time IDs assigned to the second image and above. Additionally, each of the multiple association parameter sets is a parameter set for one or more images among the multiple images, each assigned a time ID representing the level corresponding to that association parameter set.

[0748] Therefore, the encoding device 100 can encode multiple parameter sets for multiple images whose time ID is the same as or greater than the specified image, before the specified image is generated. Thus, in actions such as moving images with time IDs greater than the specified image upwards, the parameter sets can be appropriately processed. Therefore, the encoding device 100 can appropriately set information associated with the encoding of the moving image.

[0749] Furthermore, for example, the specified image can be an STSA image. Therefore, the encoding device 100 can encode the parameter set for the STSA image before the STSA image is generated. Thus, in operations such as upshifting the STSA image, the parameter set for the STSA image can be appropriately processed. Therefore, the encoding device 100 can appropriately set information associated with the encoding of the moving image.

[0750] Furthermore, for example, the second image may be one of multiple images that is encoded following the first image. Therefore, the encoding device 100 can appropriately encode the parameter set of the specified image before encoding it. Thus, the parameter set for the specified image can be appropriately processed. Therefore, the encoding device 100 can appropriately set the information associated with the encoding of the moving image.

[0751] In addition, for example, multiple images can each be an image assigned a time ID representing a hierarchy related to time scalability.

[0752] Then, circuit 160 can encode multiple aggregate parameter sets, which are multiple parameter sets containing parameter sets for the second image, before encoding the first image in the order of encoding of the multiple images.

[0753] Here, the aforementioned multiple summary parameter sets correspond to multiple levels represented by multiple time IDs assigned to multiple images. Furthermore, each of the multiple summary parameter sets is a parameter set for one or more images among the multiple images, each assigned a time ID representing the level corresponding to that summary parameter set.

[0754] Therefore, even if multiple parameter sets containing a parameter set for a specified image are encoded first, the encoding device 100 can encode the parameter set for the specified image again before the specified image. Thus, the parameter set for the specified image can be processed appropriately. Therefore, the encoding device 100 can appropriately set the information associated with the encoding of the moving image.

[0755] Furthermore, for example, the specified image can be a TSA image or an STSA image. Furthermore, for example, circuit 160 can disable the second action if the second image is the specified image.

[0756] Figure 26 It means Figure 22 A flowchart illustrating the fourth operation example of the encoding device 100 shown. For example, Figure 22 The encoding device 100 shown performs the following when encoding a moving image containing multiple images, each assigned a time ID representing a hierarchy related to time scalability: Figure 26 The actions shown are as follows. Specifically, circuit 160 uses memory 162 to perform the following actions.

[0757] First, circuit 160 encodes the first image among multiple images (S1001). Then, circuit 160 performs either the first action or the second action (S1002). At this time, circuit 160 performs the first action if the time ID assigned to the second image is greater than the minimum time ID and less than the maximum time ID.

[0758] Here, the first action is to encode the parameter set for the second image after encoding the first image, and then encode the second image after encoding the parameter set for the second image. Conversely, the second action is to encode the second image without encoding the parameter set for the second image, after encoding the first image. Furthermore, the second image is the image that comes after the first image in the encoding order.

[0759] Additionally, the smallest time ID is the smallest of the multiple time IDs assigned to multiple images. Conversely, the largest time ID is the largest of the multiple time IDs assigned to multiple images.

[0760] Therefore, the encoding device 100 can encode the parameter set for the intermediate layer image before the intermediate layer image. Thus, during processes such as moving the intermediate layer image upwards, the parameter set for the intermediate layer image can be appropriately processed. Therefore, the encoding device 100 can appropriately set information associated with the encoding of the moving image.

[0761] For example, circuit 160 can perform the first action if the time ID assigned to the second image is the maximum time ID. Therefore, encoding device 100 can encode the parameter set for the topmost image before the topmost image. Thus, in actions such as moving the topmost image upwards, the parameter set for the topmost image can be appropriately processed. Therefore, encoding device 100 can appropriately set information associated with the encoding of the moving image.

[0762] Furthermore, for example, in the first operation performed by circuit 160 under specified conditions, the parameter set and the highest-order parameter set for the second image can be encoded after the first image is encoded. Then, circuit 160 can encode the second image after encoding the parameter set and the highest-order parameter set for the second image.

[0763] Here, the specified case is when the time ID assigned to the second image is the second largest among multiple time IDs. Additionally, the topmost parameter set is the parameter set for more than one image assigned the largest time ID.

[0764] Therefore, the encoding device 100 can efficiently encode two parameter sets, which contain parameter sets for the topmost image.

[0765] Furthermore, for example, circuit 160 can perform the second operation if the time ID assigned to the second image is the largest time ID. Thus, encoding device 100 can omit encoding the parameter set of the topmost image before the topmost image.

[0766] Furthermore, for example, circuit 160 can perform the first operation under specified conditions if the time ID assigned to the second image is greater than the minimum time ID and less than the maximum time ID. Here, the specified conditions are conditions where a specified flag contained in the sequence parameter set for multiple images is a specified value. Thus, encoding device 100 is able to appropriately encode the parameter set for intermediate layer images in a specified sequence, before the images of the intermediate layer.

[0767] Furthermore, for example, if the specified flag is a specified value, and the time ID assigned to the image is not the minimum time ID, multiple images can each be TSA images. Therefore, the encoding device 100 can appropriately encode the parameter set for intermediate layer images in a specified sequence consisting of TSA images excluding the lowest layer, before the intermediate layer images.

[0768] Furthermore, for example, circuit 160 can disable the second action if the time ID assigned to the second image is greater than the minimum time ID and less than the maximum time ID. Furthermore, for example, circuit 160 can also disable the second action if the time ID assigned to the second image is the maximum time ID. Furthermore, for example, under specified conditions, circuit 160 can disable the second action according to the time ID assigned to the second image.

[0769] [Installation example of the decoding device]

[0770] Figure 27This is a block diagram showing an example of the installation of the decoding device 200. The decoding device 200 includes circuitry 260 and a memory 262. For example, Figure 10 The multiple components of the decoding device 200 shown are composed of Figure 27 The circuit 260 and memory 262 shown are installed.

[0771] Circuit 260 is an electronic circuit capable of accessing memory 262 and performing information processing. For example, circuit 260 is a dedicated or general-purpose electronic circuit that uses memory 262 to decode moving images. Circuit 260 can be a processor like a CPU. Alternatively, circuit 260 can be an assembly of multiple electronic circuits.

[0772] Furthermore, for example, circuit 260 can act as Figure 10 The decoding device 200 shown includes multiple components, excluding those used for storing information. That is, the circuit 260 can perform the aforementioned actions as actions of these components.

[0773] Memory 262 is a dedicated or general-purpose memory used by storage circuit 260 to decode moving images. Memory 262 can be an electronic circuit that can be connected to circuit 260 or included in circuit 260.

[0774] Furthermore, memory 262 can be an assembly of multiple electronic circuits, or it can be composed of multiple sub-memories. Furthermore, memory 262 can be a disk or optical disk, or it can be a storage container or recording medium. Furthermore, memory 262 can be non-volatile memory or volatile memory.

[0775] For example, memory 262 can act as Figure 10 The decoding device 200 shown includes several components for storing information. Specifically, the memory 262 can act as... Figure 10 The block memory 210, frame memory 214, current filter information storage unit 232, and reference filter information storage unit 233 are shown.

[0776] Additionally, the memory 262 may store bit strings corresponding to the encoded motion image, or it may store the decoded motion image. Furthermore, the memory 262 may also store a program for the circuit 260 to decode the motion image.

[0777] Alternatively, it is not necessary to install in the decoding device 200. Figure 10 All of the multiple constituent elements shown can also be processed without the above-mentioned multiple processing steps. Figure 10Some of the constituent elements shown may be included in other devices, and some of the aforementioned processes may be performed by other devices. Furthermore, in the decoding device 200, by installing... Figure 10 As part of the multiple components shown, and as part of the multiple processes described above, information related to the decoding of moving images can be appropriately set.

[0778] Figure 28 It means Figure 27 A flowchart illustrating the first operation example of the decoding device 200 shown. For example, Figure 27 The decoding device 200 shown applies an adaptive cyclic filter when decoding a moving image containing multiple images, each assigned a time ID representing a hierarchy related to time scalability. Figure 28 The actions shown are as follows. Specifically, circuit 260 uses memory 262 to perform the following actions.

[0779] First, circuit 260 determines first filter information by referring to second filter information associated with the second image. This first filter information applies an adaptive cyclic filter to the first image among the multiple images (S1101). Here, the second image is the image that comes before the first image in the decoding order among the multiple images.

[0780] At this time, if the NAL unit type of the first image is a specified NAL unit type, circuit 260 prohibits referencing the third filter information associated with the third image as the second filter information. Here, the third image is the image that comes before the first image in the decoding order among multiple images, and it is the image with the same time ID as the first image. Then, circuit 260 applies an adaptive cyclic filter to the first image using the determined first filter information (S1102).

[0781] Therefore, the decoding device 200 can determine the first filter information of the first image by referring to the second filter information of the second image. At this time, the decoding device 200 can prevent the third filter information of the third image, which is at the same level as the first image of the specified NAL unit type, from being referenced as the second filter information.

[0782] That is, the decoding device 200 can reference and restrict the filter information of images at the same level as the first image of the specified NAL unit type in the same way that reference restrictions can be applied to images at the same level as the first image of the specified NAL unit type. Therefore, the decoding device 200 can associate filter information with images and manage it appropriately, and can appropriately restrict and set the referenced filter information. Therefore, the decoding device 200 can appropriately set the information associated with the decoding of moving images.

[0783] For example, in determining the first filter information, circuit 260 can prevent the fourth filter information, which is associated with the fourth image, from being used as reference for the second filter information. Here, the fourth image is the image that comes before the first image in the decoding order among multiple images, and it is the image whose time ID is greater than that of the first image.

[0784] Therefore, when determining the first filter information of the first image by referring to the second filter information of the second image, it is prohibited to use the fourth filter information of the fourth image with a time ID greater than that of the first image as the reference for the second filter information.

[0785] Therefore, in terms of temporal scalability, the decoding device 200 can reference and restrict the filter information associated with the image in the same way as the reference restriction applied to the image. Thus, the decoding device 200 can appropriately restrict and set the referenced filter information.

[0786] Furthermore, for example, circuit 260 can determine fifth filter information by referring to sixth filter information associated with the sixth image, which applies an adaptive cyclic filter to the fifth image among the multiple images that is later than the first image in the decoding order. Here, the sixth image is the image among the multiple images that is earlier than the fifth image in the decoding order.

[0787] At this point, if the NAL cell type of the first image is a specified NAL cell type, circuit 260 can prevent the third filter information associated with the third image from being referenced as the sixth filter information. Then, circuit 260 can apply an adaptive cyclic filter to the fifth image using the determined fifth filter information.

[0788] Therefore, the decoding device 200 can refer to the sixth filter information of the sixth image to determine the fifth filter information of the fifth image, which is later than the first image in the decoding order. At this time, the decoding device 200 can prevent the third filter information of the third image, which is at the same level as the first image of the specified NAL unit type, from being referenced as the sixth filter information.

[0789] That is, the decoding device 200 can reference and restrict the filter information of images at the same level as the first image, located after the first image of the specified NAL unit type, in the same manner as the reference restriction that can be applied to images at the same level as the first image. Therefore, the decoding device 200 can associate filter information with images and manage it appropriately, and can appropriately restrict and set the referenced filter information. Thus, the decoding device 200 can appropriately set the information associated with the decoding of moving images.

[0790] Furthermore, for example, in determining the fifth filter information, if the NAL unit type of the first image is a specified NAL unit type, circuit 260 can prevent the fourth filter information associated with the fourth image from being referenced as the sixth filter information. Here, the fourth image is the image that comes before the first image in the decoding order among multiple images, and is the image with a time ID greater than the first image.

[0791] Therefore, when determining the fifth filter information of the fifth image by referring to the sixth filter information of the sixth image, it is prohibited to use the fourth filter information of the fourth image with a time ID greater than that of the first image as the reference for the fifth filter information.

[0792] That is, the decoding device 200 can reference and restrict filter information at a location later than the first image of the specified NAL unit type, in the same manner as the reference restriction applied to images with a time ID greater than the first image. Therefore, the decoding device 200 can associate filter information with images and manage it appropriately, and can appropriately restrict and set the referenced filter information. Thus, the decoding device 200 can appropriately set information associated with the decoding of moving images.

[0793] Furthermore, for example, in determining the first filter information, circuit 260 may, under specified circumstances, prohibit the reference of the seventh filter information associated with the seventh picture as the second filter information.

[0794] Here, the specified condition is that an eighth image exists between the first and seventh images, and the NAL unit type of the eighth image is the specified NAL unit type. Additionally, the seventh image is the image that comes before the first image in the decoding order among multiple images, and it is an image with a time ID greater than 0. Furthermore, the eighth image is an image whose time ID is the same as or less than that of the seventh image.

[0795] Therefore, the decoding device 200 is able to prevent the 7th filter information of the 7th image, which has the same or larger time ID than the 8th image, from being used as the reference for the 2nd filter information, located after the 8th image of the specified NAL unit type.

[0796] That is, the decoding device 200 can reference and restrict the seventh filter information of the seventh image at a location later than the eighth image of the specified NAL unit type, in the same manner as the reference restriction that can be applied to the seventh image. Therefore, the decoding device 200 can associate filter information with images and manage it appropriately, and can appropriately restrict and set the referenced filter information. Thus, the decoding device 200 can appropriately set information associated with the decoding of moving images.

[0797] In addition, for example, it can be specified that the NAL unit type can be the NAL unit type of a TSA image.

[0798] Therefore, the decoding device 200 can reference and restrict the filter information of images at the same level as TSA images in the same way that reference restrictions can be applied to images at the same level as TSA images. Thus, the decoding device 200 can associate filter information with images and manage it appropriately, and can appropriately restrict and set the referenced filter information. Therefore, the decoding device 200 can appropriately set information associated with the decoding of moving images.

[0799] Furthermore, for example, circuit 260 can determine fifth filter information by referring to sixth filter information associated with the sixth image, which applies an adaptive cyclic filter to the fifth image among the multiple images that is later than the first image in the decoding order. Here, the sixth image is the image among the multiple images that is earlier than the fifth image in the decoding order.

[0800] At this point, circuit 260 can, under specified conditions, prevent the third filter information associated with the third image from being referenced as the sixth filter information. Here, the specified conditions are that the NAL unit type of the first image is a specified NAL unit type, and the time ID of the fifth image is the same as the time ID of the first image. Then, circuit 260 can use the determined fifth filter information to apply an adaptive cyclic filter to the fifth image.

[0801] Therefore, the decoding device 200 can refer to the sixth filter information of the sixth image to determine the fifth filter information of the fifth image, which is located after the first image in the decoding order and at the same level as the first image. At this time, the decoding device 200 can prevent the third filter information of the third image, which is at the same level as the first image of the specified NAL unit type, from being referenced as the sixth filter information.

[0802] That is, the decoding device 200 can reference and restrict filter information in the same way as the reference restriction applied to images preceding the first image, located at the same level as the first image but later than the first image of the specified NAL unit type. Therefore, the decoding device 200 can associate filter information with images and manage it appropriately, and can appropriately restrict and set the referenced filter information. Thus, the decoding device 200 can appropriately set information associated with the decoding of moving images.

[0803] In addition, for example, it can be specified that the NAL unit type can be the NAL unit type of an STSA image.

[0804] Therefore, the decoding device 200 can reference and restrict the filter information of images at the same level as STSA images in the same way that reference restrictions can be applied to images at the same level as STSA images. Thus, the decoding device 200 can associate filter information with images and manage it appropriately, and can appropriately restrict and set the referenced filter information. Therefore, the decoding device 200 can appropriately set the information associated with the decoding of moving images.

[0805] Figure 29 It means Figure 27 A flowchart illustrating the second operation example of the decoding device 200 shown. For example, Figure 27 The decoding device 200 shown performs the following when decoding a moving image containing multiple images, each assigned a time ID representing a hierarchy related to time scalability: Figure 29 The actions shown are as follows. Specifically, circuit 260 uses memory 262 to perform the following actions.

[0806] First, circuit 260 decodes multiple parameter sets, each assigned 0 as a time ID representing a level related to time scalability (S1201). Then, after decoding the multiple parameter sets, circuit 260 decodes the first image in the decoding order among the multiple images (S1202).

[0807] Here, multiple parameter sets correspond to multiple levels represented by multiple time IDs assigned to multiple images. Furthermore, each of the multiple parameter sets is a parameter set for one or more images among the multiple images, each assigned a time ID representing the level corresponding to that parameter set.

[0808] Therefore, the decoding device 200 can first summarize and decode multiple parameter sets corresponding to multiple levels. Furthermore, each parameter set is assigned 0 as a time ID. Therefore, multiple parameter sets can be processed appropriately without discarding any. Thus, the decoding device 200 can appropriately set information associated with the decoding of the moving image.

[0809] For example, multiple images can constitute a first image group. Furthermore, multiple parameter sets can constitute a first parameter set group. Additionally, a moving image can contain multiple images constituting a second image group. Then, circuit 260 can decode the multiple parameter sets constituting the second parameter set group after decoding the images constituting the first image group.

[0810] Here, the multiple parameter sets constituting the second parameter set group can be multiple parameter sets each assigned 0 as a time ID representing a level related to time scalability. Furthermore, the multiple parameter sets constituting the second parameter set group can each correspond to multiple levels indicated by multiple time IDs assigned to multiple images constituting the second image group.

[0811] Furthermore, each parameter set constituting the second parameter set group can be a parameter set for one or more images in the second image group, each image having a time ID representing the level corresponding to that parameter set.

[0812] Furthermore, circuit 260 can decode the first image in the decoding order among the multiple images constituting the second image group after decoding the multiple parameter sets constituting the second parameter set group.

[0813] Therefore, for each image group, the decoding device 200 can first perform aggregated decoding on multiple parameter sets corresponding to multiple levels. Thus, the decoding device 200 can appropriately set information associated with the decoding of the moving image for each image group.

[0814] Figure 30 It means Figure 27 A flowchart illustrating the third operation example of the decoding device 200 shown. For example, Figure 27 The decoding device 200 shown performs the following when decoding a moving image containing multiple images: Figure 30 The actions shown are as follows. Specifically, circuit 260 uses memory 262 to perform the following actions.

[0815] First, circuit 260 decodes the first image among multiple images (S1301). Then, circuit 260 performs either the first action or the second action (S1302). At this time, if the second image is a specified image, circuit 260 performs the first action.

[0816] Here, the first action is to decode the parameter set for the second image after decoding the first image, and then decode the second image after decoding the parameter set for the second image. The second action is to decode the second image without decoding the parameter set for the second image, after decoding the first image. Furthermore, the second image is the image that appears later than the first image in the decoding order among multiple images.

[0817] Therefore, the decoding device 200 can decode the parameter set for the specified image before the specified image is rendered. Thus, in actions such as moving the specified image upwards, the parameter set for the specified image can be appropriately processed. Therefore, the decoding device 200 can appropriately set information associated with the decoding of the moving image.

[0818] For example, the image can be a TSA image. Therefore, the decoding device 200 can decode the parameter set for the TSA image before the TSA image is generated. Thus, in operations such as upshifting the TSA image, the parameter set for the TSA image can be appropriately processed. Therefore, the decoding device 200 can appropriately set information associated with the decoding of the moving image.

[0819] Furthermore, for example, the multiple images may each be an image assigned a time ID representing a hierarchy related to time scalability. Then, in the first operation, after decoding the first image, circuit 260 may decode multiple associated parameter sets, which are multiple parameter sets containing parameter sets for the second image, and decode the second image after decoding the multiple associated parameter sets.

[0820] Here, the multiple association parameter sets correspond to multiple levels indicated by multiple time IDs assigned to the second image and above. Additionally, each of the multiple association parameter sets is a parameter set for one or more images among the multiple images, each assigned a time ID representing the level corresponding to that association parameter set.

[0821] Therefore, the decoding device 200 can decode multiple parameter sets for multiple images with the same time ID as the specified image or with a time ID greater than the specified image, before the specified image is rendered. Thus, in tasks such as shifting images with time IDs greater than the specified image, the parameter sets can be appropriately processed. Therefore, the decoding device 200 can appropriately set information associated with the decoding of the moving image.

[0822] Furthermore, for example, the image can be an STSA image. Therefore, the decoding device 200 can decode the parameter set for the STSA image before the STSA image is generated. Thus, in operations such as upshifting the STSA image, the parameter set for the STSA image can be appropriately processed. Therefore, the decoding device 200 can appropriately set the information associated with the decoding of the moving image.

[0823] Furthermore, for example, the second image may be an image that is decoded following the first image among multiple images. Therefore, the decoding device 200 can appropriately decode the parameter set of a given image before decoding it. Thus, the parameter set for the given image can be appropriately processed. Therefore, the decoding device 200 can appropriately set the information associated with the decoding of the moving image.

[0824] In addition, for example, multiple images can each be an image assigned a time ID representing a hierarchy related to time scalability.

[0825] Then, circuit 260 can decode multiple summative parameter sets, which include multiple parameter sets for the second image, in the order of decoding of the first image.

[0826] Here, the multiple summary parameter sets correspond to multiple levels represented by multiple time IDs assigned to multiple images. Furthermore, each of the multiple summary parameter sets is a parameter set for one or more images among the multiple images, each assigned a time ID representing the level corresponding to that summary parameter set.

[0827] Therefore, even if multiple parameter sets containing a parameter set for a specified image are decoded first, the decoding device 200 can decode the parameter set for the specified image again before the specified image. Thus, the parameter set for the specified image can be appropriately processed. Therefore, the decoding device 200 can appropriately set the information associated with the decoding of the moving image.

[0828] Furthermore, for example, the specified image can be a TSA image or an STSA image. Furthermore, for example, circuit 260 can disable the second action if the second image is the specified image.

[0829] Figure 31 It means Figure 27 A flowchart illustrating the fourth operation example of the decoding device 200 shown. For example, Figure 27 The decoding device 200 shown performs the following when decoding a moving image containing multiple images, each assigned a time ID representing a hierarchy related to time scalability: Figure 31 The actions shown are as follows. Specifically, circuit 260 uses memory 262 to perform the following actions.

[0830] First, circuit 260 decodes the first image among multiple images (S1401). Then, circuit 260 performs either the first action or the second action (S1402). At this time, circuit 260 performs the first action if the time ID assigned to the second image is greater than the minimum time ID and less than the maximum time ID.

[0831] Here, the first action is to decode the parameter set for the second image after decoding the first image, and then decode the second image after decoding the parameter set for the second image. The second action is to decode the second image without decoding the parameter set for the second image, after decoding the first image. Furthermore, the second image is the image that appears later than the first image in the decoding order among multiple images.

[0832] Additionally, the smallest time ID is the smallest of the multiple time IDs assigned to multiple images. Conversely, the largest time ID is the largest of the multiple time IDs assigned to multiple images.

[0833] Therefore, the decoding device 200 can decode the parameter set for the intermediate layer image before the intermediate layer image. Thus, in processes such as moving the intermediate layer image upwards, the parameter set for the intermediate layer image can be appropriately processed. Therefore, the decoding device 200 can appropriately set information associated with the decoding of the moving image.

[0834] For example, circuit 260 can perform the first action if the time ID assigned to the second image is the maximum time ID. Therefore, decoding device 200 can decode the parameter set for the top-level image before the top-level image. Thus, in actions such as moving the top-level image upwards, the parameter set for the top-level image can be appropriately processed. Therefore, decoding device 200 can appropriately set the information associated with decoding the moving image.

[0835] Furthermore, for example, in the first operation performed by circuit 260 under specified conditions, the parameter set and the highest-order parameter set for the second image can be decoded after the first image is decoded. Then, circuit 260 can decode the second image after decoding the parameter set and the highest-order parameter set for the second image.

[0836] Here, the specified case is when the time ID assigned to the second image is the second largest among multiple time IDs. Additionally, the topmost parameter set is the parameter set for more than one image assigned the largest time ID.

[0837] Therefore, the decoding device 200 can efficiently decode two parameter sets, which contain the parameter set for the topmost image.

[0838] Furthermore, for example, circuit 260 can perform the second action if the time ID assigned to the second image is the largest time ID. Thus, decoding device 200 can omit decoding the parameter set for the top-level image before the top-level image.

[0839] Furthermore, for example, circuit 260 can perform the first operation under specified conditions, where the time ID assigned to the second image is greater than the minimum time ID and less than the maximum time ID. Here, the specified conditions are conditions where a specified flag contained in the sequence parameter set for multiple images is a specified value. Thus, decoding device 200 is able to appropriately decode the parameter set for intermediate layer images in a specified sequence, before the images of the intermediate layer.

[0840] Furthermore, for example, if the specified flag is a specified value, and the time ID assigned to the image is not the smallest time ID, multiple images can each be TSA images. Thus, the decoding device 200 can appropriately decode the parameter set for intermediate layer images within a specified sequence consisting of TSA images excluding the lowest layer, before the intermediate layer images.

[0841] Furthermore, for example, circuit 260 can disable the second action if the time ID assigned to the second image is greater than the minimum time ID and less than the maximum time ID. Furthermore, for example, circuit 260 can disable the second action if the time ID assigned to the second image is the maximum time ID. Furthermore, for example, under specified conditions, circuit 260 can disable the second action according to the time ID assigned to the second image.

[0842] [Replenish]

[0843] The encoding device 100 and the decoding device 200 in this embodiment can be used as an image encoding device and an image decoding device, respectively, or they can be used as a motion picture encoding device and a motion picture decoding device.

[0844] Alternatively, the encoding device 100 and the decoding device 200 can be used as a parameter encoding device and a parameter decoding device, respectively. That is, the encoding device 100 and the decoding device 200 can correspond to the entropy encoding unit 110 and the entropy decoding unit 202, respectively. In addition, other components such as the inter-frame prediction unit 126 or 218 can be included in other devices.

[0845] Alternatively, the encoding device 100 and the decoding device 200 can each be used as filter devices. That is, the encoding device 100 and the decoding device 200 can each correspond only to the cyclic filtering unit 120 and the cyclic filtering unit 212, respectively. In addition, other components such as the inter-frame prediction unit 126 or 218 can be included in other devices.

[0846] Alternatively, the encoding device 100 may correspond only to the entropy encoding unit 110 and the cyclic filtering unit 120. The decoding device 200 may correspond only to the entropy decoding unit 202 and the cyclic filtering unit 212.

[0847] In addition, at least a portion of this embodiment can be used as an encoding method, a decoding method, a parameter setting method, or other methods.

[0848] In addition, while each component is constructed using dedicated hardware in this embodiment, it can also be implemented by executing software programs suitable for each component. Each component can be implemented by reading and executing software programs recorded on recording media such as hard disks or semiconductor memory using a program execution unit such as a CPU or processor.

[0849] Specifically, each of the encoding device 100 and the decoding device 200 may include a processing circuitry and a storage device electrically connected to and accessible from the processing circuitry. For example, the processing circuitry corresponds to circuitry 160 or 260, and the storage device corresponds to memory 162 or 262.

[0850] The processing circuit includes at least one of dedicated hardware and a program execution unit, and uses a storage device to perform the processing. Furthermore, if the processing circuit includes a program execution unit, the storage device stores a software program executed by that program execution unit.

[0851] Here, the software for implementing the encoding device 100 or decoding device 200 of this embodiment is the following program.

[0852] That is, the program enables a computer to execute an encoding method that applies an adaptive cyclic filter in encoding a moving image containing multiple images, each assigned a time ID representing a level related to time scalability. The method includes: determining first filter information by referring to second filter information, which is used to apply the adaptive cyclic filter to a first image among the multiple images; the second filter information being associated with a second image amo...

Claims

1. An encoding method, wherein, The first or second action is executed based on the network abstraction layer (NAL) unit type. The first action mentioned above includes: Encode the first image; and After encoding the parameter set of the second image, the second image itself is encoded. The second image follows the first image in the encoding order. The parameter set of the second image and the first image share the same time ID. This time ID indicates the hierarchical level of time scalability. The NAL unit type of the second image is the NAL unit type for step-time sublayer access to STSA images. The second action mentioned above includes: Encode the first image mentioned above; and After encoding the first image, the third image is encoded without encoding the parameter set of the third image. The third image is encoded after the first image. The third image has a different time ID than the second image. The NAL unit type of the third image is not the NAL unit type of the STSA image.

2. A decoding method, wherein, The first or second action is executed based on the network abstraction layer (NAL) unit type. The first action mentioned above includes: Decode the first image; and After decoding the parameter set of the second image, the second image itself is decoded. The second image follows the first image in the decoding order. The parameter set of the second image and the first image share the same time ID. This time ID indicates the layer of time scalability. The NAL unit type of the second image is the NAL unit type for step-time sublayer access to STSA images. The second action mentioned above includes: Decode the first image mentioned above; and After decoding the first image, the third image is decoded without decoding the parameter set of the third image. The third image is after the first image in the decoding order. The third image has a different time ID than the second image. The NAL unit type of the third image is not the NAL unit type of the STSA image.

Images