Image filter device, filtering method, and dynamic image decoding device

Abstract

To realize an image filtering device which is capable of reducing block distortion while suppressing increase in memory size.SOLUTION: An adaptive offset filter (60) adds an offset to each pixel value of an input image. The adaptive offset filter refers to offset-type specifying information, sets offset attributes for a subject unit area of the input image, decodes an offset having bit width corresponding to an offset value range included in the set offset attributes, and adds the offset to each pixel value of the input image.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an image filter device for filtering an image, an offset decoding device for decoding an offset referenced by an image filter, an offset encoding device for encoding an offset referenced by an image filter, and a data structure of encoded data. [Background technology]

[0002] In order to efficiently transmit or record moving images, a moving image coding device (coding device) that generates coded data by coding moving images, and a moving image decoding device (decoding device) that generates decoded images by decoding the coded data are used. Specific moving image coding methods include, for example, a method adopted in H.264 / MPEG-4.AVC, a method adopted in KTA software, which is a codec jointly developed by VCEG (Video Coding Expert Group), a method adopted in TMuC (Test Model under Consideration) software, which is a successor codec, and a method adopted in HM (HEVC Test Model) software.

[0003] In such coding methods, images (pictures) that make up a moving image are managed in a hierarchical structure consisting of slices obtained by dividing images, largest coding units (LCUs, also called tree blocks) obtained by dividing slices, coding units (CUs, also called coding nodes) obtained by dividing the largest coding units, and blocks and partitions obtained by dividing the coding units, and in many cases, pictures are coded using blocks as the smallest unit.

[0004] In addition, in such a coding method, a predicted image is usually generated based on a locally decoded image obtained by coding / decoding an input image, and difference data between the predicted image and the input image is coded. Also, as a method of generating a predicted image, a method called inter-picture prediction (inter prediction) and a method called intra-picture prediction (intra prediction) are known.

[0005] In intra prediction, a predicted image in the same frame is sequentially generated based on a local decoded image in the same frame. Specifically, in intra prediction, a prediction direction is usually selected from a group of prediction directions (prediction modes) that are determined in advance for each prediction unit (e.g., block), and a predicted pixel value in a prediction target area is generated by extrapolating a pixel value of a reference pixel in a local decoded image to the selected prediction direction. In inter prediction, a predicted image in a prediction target frame is generated for each prediction unit (e.g., block) by applying motion compensation using a motion vector to a reference image in a reference frame (decoded image) in which the entire frame is decoded.

[0006] Non-Patent Document 1 and Non-Patent Document 2 disclose an adaptive offset filter (also called "adaptive offset filter") that is introduced in the latter stage of a deblocking filter that reduces block distortion of a decoded image and in the former stage of an adaptive loop filter (also called "adaptive filter") that performs filtering using adaptively determined filter coefficients. This adaptive offset filter adds an adaptively set offset to each pixel value of the image output from the deblocking filter.

[0007] By providing such an adaptive offset filter, block distortion can be suppressed more effectively. [Prior art documents] [Non-patent literature]

[0008] [Non-Patent Document 1] "JCTVC-D122",Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISO / IEC JTC1 / SC29 / WG11, 4th Meeting: Daegu, KR, 01 / 2011 [Non-Patent Document 2] "JCTVC-E049",Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISO / IEC JTC1 / SC29 / WG11, 5th Meeting: Geneva, CH, 03 / 2011 Summary of the Invention [Problem to be solved by the invention]

[0009] However, the offset used in conventional adaptive offset filters has no set value range and has a large number of bits, which poses the problem of requiring a large memory size to store the offset.

[0010] The present invention has been made in view of the above problems, and has an object to realize an image filter device capable of reducing block distortion while suppressing an increase in memory size. [Means for solving the problem]

[0011] In order to solve the above problems, the image filter device of the present invention is an image filter device that adds an offset to each pixel value of an input image composed of a plurality of unit areas, and is characterized in that it comprises an offset attribute setting means that refers to encoded data and sets an offset value range, an offset decoding means that decodes an offset limited to the set offset value range, and a filter means that adds the offset to each pixel value of the input image.

[0012] According to the image filter device configured as described above, the offset attribute setting means sets an offset value range, and the offset decoding means decodes an offset having a bit width according to the set offset value range, thereby effectively reducing the memory size for storing the offset.

[0013] Therefore, according to the above configuration, it is possible to perform appropriate offset filtering while reducing the memory size for storing offsets.

[0014] Furthermore, an offset decoding device according to the present invention is an offset decoding device that decodes each offset referenced by an image filter that adds an offset to each pixel value of an input image, and is characterized in that it comprises: offset residual decoding means that decodes each offset residual from encoded data; predicted value derivation means that derives a predicted value of each offset from an already decoded offset; and offset calculation means that calculates each offset from the predicted value derived by the predicted value derivation means and the offset residual decoded by the offset residual decoding means.

[0015] The offset decoding device configured as described above includes offset residual decoding means which decodes each offset residual from encoded data, predicted value derivation means which derives a predicted value of each offset from an already decoded offset, and offset calculation means which calculates each offset from the predicted value derived by the predicted value derivation means and the offset residual decoded by the offset residual decoding means. Therefore, it is possible to appropriately decode offsets from encoded data with a smaller amount of code compared to the case where each offset is encoded as is.

[0016] Furthermore, an image filter device according to the present invention is an image filter device that acts on an input image, and is characterized in that it comprises: calculation means for calculating a difference value between a pixel value of a target pixel in the input image and a pixel value of a pixel surrounding the target pixel; bit shift means for bit-shifting the pixel value referred to by the calculation means or the difference value calculated by the calculation means by a predetermined shift value to the right; classification means for classifying the target pixel into one of a plurality of offset classes depending on the magnitude relationship between the difference value bit-shifted to the right by the bit shift means and 0; and offset means for adding an offset associated with the offset class classified by the classification means to the pixel value of the target pixel.

[0017] According to the image filter device configured as described above, the target pixel is classified into one of a plurality of offset classes according to the magnitude relationship between the right bit-shifted difference value and 0, and an offset associated with the offset class classified by the classification means is added to the pixel value of the target pixel, so that the class classification process is less susceptible to the effects of noise and coding efficiency is improved.

[0018] Furthermore, an image filter device according to the present invention is an image filter device that acts on an input image, and is characterized by comprising: a calculation means for calculating a difference value between a pixel value of a target pixel in the input image and pixel values ​​of pixels surrounding the target pixel; a classification means for classifying the target pixel into one of a plurality of offset classes depending on the magnitude relationship between the difference value calculated by the calculation means and predetermined first and second threshold values; and an offset means for adding an offset associated with the offset class classified by the classification means to the pixel value of the target pixel.

[0019] The image filter device configured as described above classifies the target pixel into one of a plurality of offset classes depending on the magnitude relationship between the difference value calculated by the calculation means and predetermined first and second threshold values, and adds an offset associated with the offset class classified by the classification means to the pixel value of the target pixel, so that the class classification process is less susceptible to the effects of noise and encoding efficiency is improved.

[0020] In addition, an image filter device according to the present invention is an image filter device that acts on an input image composed of a plurality of unit areas, and includes a determination means for determining, from first and second offset types, an offset type to which a target unit area including a target pixel in the input image belongs; a classification means for classifying the target pixel into either an offset class that does not add an offset or a plurality of offset classes to which an offset is added, depending on the offset type to which the target unit area belongs and the pixel value of the target pixel; and an offset means for adding, to the pixel value of the target pixel, an offset associated with the offset type to which the target unit area belongs and the offset class classified by the classification means, wherein, when the pixel value of the target pixel is within a predetermined range, the classification means classifies the target pixel into an offset class to which an offset is added, regardless of whether the offset type to which the unit area including the target pixel belongs is the first or second offset type.

[0021] According to the image filter device configured as above, when the pixel value of the target pixel is within a predetermined range, the target pixel is classified into an offset class to which an offset is added regardless of whether the offset type to which the unit area including the target pixel belongs is the first or second offset type, so that block noise can be effectively removed. Therefore, according to the configuration, it is possible to improve the coding efficiency.

[0022] Furthermore, an image filter device according to the present invention is an image filter device that adds an offset to each pixel value of an input image composed of a plurality of unit areas, and is characterized in comprising a determination means that determines an offset type to which a unit area to be processed belongs among a plurality of offset types, an offset encoding means that determines an offset having a different bit width according to the offset type and encodes the offset, and a filter means that adds the determined offset to each pixel value of the input image.

[0023] According to the image filter device configured as above, an offset type to which a unit region to be processed belongs is determined from among a plurality of offset types, offsets having different bit widths are determined according to the determined offset type, the determined offsets are added to each pixel of the input image, and the determined offsets are encoded.

[0024] Therefore, according to the above configuration, it is possible to perform appropriate offset filtering while reducing the memory size for storing offsets. Also, according to the above configuration, the amount of coding data is reduced, thereby improving coding efficiency.

[0025] Furthermore, an offset coding device according to the present invention is an offset coding device that codes each offset referenced by an image filter that adds an offset to each pixel value of an input image, and is characterized in comprising: predicted value derivation means that derives a predicted value of each offset from an already-coded offset; offset residual calculation means that calculates an offset residual from each offset and the predicted value derived by the predicted value derivation means; and offset residual coding means that codes the offset residual calculated by the offset residual calculation means.

[0026] The offset coding device configured as described above is provided with predicted value derivation means for deriving a predicted value of each offset from an already-encoded offset, offset residual calculation means for calculating an offset residual from each offset and the predicted value derived by the predicted value derivation means, and offset residual coding means for coding the offset residual calculated by the offset residual calculation means, so that the amount of code of encoded data can be reduced.

[0027] Furthermore, the data structure of the encoded data according to the present invention is a data structure of encoded data referenced by an image filter that adds an offset to each pixel value of an input image composed of a plurality of unit areas, and includes offset type designation information that designates the offset type to which each unit area belongs, and an offset having a different bit width depending on the offset type, and the image filter refers to the offset type designation information included in the encoded data, determines the offset type to which the unit area to be processed belongs, and decodes the offset having a different bit width depending on the determined offset type.

[0028] The coded data configured as above includes offsets having different bit widths depending on the offset type, so that the amount of code of the coded data is reduced. Also, the image filter that decodes the coded data refers to the offset type designation information, determines the offset type to which the unit area to be processed belongs, and decodes the offsets having different bit widths depending on the determined offset type, so that it is possible to perform appropriate offset filter processing while reducing the memory size for storing the offsets.

[0029] The offset type designation information may be determined for each of the input images, or for each of the unit regions, or for each of a predetermined set of the input images, or for each of a predetermined set of the unit regions. Effect of the Invention

[0030] As described above, the image filter device of the present invention is an image filter device that adds an offset to each pixel value of an input image composed of a plurality of unit areas, and is equipped with an offset attribute setting means that refers to encoded data and sets an offset value range, an offset decoding means that decodes an offset limited to the set offset value range, and a filter means that adds the offset to each pixel value of the input image.

[0031] In addition, the image filter device of the present invention is an image filter device that adds an offset to each pixel value of an input image composed of a plurality of unit areas, and is equipped with a determination means that determines an offset type to which a unit area to be processed belongs among a plurality of offset types, an offset encoding means that determines an offset having a different bit width according to the offset type and encodes the offset, and a filter means that adds the determined offset to each pixel value of the input image.

[0032] Furthermore, the data structure of the encoded data according to the present invention is a data structure of encoded data referenced by an image filter that adds an offset to each pixel value of an input image composed of a plurality of unit areas, and includes offset type designation information that designates the offset type to which each unit area belongs, and an offset having a different bit width depending on the offset type, and the image filter refers to the offset type designation information included in the encoded data, determines the offset type to which the unit area to be processed belongs, and decodes the offset having a different bit width depending on the determined offset type.

[0033] According to the above configuration, it is possible to reduce the memory size for storing offsets while allowing the image filter to perform appropriate offset filtering. [Brief description of the drawings]

[0034] [Figure 1]1 is a block diagram showing a configuration of an adaptive offset filter according to a first embodiment of the present invention. [Diagram 2] A figure showing the data structure of encoded data generated by a video encoding device related to the first embodiment of the present invention and decoded by a video decoding device, where (a) to (d) respectively show the picture layer, slice layer, tree block layer, and CU layer, (e) shows the structure of QAOU information for a non-leaf QAOU, and (f) shows the structure of QAOU information for a leaf QAOU. [Diagram 3] 3 is a diagram showing each syntax included in offset information OI of encoded data according to the first embodiment of the present invention. FIG. [Figure 4] 1A to 1E are diagrams showing the manner of division of offset units according to the first embodiment of the present invention, where (a) shows the case where sao_curr_depth=0, (b) shows the case where sao_curr_depth=1, (c) shows the case where sao_curr_depth=2, (d) shows the case where sao_curr_depth=3, and (e) shows the case where sao_curr_depth=4. [Diagram 5] 1 is a block diagram showing a configuration of a video decoding device according to a first embodiment of the present invention. [Figure 6] A figure for explaining a first embodiment of the present invention, where (a) shows each QAOMU of a division depth of 3 that constitutes the target processing unit and the QAOU index assigned to each QAOMU, and (b) shows the offset type associated with each QAOU index 0 to 9 and the offset for each class that can be selected for each offset type. [Figure 7]A figure showing an example of QAOMU numbers assigned to QAOMUs included in a target processing unit in the first embodiment of the present invention, where (a) shows a QAOMU number assigned to a QAOMU with a partitioning depth of 0, (b) shows a QAOMU number assigned to a QAOMU with a partitioning depth of 1, (c) shows a QAOMU number assigned to a QAOMU with a partitioning depth of 2, (d) shows a QAOMU number assigned to a QAOMU with a partitioning depth of 3, and (e) shows a QAOMU number assigned to a QAOMU with a partitioning depth of 4. [Figure 8] 5 is a flowchart showing a flow of processing by an adaptive offset filter processing unit according to the first embodiment of the present invention. [Figure 9] FIG. 11 shows examples of offset bit depths and shift values ​​set by an offset attribute setting unit provided in an adaptive filter according to a first embodiment of the present invention, together with pixel bit depths, where (a) to (d) show examples corresponding to patterns S1 to S4, respectively, and (e) shows the case where STEP=2 is taken in (d). [Figure 10] FIG. 1 is a diagram for explaining offset processing by the adaptive offset filter according to the first embodiment of the present invention, where (a) to (d) show pixels to be referenced when sao_type_idx=1 to 4, respectively. [Figure 11] FIG. 11 is a diagram for explaining offset processing by an adaptive offset filter according to a first embodiment of the present invention, in which (a) shows a graph showing the magnitude relationship between the pixel value pic[x] of the pixel x to be processed and the pixel value of pixel a or b, and the value of the function Sign corresponding to this magnitude relationship, (b) shows a graph showing the magnitude relationship between the pixel value of the pixel x to be processed and the pixel values ​​of pixels a and b, and the value of EdgeType corresponding to this magnitude relationship, (c) shows the correspondence between each graph shown in (b) and class_idx, and (d) shows a conversion table indicating the conversion from EdgeType to class_idx. [Figure 12]FIG. 11 is a diagram for explaining offset processing by an adaptive offset filter according to the first embodiment of the present invention, in which (a) shows a schematic diagram of class classification when sao_type_idx=5, (b) shows a schematic diagram of class classification when sao_type_idx=6, and (c) is a table showing an example of class classification when a band offset is specified. [Figure 13] FIG. 13 is a diagram for explaining offset processing by the adaptive offset filter according to the first embodiment of the present invention, and is a table showing another example of class classification when a band offset is specified. [Figure 14] 1 is a block diagram showing a configuration of a video encoding device according to a first embodiment of the present invention. [Figure 15] 1 is a block diagram showing a configuration of an adaptive offset filter included in a video encoding device according to a first embodiment of the present invention. [Figure 16] 5 is a flowchart showing a flow of processing by an offset calculation unit of an adaptive offset filter included in the video encoding device according to the first embodiment of the present invention. [Figure 17] 5 is a flowchart showing a flow of processing by an offset information selection unit of an adaptive offset filter included in the video encoding device according to the first embodiment of the present invention. [Figure 18] FIG. 11 is a diagram for explaining processing by an offset information selection unit of an adaptive offset filter provided in a video encoding device according to a first embodiment of the present invention, in which (a) shows the manner of division when the division depth is 0 and 1, (b) shows the manner of division when the division depth is 1, (c) shows the manner of division when the division depth is 2, and (d) shows an example of division determined by the offset information selection unit. [Figure 19] FIG. 11 is a block diagram showing a configuration of an adaptive offset filter included in a video decoding device according to a second embodiment of the present invention. [Figure 20]FIG. 11 is a diagram for explaining an adaptive offset filter according to a second embodiment of the present invention, in which (a) shows a first specific example of a function merge_tbl[sao_type_idx], and (b) shows a second specific example of the function merge_tbl[sao_type_idx]. [Figure 21] FIG. 11 is a block diagram showing a configuration of an adaptive offset filter included in a video encoding device according to a second embodiment of the present invention. [Figure 22] FIG. 11 is a block diagram showing a configuration of an adaptive offset filter according to a third embodiment of the present invention. [Diagram 23] FIG. 13 is a diagram showing the syntax of offset information OI of the encoded data according to the third embodiment. [Figure 24] A figure for explaining the above-mentioned third embodiment, where (a) is a figure showing each QAOMU of a division depth of 3 that constitutes the target processing unit and the QAOU index assigned to each QAOMU, and (b) is a figure showing the offset type associated with each of the QAOU indexes 0 to 9 and the offsets related to each class that can be selected for each offset type. [Diagram 25] FIG. 13 is a diagram showing a conversion table used by an offset information decoding unit according to the third embodiment. [Figure 26] FIG. 11 is a diagram for explaining offset processing by an adaptive offset filter according to the third embodiment, in which (a) is a graph showing the magnitude relationship between the pixel value pic[x] of the pixel x to be processed and the pixel value of pixel a or b, and a diagram showing the value of the function Sign according to this magnitude relationship, (b) is a graph showing the magnitude relationship between the pixel value of the pixel x to be processed and the pixel values ​​of pixels a and b, and a diagram showing the value of EdgeType according to this magnitude relationship, (c) is a diagram showing the correspondence between each graph shown in (b) and class_idx, and (d) to (f) are diagrams showing conversion tables indicating the conversion from EdgeType to class_idx. [Figure 27]FIG. 13 is a diagram for explaining offset processing by the adaptive offset filter according to the third embodiment, in which (a) is a diagram showing a schematic diagram of class classification when sao_type_idx=5, and (b) is a diagram showing a schematic diagram of class classification when sao_type_idx=6. [Figure 28] 11A and 11B are diagrams for explaining offset processing by an adaptive offset filter in the third embodiment, in which (a) is a diagram showing a schematic diagram of class classification when the hierarchical depth of the QAOU to be processed is less than a threshold, and (b) is a diagram showing a schematic diagram of class classification when the hierarchical depth of the QAOU to be processed is greater than or equal to the threshold. [Figure 29] 11A and 11B are diagrams showing an example of class classification when a band offset is specified in the third embodiment, where (a) is a diagram showing an example of class classification when the hierarchical depth of the QAOU to be processed is less than a threshold, and (b) is a diagram showing an example of class classification when the hierarchical depth of the QAOU to be processed is greater than or equal to the threshold. [Diagram 30] FIG. 13 is a block diagram showing a configuration of an adaptive offset filter included in the video encoding device according to the third embodiment. [Diagram 31] FIG. 13 is a diagram showing an overview of calculating squared errors for each offset type for a QAOU with a QAOU index of "x" in the third embodiment. [Diagram 32] 13A and 13B are diagrams for explaining a configuration for switching between EO and BO depending on pixel values ​​in a fourth embodiment of the present invention, where (a) is a diagram for explaining an overview of a configuration for switching between EO and BO depending on pixel values, (b) is a diagram for explaining specific switching values, and (c) is a diagram showing the contents of a list memory stored in offset information storage unit 621. [Diagram 33]13A and 13B are diagrams for explaining a case in which the EO type is limited to the horizontal direction in the fifth embodiment of the present invention, where (a) is a diagram for explaining an overview of the effect, (b) is a diagram for explaining the position of pixels in the horizontal direction, (c) is a diagram for explaining a state different from (b), and (d) is a diagram for explaining the effect of (c). [Diagram 34] FIG. 13 is a diagram for explaining a case where the EO type is limited to the horizontal direction in the above-mentioned fifth embodiment, in which (a) and (b) are diagrams showing a case where the reference pixel is in an asymmetric position, and (c) is a diagram showing an overview of a horizontal edge. [Diagram 35] FIG. 23 is a diagram showing an overview of a case in which the accuracy of offset is improved in the sixth embodiment according to the present invention. [Diagram 36] FIG. 13 is a diagram showing an overview of a case where class classification is subdivided in the sixth embodiment, where (a) is a diagram showing a conversion table, and (b) to (d) are diagrams for explaining class division. [Figure 37] 13A and 13B are diagrams for explaining the case where class classification is performed according to color difference in the sixth embodiment, where (a) is a diagram for explaining class classification taking into account achromatic pixel values, (b) is a diagram for explaining asymmetric class classification in two value ranges that sandwich the achromatic pixel values, (c) and (d) are diagrams for explaining different class classification for each color channel (Cr or Cb), and (e) is a diagram for explaining the case where BO class classification is reduced to one. [Figure 38] FIG. 23 is a block diagram showing a configuration of an offset information decoding unit in the seventh embodiment according to the present invention. [Figure 39] FIG. 23 is a diagram showing an overview of a case in which there is a class to which no pixels can be classified in the seventh embodiment. [Diagram 40] FIG. 23 is a diagram showing syntax when a prediction candidate flag is used in the seventh embodiment. [Diagram 41] FIG. 13 is a block diagram showing a configuration of an offset information decoding unit according to the fifth embodiment. [Diagram 42]FIG. 13(a) is a block diagram showing a configuration of a use offset type selection unit according to the fifth embodiment, and FIG. 13(b) is a block diagram showing a configuration of another use offset type selection unit. [Diagram 43] FIG. 13 is a block diagram showing a configuration of a classifying unit according to the fifth embodiment. [Diagram 44] 11A and 11B are diagrams showing the syntax of offset information and QAOU information in the third embodiment, where (a) is a diagram showing the syntax of offset information, (b) is a diagram showing the syntax of QAOU information, and (c) is a diagram showing the syntax of the entire adaptive offset filter that calls the syntax shown in (a) and (b). [Diagram 45] 1A and 1B are diagrams for explaining that a video decoding device and a video encoding device can be used to transmit and receive video, where (a) is a block diagram showing the configuration of a transmitting device equipped with a video encoding device, and (b) is a block diagram showing the configuration of a receiving device equipped with a video decoding device. [Figure 46] This figure explains that a video decoding device and a video encoding device can be used to record and play back video, where (a) is a block diagram showing the configuration of a recording device equipped with a video encoding device 2, and (b) is a block diagram showing the configuration of a playback device equipped with a video decoding device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0035] [First embodiment] (Encoded Data #1) Prior to a detailed description of the video encoding device 2 and video decoding device 1 according to this embodiment, the data structure of encoded data #1 generated by the video encoding device 2 and decoded by the video decoding device 1 will be described.

[0036] 2 is a diagram showing the data structure of the coded data #1. The coded data #1 illustratively includes a sequence and a plurality of pictures that make up the sequence.

[0037] The structure of the layers below the picture layer in the coded data #1 is shown in Fig. 2. (a) to (d) in Fig. 2 respectively show a picture layer that defines a picture PICT, a slice layer that defines a slice S, a tree block layer that defines a tree block TBLK, and a CU layer that defines a coding unit (CU) included in the tree block TBLK.

[0038] (Picture Layer) The picture layer defines a set of data to be referenced by the video decoding device 1 in order to decode a picture PICT to be processed (hereinafter also referred to as a target picture). As shown in (a) of FIG. 2, the picture PICT includes a picture header PH and slices S1 to S NS where NS is the total number of slices contained in the picture PICT.

[0039] In the following, slices S1 to S NS When there is no need to distinguish between the above, the subscripts of the codes may be omitted. The same applies to other data that are included in the encoded data #1 described below and have subscripts.

[0040] The picture header PH includes a set of coding parameters to be referred to by the video decoding device 1 in order to determine a decoding method for a target picture. For example, coding mode information (entropy_coding_mode_flag) indicating the variable-length coding mode used by the video coding device 2 during coding is an example of the coding parameters included in the picture header PH.

[0041] When entropy_coding_mode_flag is 0, the picture PICT is coded by CAVLC (Context-based Adaptive Variable Length Coding). When entropy_coding_mode_flag is 1, the picture PICT is coded by CABAC (Context-based Adaptive Binary Arithmetic Coding).

[0042] The picture header PH is also called a Picture Parameter Set (PPS).

[0043] (Slice Layer) The slice layer defines a set of data to be referred to by the video decoding device 1 in order to decode a slice S to be processed (also referred to as a target slice). As shown in (b) of FIG. 2, the slice S includes a slice header SH and tree blocks TBLK1 to TBLK2. NC (NC is the total number of treeblocks contained in slice S).

[0044] The slice header SH includes a group of coding parameters to be referred to by the video decoding device 1 in order to determine a decoding method for the current slice. Slice type designation information (slice_type) that designates a slice type is an example of a coding parameter included in the slice header SH.

[0045] Slice types that can be specified by the slice type specification information include (1) an I slice that uses only intra prediction when encoding, (2) a P slice that uses unidirectional prediction or intra prediction when encoding, and (3) a B slice that uses unidirectional prediction, bidirectional prediction, or intra prediction when encoding.

[0046] Furthermore, the slice header SH includes filter parameters FP referenced by an adaptive filter included in the video decoding device 1. Note that the filter parameters FP may be configured to be included in the picture header PH.

[0047] (Treeblock Layer) In the tree block layer, a set of data to be referred to by the video decoding device 1 in order to decode a tree block TBLK to be processed (hereinafter also referred to as a target tree block) is defined. The tree block is also called a Largest Coding Unit (LCU).

[0048] The tree block TBLK includes a tree block header TBLKH and coding unit information CU1 to CU NL (NL is the total number of coding unit information included in the tree block TBLK.) First, the relationship between the tree block TBLK and the coding unit information CU will be described below.

[0049] The tree block TBLK is divided into partitions to specify block sizes for each process of intra prediction or inter prediction and transform.

[0050] The partitions of the tree block TBLK are divided by a recursive quadtree division. Hereinafter, the tree structure obtained by this recursive quadtree division is called a coding tree.

[0051] Hereinafter, a partition corresponding to a leaf, which is a terminal node of a coding tree, is referred to as a coding node. Since a coding node is a basic unit of coding processing, a coding node is also referred to as a coding unit (CU) below.

[0052] That is, coding unit information (hereinafter referred to as CU information) CU1 to CU NLis information corresponding to each coding node (coding unit) obtained by recursively dividing the tree block TBLK into quadtrees.

[0053] The root of the coding tree is associated with the tree block TBLK. In other words, the tree block TBLK is associated with the top node of a tree structure of a quadtree division that recursively includes a plurality of coding nodes.

[0054] The size of each coding node is half in both length and width of the size of the coding node to which the coding node directly belongs (that is, the partition of the node one layer above the coding node in question).

[0055] Also, the size that each coding node can have depends on the coding node size designation information and maximum hierarchical depth included in the sequence parameter set SPS of the coded data #1. For example, if the size of the tree block TBLK is 64×64 pixels and the maximum hierarchical depth is 3, the coding nodes in the hierarchy below the tree block TBLK can have one of three sizes, namely, 64×64 pixels, 32×32 pixels, and 16×16 pixels.

[0056] (Tree Block Header) The tree block header TBLKH includes coding parameters that the video decoding device 1 refers to in order to determine a decoding method for the target tree block. Specifically, as shown in (c) of Fig. 2, the tree block header TBLKH includes tree block division information SP_TBLK that specifies a division pattern of the target tree block into CUs, and a quantization parameter difference Δqp (qp_delta) that specifies the size of the quantization step.

[0057] The tree block division information SP_TBLK is information that represents a coding tree for dividing a tree block, and specifically, is information that specifies the shape, size, and position within the target tree block of each CU included in the target tree block.

[0058] The tree block division information SP_TBLK may not explicitly include the shape and size of the CU. For example, the tree block division information SP_TBLK may be a set of flags (split_coding_unit_flag) indicating whether to divide the entire target tree block or a partial area of ​​the tree block into four. In this case, the shape and size of each CU can be specified by using the shape and size of the tree block.

[0059] Furthermore, the quantization parameter difference Δqp is the difference qp-qp' between the quantization parameter qp in the current tree block and the quantization parameter qp' in the tree block encoded immediately before the current tree block.

[0060] (CU layer) In the CU layer, a set of data to be referenced by the video decoding device 1 in order to decode a CU to be processed (hereinafter also referred to as a target CU) is defined.

[0061] Before describing the specific contents of the data included in the CU information CU, the tree structure of the data included in the CU will be described. The coding node is the root node of a prediction tree (PT) and a transform tree (TT). The prediction tree and the transform tree will be described as follows.

[0062] In the prediction tree, a coding node is divided into one or more prediction blocks, and the position and size of each prediction block are specified. In other words, a prediction block is one or more non-overlapping regions that constitute a coding node. In addition, the prediction tree includes one or more prediction blocks obtained by the above-mentioned division.

[0063] The prediction process is performed for each prediction block. Hereinafter, the prediction block, which is the unit of prediction, is also referred to as a prediction unit (PU).

[0064] Roughly speaking, there are two types of division in the prediction tree: intra prediction and inter prediction.

[0065] In the case of intra prediction, the division method includes 2N×2N (same size as the coding node) and N×N.

[0066] In the case of inter prediction, the division method includes 2N×2N (same size as the coding node), 2N×N, N×2N, and N×N.

[0067] In addition, in the transform tree, the coding node is divided into one or more transform blocks, and the position and size of each transform block are specified. In other words, a transform block is one or more non-overlapping regions that constitute a coding node. The transform tree includes one or more transform blocks obtained by the above division.

[0068] The division of the transform tree can be performed by allocating an area of ​​the same size as the coding node as a transform block, or by recursive quad-tree division, similar to the division of the tree block described above.

[0069] The transform process is performed for each transform block. Hereinafter, the transform block, which is the unit of transform, is also referred to as a transform unit (TU).

[0070] (CU information data structure) Next, the specific contents of the data included in the CU information CU will be described with reference to (d) of Fig. 2. As shown in (d) of Fig. 2, the CU information CU specifically includes a skip flag SKIP, PT information PTI, and TT information TTI.

[0071] The skip flag SKIP is a flag indicating whether or not the skip mode is applied to the target PU, and when the value of the skip flag SKIP is 1, that is, when the skip mode is applied to the target CU, the PT information PTI and the TT information TTI in the CU information CU are omitted. Note that the skip flag SKIP is omitted in an I slice.

[0072] The PT information PTI is information about a PT included in a CU. In other words, the PT information PTI is a collection of information about one or more PUs included in a PT, and is referred to when a predicted image is generated by the video decoding device 1. As shown in (d) of FIG. 2, the PT information PTI includes prediction type information PType and prediction information PInfo.

[0073] The prediction type information PType is information that specifies whether intra prediction or inter prediction is to be used as a predicted image generation method for the target PU.

[0074] The prediction information PInfo is composed of intra prediction information or inter prediction information depending on which prediction method is specified by the prediction type information PType. Hereinafter, a PU to which intra prediction is applied is also referred to as an intra PU, and a PU to which inter prediction is applied is also referred to as an inter PU.

[0075] Furthermore, the prediction information PInfo includes information specifying the shape, size, and position of the target PU. As described above, the predicted image is generated for each PU. The prediction information PInfo will be described in detail later.

[0076] The TT information TTI is information about a TT included in a CU. In other words, the TT information TTI is a collection of information about one or more TUs included in a TT, and is referred to when the video decoding device 1 decodes residual data. In the following, a TU may also be referred to as a block.

[0077] As shown in (d) of FIG. 2, the TT information TTI includes TT division information SP_TT that specifies a division pattern of the target CU into each transform block, and quantized prediction residuals QD1 to QD NT (NT is the total number of blocks contained in the target CU).

[0078] Specifically, the TT division information SP_TT is information for determining the shape, size, and position of each TU included in the target CU within the target CU. For example, the TT division information SP_TT can be realized by information (split_transform_unit_flag) indicating whether or not to divide the target node, and information (trafoDepth) indicating the depth of the division.

[0079] Also, for example, when the size of a CU is 64×64, each TU obtained by division can have a size ranging from 32×32 pixels to 2×2 pixels.

[0080] Each quantized prediction residual QD is coded data generated by the video coding device 2 applying the following processes 1 to 3 to a target block that is a block to be processed.

[0081] Process 1: The prediction residual obtained by subtracting the prediction image from the image to be coded is subjected to DCT (Discrete Cosine Transform); Process 2: quantize the transform coefficients obtained in process 1; Process 3: Variable-length coding the transform coefficients quantized in process 2; The quantization parameter qp mentioned above represents the size of the quantization step QP used when the video encoding device 2 quantizes the transform coefficients (QP=2 qp / 6 ).

[0082] (Prediction information PInfo) As described above, there are two types of prediction information PInfo: inter prediction information and intra prediction information.

[0083] The inter prediction information includes coding parameters to be referred to when the video decoding device 1 generates an inter prediction image by inter prediction. More specifically, the inter prediction information includes inter PU division information that specifies a division pattern of the target CU into each inter PU, and inter prediction parameters for each inter PU.

[0084] The inter prediction parameters include a reference picture index, an estimated motion vector index, and a motion vector residual.

[0085] On the other hand, the intra prediction information includes coding parameters that are referred to when the video decoding device 1 generates an intra prediction image by intra prediction. More specifically, the intra prediction information includes intra PU division information that specifies a division pattern of the target CU into each intra PU, and intra prediction parameters for each intra PU. The intra prediction parameters are parameters for specifying an intra prediction method (prediction mode) for each intra PU.

[0086] (Offset units) In this embodiment, each picture or each slice is recursively divided into a plurality of offset units (also called Quad Adaptive Offset Units (QAOUs)) by a quadtree structure. Here, a QAOU is a processing unit of offset filter processing by the adaptive offset filter according to this embodiment.

[0087] As shown in Figures 2(e) to 2(f), the QAOU information, which is information about each QAOU, includes sao_split_flag, which indicates whether the QAOU is further divided or not. More specifically, sao_split_flag is specified by arguments (sao_curr_depth, ys, xs) described later, and is also represented as sao_split_flag[sao_curr_depth][ys][xs].

[0088] If the sao_split_flag included in a QAOU indicates that the QAOU is to be further split (i.e., if the QAOU is not a leaf), the QAOU information for that QAOU includes QAOU information for each of the multiple QAOUs included in that QAOU, as shown in Figure 2(e).

[0089] On the other hand, if the sao_split_flag included in a QAOU indicates that the QAOU will not be split any further (i.e., the QAOU is a leaf), the QAOU information for the QAOU includes offset information OI for the QAOU, as shown in Figure 2(f). Also, the offset information OI includes offset type designation information OTI that designates the offset type, and an offset group determined according to the offset type, as shown in Figure 2(f). Furthermore, the offset group includes multiple offsets, as shown in Figure 2(f).

[0090] The offset in the encoded data is a quantized value. The offset in the encoded data may be a prediction residual obtained by using some kind of prediction, for example, linear prediction. In addition, when the pixel bit depth described later is specified differently for each certain unit area, in this embodiment in which the bit depth of the offset, the offset value range, and the shift value are changed according to the pixel bit depth, it is also possible to include the pixel bit depth of the processing target area in the offset type specification information OTI.

[0091] The offset information OI will be described below with reference to different drawings.

[0092] FIG. 3 is a diagram showing each syntax included in the offset information OI (denoted as sao_offset_param() in FIG. 3).

[0093] 3, the offset information OI includes the syntax sao_type_idx[sao_curr_depth][ys][xs]. In addition, when sao_type_idx[sao_curr_depth][ys][xs] is not 0, the offset information OI includes sao_offset[sao_curr_depth][ys][xs][i].

[0094] (sao_curr_depth, ys, xs) The arguments sao_type_idx and sao_offset, sao_curr_depth, are indexes representing the division depth of the QAOU, and ys and xs are indexes for respectively representing the y-direction position and x-direction position of the QAOU (or the QAOMU described below).

[0095] 4(a) to (e) are diagrams showing the manner of division of QAOUs according to the value of sao_curr_depth, where (a) shows the case where sao_curr_depth = 0, (b) shows the case where sao_curr_depth = 1, (c) shows the case where sao_curr_depth = 2, (d) shows the case where sao_curr_depth = 3, and (e) shows the case where sao_curr_depth = 4. As shown in Fig. 4(a) to (e), each QAOU is specified by sao_curr_depth and (xs, ys).

[0096] As shown in FIG. 4(a), when sao_curr_depth=0, xs and ys are each 0. As shown in FIG. 4(b), when sao_curr_depth=1, xs and ys can each take on a value of 0 or 1. As shown in FIG. 4(c), when sao_curr_depth=2, xs and ys can each take on a value of 0, 1, 2, or 3. In general, for a given sao_curr_depth, xs and ys can each take on a value between 0 and 2. sao_curr_depth Can have a value of -1.

[0097] (sao_type_idx) sao_type_idx[sao_curr_depth][ys][xs] corresponds to the offset type specification information OTI described above, and is a syntax for specifying the offset type for each QAOU. Hereinafter, sao_type_idx may be simply referred to as the offset type.

[0098] In this embodiment, sao_type_idx[sao_curr_depth][ys][xs] takes an integer value ranging from 0 to 6. sao_type_idx[sao_curr_depth][ys][xs]=0 indicates that offset filter processing is not performed on the offset filter pre-image in the target QAOU (for example, a deblocked decoded image P_DB described later), sao_type_idx[sao_curr_depth][ys][xs]=1 to 4 indicates that edge offset processing is performed on the offset filter pre-image in the target QAOU, and sao_type_idx[sao_curr_depth][ys][xs]=5 to 6 indicates that band offset processing is performed on the offset filter pre-image in the target QAOU. Specific contents of edge offset processing and band offset processing will be described later.

[0099] (sao_offset) sao_offset[sao_curr_depth][ys][xs][i] is a syntax that represents a specific value of an offset that is added to each pixel included in a target QAOU in an offset filter process by an adaptive offset filter according to the present embodiment. In the present embodiment, sao_offset may be simply referred to as an offset.

[0100] sao_offset[sao_curr_depth][ys][xs][i] is specified by the arguments sao_curr_depth, ys, and xs, as well as the index i. Here, the index i is an index for specifying a class, and is also written as class_idx. When the value of sao_type_idx[sao_curr_depth][ys][xs] is any of 1 to 4 (i.e., in the case of edge offset), the index i takes any integer value of i=0 to 4. When the value of sao_type_idx[sao_curr_depth][ys][xs] is any of 5 to 6 (i.e., in the case of band offset), the index i takes any integer value of i=0 to 16. Note that, as described later, in either case, i=0 indicates that no offset is added.

[0101] As described below, the adaptive offset filter according to this embodiment classifies a target pixel included in a target QAOU into one of the above-mentioned multiple classes, and adds to the target pixel an offset Offset for the class into which the target pixel is classified.

[0102] In addition, the Descriptor ue(v) shown in Figure 3 indicates that the syntax associated with this descriptor is an unsigned numeric value, and the value is variable-length coded, while se(v) means that the syntax associated with this descriptor is a signed numeric value, and the sign and absolute value are separated and variable-length coded.

[0103] (Video Decoding Device 1) A video decoding device 1 according to this embodiment will be described below with reference to Fig. 1 and Fig. 5 to Fig. 13. The video decoding device 1 includes, as a part thereof, a method adopted in H.264 / MPEG-4.AVC, a method adopted in KTA software which is a codec jointly developed by VCEG (Video Coding Expert Group), a method adopted in TMuC (Test Model under Consideration) software which is a successor codec thereto, and a technology adopted in HM (HEVC Test Model) software.

[0104] Fig. 5 is a block diagram showing the configuration of a video decoding device 1. As shown in Fig. 5, the video decoding device 1 includes a variable-length code decoding unit 13, a motion vector restoration unit 14, a buffer memory 15, an inter-prediction image generating unit 16, an intra-prediction image generating unit 17, a prediction method determining unit 18, an inverse quantization and inverse transform unit 19, an adder 20, a deblocking filter 41, an adaptive filter 50, and an adaptive offset filter 60. The video decoding device 1 is a device for generating a video #4 by decoding encoded data #1.

[0105] The variable-length code decoding unit 13 decodes prediction parameters PP for each partition from the coded data #1. That is, for the inter-prediction partition, the reference image index RI, the estimated motion vector index PMVI, and the motion vector residual MVD are decoded from the coded data #1, and these are supplied to the motion vector restoration unit 14. On the other hand, for the intra-prediction partition, (1) size designation information designating the size of the partition, and (2) prediction index designation information designating the prediction index are decoded from the coded data #1, and these are supplied to the intra-prediction image generation unit 17. In addition, the variable-length code decoding unit 13 decodes CU information from the coded data, and supplies this to the prediction method determination unit 18 (not shown). Furthermore, the variable-length code decoding unit 13 decodes quantized prediction residual QD for each block, and quantization parameter difference Δqp for the tree block including the block, from the coded data #1, and supplies these to the inverse quantization and inverse transformation unit 19. Furthermore, the variable-length code decoding unit 13 extracts QAOU information from the coded data # 1 , and supplies the extracted QAOU information to the adaptive offset filter 60 .

[0106] The motion vector restoration unit 14 restores a motion vector mv for each inter-prediction partition from a motion vector residual MVD for that partition and a restored motion vector mv' for other partitions. Specifically, (1) an estimated motion vector pmv is derived from the restored motion vector mv' according to an estimation method specified by an estimated motion vector index PMVI, and (2) a motion vector mv is obtained by adding the derived estimated motion vector pmv and the motion vector residual MVD. The restored motion vector mv' for other partitions can be read from the buffer memory 15. The motion vector restoration unit 14 supplies the restored motion vector mv together with the corresponding reference image index RI to the inter-prediction image generation unit 16. For an inter-prediction partition that performs bidirectional prediction (weighted prediction), the two restored motion vectors mv1 and mv2 are supplied to the inter-prediction image generation unit 16 together with the corresponding reference image indexes RI1 and RI2.

[0107] The inter-prediction image generating unit 16 generates a motion compensation image mc for each inter-prediction partition. Specifically, using the motion vector mv supplied from the motion vector restoring unit 14, the motion compensation image mc is generated from the filtered decoded image P_FL' specified by the reference image index RI also supplied from the motion vector restoring unit 14. Here, the filtered decoded image P_FL' is an image obtained by performing a deblocking process by the deblocking filter 41, an offset filtering process by the adaptive offset filter 60, and an adaptive filtering process by the adaptive filter 50 on the decoded image P that has already been decoded, and the inter-prediction image generating unit 16 can read out the pixel values ​​of each pixel constituting the filtered decoded image P_FL' from the buffer memory 15. The motion compensation image mc generated by the inter-prediction image generating unit 16 is supplied to the prediction method determining unit 18 as an inter-prediction image Pred_Inter. In addition, for inter-prediction partitions that perform bidirectional prediction (weighted prediction), (1) a motion compensation image mc1 is generated from the filtered decoded image P_FL1' specified by the reference image index RI1 using the motion vector mv1, (2) a motion compensation image mc2 is generated from the filtered decoded image P_FL2' specified by the reference image index RI2 using the motion vector mv2, and (3) an inter-prediction image Pred_Inter is generated by adding an offset value to the weighted average of the motion compensation images mc1 and mc2.

[0108] The intra-prediction image generating unit 17 generates a prediction image Pred_Intra for each intra-prediction partition. Specifically, first, the prediction mode decoded from the encoded data #1 is referred to, and the prediction mode is assigned to the target partition, for example, in raster scan order. Then, the prediction image Pred_Intra is generated from the decoded image P according to the prediction method indicated by the prediction mode. The intra-prediction image Pred_Intra generated by the intra-prediction image generating unit 17 is supplied to the prediction method determining unit 18.

[0109] Furthermore, the intra-predicted image generation unit 17 supplies the adaptive filter 50 with intra-encoding mode information IEM, which is information indicating the size of the target partition and the prediction mode assigned to the target partition.

[0110] The prediction method determination unit 18 determines whether each partition is an inter-prediction partition on which inter prediction should be performed or an intra-prediction partition on which intra prediction should be performed, based on the CU information. In the former case, the prediction method determination unit 18 supplies the inter-prediction image Pred_Inter generated by the inter-prediction image generation unit 16 to the adder 20 as the predicted image Pred, and in the latter case, the prediction method determination unit 18 supplies the intra-prediction image Pred_Intra generated by the intra-prediction image generation unit 17 to the adder 20 as the predicted image Pred.

[0111] The inverse quantization and inverse transform unit 19 (1) inverse quantizes the quantized prediction residual QD, (2) performs inverse DCT (Discrete Cosine Transform) on the DCT coefficients obtained by the inverse quantization, and (3) supplies the prediction residual D obtained by the inverse DCT transformation to the adder 20. When inverse quantizing the quantized prediction residual QD, the inverse quantization and inverse transform unit 19 derives a quantization step QP from the quantization parameter difference Δqp supplied from the variable-length code decoding unit 13. The quantization parameter qp can be derived by adding the quantization parameter difference Δqp to the quantization parameter qp' related to the tree block that was previously inverse quantized / inverse DCT transformed, and the quantization step QP is calculated by subtracting QP from the quantization step qp by QP=2. pq / 6 Furthermore, the generation of the prediction residual D by the inverse quantization and inverse transform unit 19 is performed in units of blocks (transformation units).

[0112] The adder 20 generates a decoded image P by adding the predicted image Pred supplied from the prediction method determination unit 18 and the prediction residual D supplied from the inverse quantization and inverse transform unit 19 .

[0113] When a difference in pixel values ​​between adjacent pixels across a block boundary or a CU boundary in the decoded image P is smaller than a predetermined threshold, the deblocking filter 41 performs a deblocking process on the block boundary or the CU boundary in the decoded image P, thereby smoothing an image near the block boundary or the CU boundary. The image deblocked by the deblocking filter 41 is output to the adaptive offset filter 60 as a deblocked decoded image P_DB.

[0114] The adaptive offset filter 60 generates an offset-filtered decoded image P_OF by performing offset filtering using an offset decoded from the encoded data #1 on the deblocked decoded image P_DB supplied from the deblocking filter 41, with QAOU as a processing unit. The generated offset-filtered decoded image P_OF is supplied to the adaptive filter 50. A specific configuration of the adaptive offset filter 60 will be described later, and therefore will not be described here.

[0115] The adaptive filter 50 generates a filtered decoded image P_FL by performing a filter process using the filter parameters FP decoded from the encoded data #1 on the offset filtered decoded image P_OF supplied from the adaptive offset filter 60. The image filtered by the adaptive filter 50 is output to the outside as the filtered decoded image P_FL, and is stored in the buffer memory 15 in association with the POC designation information decoded from the encoded data by the variable length code decoding unit 13.

[0116] (Adaptive offset filter 60) 1 is a block diagram showing the configuration of an adaptive offset filter 60. As shown in FIG. 1, the adaptive offset filter 60 includes an adaptive offset filter information decoding unit 61 and an adaptive offset filter processing unit 62.

[0117] As shown in FIG. 1, the adaptive offset filter information decoding unit 61 includes an offset information decoding unit 611, a QAOU structure decoding unit 612, and an offset attribute setting unit 613.

[0118] The offset information decoding unit 611 refers to the QAOU information included in the coded data #1 and decodes the offset information OI included in the QAOU information. Also, the offset information decoding unit 611 supplies the values ​​of sao_type_idx[sao_curr_depth][ys][xs] and sao_offset[sao_curr_depth][ys][xs][i] obtained by decoding the offset information OI to the offset information storage unit 621 in association with the arguments (sao_curr_depth, ys, xs) and (sao_curr_depth, ys, xs, i), respectively.

[0119] The QAOU structure decoding unit 612 determines the division structure of the QAOU by decoding sao_split_flag[sao_curr_depth][ys][xs] included in the QAOU information, and supplies the QAOU structure information representing the determined division structure of the QAOU to the offset information storage unit 621.

[0120] The offset attribute setting unit 613 determines the bit depth of the offset (also called SAO_DEPTH) and the value range of the offset. Here, the bit depth of the offset is determined from the pixel bit depth (also called PIC_DEPTH) (not shown) input to the offset attribute setting unit 613. The pixel bit depth indicates the value range of the pixel values ​​constituting the input image of the adaptive offset filter 60 in bit width. When the pixel bit depth is N bits, the pixel value ranges from 0 to 2. N The pixel value ranges from 0 to 255. For example, when the pixel bit depth is 8 bits, the pixel value can range from 0 to 255. When the input image to the adaptive offset filter 60 is a decoded image from the video decoding device 1 or a locally decoded image from the video encoding device 2, the bit depth of the decoded image / locally decoded image is used. The precision of the input image and the precision of the output image of the adaptive offset filter 60 are the pixel bit depth.

[0121] The pixel bit depth can be used by decoding a value determined using a syntax in the coded data #1. For example, in the case of H.264 / AVC, it can be determined using bit_depth_luma_minus8 in the sequence parameter set. In addition, as another form, the pixel bit depth may be determined for each unit area to be processed, and in that case, the pixel bit depth is decoded from the header of the unit area. The pixel bit depth of the unit area may be included in parameter information or a header in the coded data #1 for decoding an input image of the adaptive offset filter 60, or may be included as part of the QAOU information. As parameter information and a header for decoding an input image, the pixel bit depth may be included in a picture parameter header, a slice header, an LCU, a CU, etc. Also, as part of the QAOU information, the pixel bit depth may be included in a leaf QAOU, or may be included in the QAOU information of a predetermined layer (for example, the QAOU of the highest layer or the QAOU of the first layer). Also, as the coding of the pixel bit depth, coding as a difference value from 8 bits is preferable. In addition to the pixel bit depth, it is also appropriate to directly code the shift value in the QAOU information. Also, it is appropriate to code the shift value only if it exceeds 8 bits. Here, the shift value indicates the bit shift amount required for inverse quantization. In addition to the shift value, the step value may be used as a parameter for inverse quantization. In this case, inverse quantization of the offset is performed by multiplying it with the step value, and quantization of the offset is performed by division by the step value.

[0122] Moreover, the offset bit depth is a value indicating the precision of the offset to be coded in the coded data #1. The offset included in the coded data #1 is quantized, and after decoding, this quantized offset is inversely quantized to a bit depth matching the pixel bit depth by an offset derivation unit 625 described below. Furthermore, the inversely quantized offset is added by an offset addition unit 626 described below. The offset bit depth has a value equal to or less than the pixel bit depth. If the offset bit depth is smaller than the pixel bit depth by k bits, the offset value to be coded is 2 k The offset deriving unit 625 converts the decoded bit depth (offset bit depth) into the pixel bit depth, as described below.

[0123] The offset attribute setting unit 613 further sets an offset value range according to the determined bit depth of the offset. Furthermore, a shift value is set from the pixel bit depth of the offset and the offset value range. The set offset value range and shift value are supplied to the offset information storage unit 621 and the offset information decoding unit 611. The offset bit depth is determined and the shift value is set in one of patterns S1 to S6, which will be described later. The offset value range is set in one of patterns C1 and C2, which will be described later. Attributes common to offsets, such as the offset value range and the shift value, are called offset attributes. On the other hand, as shown in FIG. 1, the adaptive offset filter processing unit 62 includes an offset information storage unit 621, a QAOU control unit 622, an offset type derivation unit 623, a class classification unit 624, an offset derivation unit 625, and an offset addition unit 626.

[0124] The offset information storage unit 621 is configured to manage and store the offset type specified for each QAOU and the specific values ​​of the offset for each class selectable for that offset type based on QAOU structure information, sao_type_idx[sao_curr_depth][ys][xs], and sao_offset[sao_curr_depth][ys][xs][i], and is equipped with a map memory and a list memory.

[0125] The map memory stores QAOU indexes (described later) that are assigned to each offset minimum unit (also referred to as QAOMU: Quad Adaptive Offset Minimum Unit) determined according to the division depth. The map memory is described in FIG. 6(a). FIG. 6(a) is for explaining an example of a QAOU index stored in the map memory, and is a diagram showing each QAOMU with a division depth of 3 that constitutes a target processing unit (for example, an LCU) and the QAOU indexes assigned to each QAOMU. In FIG. 6(a), indexes 0 to 9 are used to simply specify a QAOU without considering the division depth of the QAOU. In the example shown in FIG. 6(a), the QAOU specified by the QAOU index=I is denoted as QAOUI. Also, thin lines in FIG. 6(a) indicate boundaries of QAOMUs, and thick lines indicate boundaries of QAOUs.

[0126] As shown in Fig. 6(a), QAOU0 is composed of four QAOMUs, and these four QAOMUs are assigned a QAOU index of 0. On the other hand, QAOU3 is composed of one QAOMU, and this QAOMU is assigned a QAOU index of 3. In this way, the map memory stores the QAOU index assigned to each QAOMU.

[0127] On the other hand, the list memory stores each QAOU index, an offset type associated with the QAOU index, and a specific value of an offset for each class selectable in the offset type, in association with each other. The offset information storage unit 621 stores an offset whose value range is limited to an offset value range set by the offset attribute setting unit 613.

[0128] Fig. 6(b) is for explaining an example of information stored in the list memory, and shows offset types associated with each of the QAOU indexes 0 to 9 and offsets related to each class selectable for each offset type. "xxx" in Fig. 6(b) represents specific numerical values ​​that may differ from one another.

[0129] Also, "BO_1" in Fig. 6(b) represents the offset type specified by sao_type_idx=5. Also, "EO_1" represents the offset type specified by sao_type_idx=1. In this way, edge offsets, which are offset types specified by sao_type_idx=1, 2, 3, 4, are also written as EO_1, 2, 3, 4, respectively, and band offsets, which are offset types specified by sao_type_idx=5, 6, are also written as BO_1, 2, 3, 4, respectively.

[0130] 6(b), when the offset type is a band offset, the offsets stored in the list memory for that offset type are a total of 16, offset 1 to offset 16. Here, offset 1 to offset 16 refer to the values ​​specified by sao_offset[sao_curr_depth][ys][xs][1] to sao_offset[sao_curr_depth][ys][xs]

[16] , respectively, when the value of sao_type_idx[sao_curr_depth][ys][xs] is 5 or 6. On the other hand, when the offset type is an edge offset, the offsets stored in the list memory for that offset type are a total of four, offset 1 to offset 4. Here, offsets 1 to 4 refer to the values ​​specified by sao_offset[sao_curr_depth][ys][xs][1] to sao_offset[sao_curr_depth][ys][xs][4] when the value of sao_type_idx[sao_curr_depth][ys][xs] is 1, 2, 3, or 4. Nothing is stored in offsets 5 to 16.

[0131] In addition, the memory size of each offset stored in the list memory is determined by the offset value range supplied from the offset attribute setting unit 613. 4 ~2 4 If it is -1, each offset can be expressed with 5 bits, and a memory size of 5 bits is required.

[0132] Each QAOMU is assigned a QAOMU number, which allows each QAOMU to be distinguished from others. In the following, the QAOMU number is N Q QAOMU is QAOMUN Q It can also be written as:

[0133] The QAOU index is the number of the QAOU block specified from the partition depth 0 to the maximum partition depth. When the maximum partition depth is 4, the values ​​0 to 340 are specified by the QAOU index for all blocks of the partition depth (1+4+16+64+256=341 blocks), as shown in Figure 7.

[0134] In the offset information storage unit 621 used in the video decoding device 1, since it is not necessary to store offsets of all QAOUs of all division depths, it is not necessary to secure 341 memory areas, and it is sufficient to have memory areas for the number of QAOUs actually used in the structure specified by the input QAOU information. When the maximum division depth is 4, the number of blocks is 256 or less, so 256 map memories and list memories are sufficient. In this case, the QAOU index used in the offset information storage unit 621 is a unique index for identifying a leaf QAOU, for example, an index from 0 to 255 that is incremented by 1 each time a leaf QAOU is decoded. In addition, the offset information storage unit 621 can also store a map list in units of QAOU information corresponding to the maximum division depth. In this case, the QAOU index used in the offset information storage unit 621 uses numbers from 0 to 255, which correspond to QAOMU numbers 85 to 340 in FIG. 7.

[0135] Figures 7(a) to (e) are figures showing examples of QAOMU numbers assigned to QAOMUs included in a target processing unit (e.g., an LCU), where (a) shows a QAOMU number assigned to a QAOMU with a partitioning depth of 0, (b) shows a QAOMU number assigned to a QAOMU with a partitioning depth of 1, (c) shows a QAOMU number assigned to a QAOMU with a partitioning depth of 2, (d) shows a QAOMU number assigned to a QAOMU with a partitioning depth of 3, and (e) shows a QAOMU number assigned to a QAOMU with a partitioning depth of 4.

[0136] The QAOU control unit 622 controls each unit included in the adaptive offset filter processing unit 62. The QAOU control unit 622 also refers to the QAOU structure information, divides the deblocked decoded image P_DB into one or more QAOUs, and scans each QAOU in a predetermined order. The QAOU control unit 622 also supplies the offset type derivation unit 623 with a QAOMU number indicating the target QAOMU to be processed.

[0137] The offset type derivation unit 623 refers to the map memory and list memory of the offset information storage unit 621, and derives the offset type specified by the QAOMU number supplied from the QAOU control unit 622. In addition, it supplies the derived offset type to the class classification unit 624.

[0138] The class classification unit 624 classifies each pixel included in the target QAOU into one of a plurality of classes selectable in the offset type supplied from the offset type derivation unit 623. In addition, the class classification unit 624 supplies the offset type and a class index indicating the class into which each pixel has been classified to the offset derivation unit 625. Note that the specific classification process by the class classification unit 624 will be described later, and therefore will not be described here.

[0139] The offset derivation unit 625 refers to the list memory of the offset information storage unit 621, and derives an offset specified by the offset type and class index supplied from the classifying unit 624 for each pixel included in the target QAOU. It further includes an offset inverse shift unit (not shown) that bit-shifts the offset to the left by the shift value set by the offset attribute setting unit 613. The offset inverse shift unit inversely quantizes the offset so as to match the bit depth of the offset with the pixel bit depth. By performing such inverse quantization, it is possible to add the pixel value and the offset at the same bit depth in the addition process of the offset addition unit 626 described later. The inversely quantized offset for each pixel is supplied to the offset addition unit 626.

[0140] The offset addition unit 626 adds, to each pixel of the deblocked decoded image P_DB in the target QAOU, the offset supplied from the offset derivation unit 625. The offset addition unit 626 outputs an image obtained by performing processing on all QAOUs included in the deblocked decoded image P_DB as an offset-filtered decoded image P_OF.

[0141] FIG. 8 is a flowchart showing the flow of processing by the adaptive offset filter processor 62.

[0142] (Step S101) First, the QAOU control unit 622 acquires the QAOU structure information from the offset information storage unit 621 .

[0143] (Step S102) Next, the QAOU control unit 622 starts a first loop in which the QAOMU number of the target QAOMU to be processed is used as a loop variable.

[0144] (Step S103) The QAOU control unit 622 supplies a QAOMU number to the offset type derivation unit 623. Based on the control of the QAOU control unit 622, the offset type derivation unit 623 reads out the offset type specified by the QAOMU number supplied from the QAOU control unit 622 from the map memory and list memory of the offset information storage unit 621. In addition, the offset type derivation unit 623 supplies the read offset type to the class classification unit 624.

[0145] (Step S104) Next, the QAOU control unit 622 starts a second loop in which the pixel number of each pixel included in the target QAOMU is used as a loop variable. Here, the pixel number is used to distinguish the pixels included in the target QAOMU from each other, and for example, a pixel number assigned to each pixel included in the target QAOMU in a predetermined scan order can be used. Also, instead of such a pixel number, the x coordinate and y coordinate of each pixel included in the target QAOMU may be used as a loop variable.

[0146] (Step S105) Next, under the control of the QAOU control unit 622, the class classification unit 624 classifies the pixel to be processed into one of a plurality of classes selectable in the offset type supplied from the offset type derivation unit 623. In addition, the class classification unit 624 supplies the offset type and a class index indicating the class into which the pixel to be processed has been classified to an offset derivation unit 625.

[0147] (Step S106) Next, the offset derivation unit 625 reads an offset to be added to the pixel being processed from the offset information storage unit 621 under the control of the QAOU control unit 622. That is, it reads an offset specified by the offset type and class index supplied from the class classification unit 624. In addition, it supplies an offset that has been dequantized by bit-shifting the offset derived for the pixel being processed to the left by the shift value supplied from the offset attribute setting unit 613 to an offset addition unit 626.

[0148] (Step S107) Next, under the control of the QAOU control unit 622, the offset addition unit 626 adds the offset supplied from the offset derivation unit 625 to the pixel value of the processing target pixel of the deblocked decoded image P_DB.

[0149] (Step S108) This step is the end of the second loop.

[0150] (Step S109) This step is the end of the first loop.

[0151] In addition, in step S103, when the offset read by the offset type derivation unit 623 is 0 (offset type=0), the QAOU control unit 622 controls the offset addition unit 626 so as not to add an offset to each pixel of the QAOMU to be processed.

[0152] Furthermore, in step S105, if the target pixel is classified into class 0 (class index=0), the offset adding unit 626 is controlled so as not to add an offset to the target pixel.

[0153] (number of bits required to store the offset) Next, we will explain the number of bits required to store the offset (sao_offset). If the pixel bit depth is 10 bits and the offset bit depth is 9 bits, the offset is -2 9 ~2 9 The offset information storage unit 621 has a maximum memory size for storing the offset of 10 bits per picture. (Total number of QAOMUs per picture) x (Number of classes) x (Number of bits per offset) = 256 x 16 x 10 (bits) = 40960 (bits) The total number of QAOMUs per picture is 256 in the decoder, but 341 are used in the encoder described below, so a larger memory is required. In this way, if the offset range is not limited and the range allowed by the offset bit depth is used, a large memory size is required to store the offsets because the number of bits in the offsets is large.

[0154] (Relationship between SAO_DEPTH and coding efficiency) SAO_DEPTH and PIC_DEPTH are closely related to each other in terms of quantization error. The bit depth of the output image of the adaptive offset filter 60 is the pixel bit depth PIC_DEPTH, and SAO_DEPTH is the bit depth of the offset added to the pixel. Therefore, even if an offset with a precision exceeding the pixel bit depth is used, it will be discarded in the output process. Therefore, it is preferable that SAO_DEPTH, which is the offset precision, is set to PIC_DEPTH or less. Also, when SAO_DEPTH is smaller than PIC_DEPTH, the input image can only be corrected coarser than the precision (PIC_DEPTH) at which the filter can correct the image, and the filter effect is reduced.

[0155] Note that when the offset precision SAO_DEPTH is high, the offset code amount increases. In general, as can be understood from the fact that coding efficiency is optimized by minimizing the rate-distortion cost D+λR expressed using the code amount R of the coded data, the distortion D of the input image, and the weight λ, the offset precision has a negative effect on the distortion D and a positive effect on the rate R, so there is a trade-off in the precision and a specific optimal value.

[0156] Furthermore, in this embodiment, by limiting the quantized offset to an offset value range that can be expressed with a certain bit width, the bit width for storing the quantized offset in the offset information storage unit 621 can be limited. Compared to the case where no limit is imposed, the effect of reducing memory size can be obtained. However, excessively narrowing the offset value range reduces the effect of the offset in correcting the distortion of the decoded image, so that the offset addition process cannot remove the distortion of the decoded image, and the coding efficiency may decrease. Therefore, it is preferable to set the offset value range to an optimal range so that the coding efficiency does not decrease.

[0157] In this embodiment, the offset attribute setting unit 613 sets the offset bit depth and the offset shift value according to one of patterns S1 to S6, and sets the offset value range according to one of patterns C1 to C3.

[0158] (Pattern S1) In pattern S1, as shown in Fig. 9(a), the offset bit depth SAO_DEPTH is set equal to the pixel bit depth PIC_DEPTH. Since the maximum value of the offset precision is the pixel bit depth, pattern S1 encodes the offset with the maximum precision.

[0159] (Pattern S2) In pattern S2, as shown in Fig. 9(b), when PIC_DEPTH is 10 bits or less, SAO_DEPTH is set to be equal to PIC_DEPTH, and when PIC_DEPTH is 11 bits or more, SAO_DEPTH is set to be 10. In pattern S2, the upper limit of the offset bit depth is set to 10 bits. According to the inventor's knowledge, when the value of the quantization step QP of the decoded image is small (when the bit rate is high), the coding efficiency is higher when the offset bit depth is the same as the pixel bit depth compared to when the offset bit depth is small, and conversely, when the value of the quantization step QP is large, the coding efficiency is higher when the offset bit depth is smaller than the pixel bit depth than when it is the same as the pixel bit depth. In the inventor's experiment, it was confirmed that in the region where the quantization parameter qp is from 12 to 27, by determining the offset bit depth as in pattern S2, the coding efficiency is improved compared to when the offset bit depth is 8 bits when the pixel bit depth is 9 bits or less, and when the offset bit depth is 9 bits when the pixel bit depth is 10 bits or more. Therefore, by changing the dependency on the pixel bit depth at the boundary of 10 bits in pattern S2, the amount of offset code can be reduced compared to when the offset bit depth and the pixel bit depth are equal as in pattern S1, and high coding efficiency can be obtained.

[0160] (Pattern S3) In pattern S3, as shown in Fig. 9(c), when PIC_DEPTH is 9 bits or less, SAO_DEPTH is set equal to PIC_DEPTH, and when PIC_DEPTH is 10 bits or more, SAO_DEPTH is set to 9. In pattern S3, the upper limit of the offset bit depth is set to 9 bits. In pattern S3, as in pattern S2, the amount of offset coding can be reduced, making it possible to obtain high coding efficiency.

[0161] (Pattern S4) In pattern S4, as shown in FIG. 9(d), when PIC_DEPTH is 10 bits or less, SAO_DEPTH is equal to PIC_DEPTH, and when PIC_DEPTH is 11 bits or more, SAO_DEPTH is 10-floor((PIC_DEPTH-10) / STEP). Here, function floor(x) is a function that gives the largest integer less than or equal to x. In pattern S4, when the pixel bit depth is 11 bits or more, it means that the offset bit depth increases (decreases) by 1 bit every time the pixel bit increases (decreases) by STEP bits. FIG. 9(e) shows the case of STEP=2 in pattern S4, and the offset bit depth increases by 1 bit every time the pixel bit depth increases by 2 bits. With this configuration, it is possible to deal with the size of the bit rate with a higher degree of freedom than patterns S2 and S3 while taking into account the size of the pixel bit depth.

[0162] In addition, in common to patterns S1 to S4, the shift value is expressed as a difference value between PIC_DEPTH and SAO_DEPTH, PIC_DEPTH-SAO_DEPTH. The change of the offset bit depth and the shift value in the above patterns S1 to S4 sets the offset bit depth and the shift value without increasing the memory size and the amount of processing, so that it is possible to improve the coding efficiency.

[0163] (Pattern S5) In pattern S5, the bit depth of the offset is explicitly coded. Specifically, the difference between the bit depth of the offset and a predetermined value is coded. The predetermined value is suitably 8 or the pixel bit depth. In the former case, SAO_DEPTH-8 is coded, and in the latter case, PIC_DEPTH-SAO_DEPTH is suitable. The bit depth of the offset may be coded as part of various parameter information or header of the coded data, or as part of the QAOU information. As part of the QAOU information, the pixel bit depth may be included in the leaf QAOU, or may be included in the QAOU information of a predetermined layer (for example, the QAOU of the highest layer or the QAOU of the first layer). By coding as part of the coded data, the effect of maximizing coding efficiency can be obtained by setting the bit depth to an optimal value in the decoding device and the coding device. In addition, when the bit depth is coded with the QAOU information, it is possible to reduce the memory for storing the offset by changing the bit depth according to the depth sao_curr_depth of the QAOU. Since many offsets may appear when sao_curr_depth is large, the required memory size can be reduced by setting the bit depth small when sao_curr_depth is large and setting the bit depth large when sao_curr_depth is small. For example, when sao_curr_depth=0 to 1, it is appropriate to set the offset bit depth=pixel bit depth (pattern S1), and when sao_curr_depth=2 to 4, it is appropriate to set an upper limit on the offset bit depth (pattern S2), and to code the bit depth in this manner. It is also appropriate to code a flag indicating whether to code the offset bit depth for each QAOU depth or to use one offset bit depth regardless of the QAOU depth, and to switch whether to code for each QAOU depth.

[0164] (Pattern S6) In pattern S6, the bit depth is determined according to sao_curr_depth without explicit coding. For example, when sao_curr_depth=0 to 1, the offset bit depth is set to the pixel bit depth (pattern S1), and when sao_curr_depth=2 to 4, an upper limit is set to the offset bit depth (pattern S2).

[0165] (Pattern C1) In pattern C1, the offset range is set according to SAO_DEPTH. In the following, the maximum bit length that represents the value of the offset range is CLIP_BIT. Specifically, the offset range is set to -2 by calculating CLIP_BIT=SAO_DEPTH-K. CLIP_BIT-1 ~2 CLIP_BIT-1 -1. According to the inventor's experiments, it was found that K=4 is appropriate. That is, it was confirmed that when K=4, there is no decrease in encoding efficiency even if the offset range is limited by the offset value range. K=4 is appropriate when the bit depth of the most commonly used pixel is 8. When the pixel bit depth is 8, the offset bit depth SAO_DEPTH is also 8, and CLIP_BIT=8-K=4. The fact that one offset can be stored in 4 bits means that in software that handles 8-bit bytes as units, two offsets can be packed and stored in one byte, making it possible to easily reduce memory size.

[0166] (Pattern C2) In pattern C2, the offset range is set regardless of SAO_DEPTH. Specifically, CLIP_BIT=8 and the offset range is set to -2 7 ~2 7 Set it to -1.

[0167] Generally, a constant N independent of SAO_DEPTH may be used to set CLIP_BIT=N. When setting an offset value range independent of SAO_DEPTH, it is preferable to set the value to a smaller value than the offset bit depth in order to obtain the effect of reducing memory size.

[0168] (Pattern C3) In pattern C3, the offset value range is determined according to the QAOU hierarchy. When sao_curr_depth is small (e.g., 0 to 1), the offset value range is determined regardless of the offset bit depth, and when sao_curr_depth is large (e.g., 2 to 4), it is appropriate to determine the offset value range according to the offset bit depth. For example, the former can be CLIP_BIT=8 (pattern C2), and the latter can be SAO_DEPTH-K bits (pattern C1). Note that when the offset bit depth is changed according to the QAOU hierarchy, a fixed number of bits may be acceptable as a result. For example, CLIP_BIT=4 is appropriate.

[0169] (Example 1 of offset bit count) Next, a first specific example of the number of bits of the offset (sao_offset) according to this embodiment will be described. In this example, a case will be described in which the pixel bit depth is 10 bits, the shift value is set according to pattern S2, and the offset value range is set according to pattern C1. In pattern S2, the offset bit depth is set to 10 bits, and in pattern C1, the offset value range is set to 10-4=6 bits. The number of bits per offset being 6 bits corresponds to limiting the possible values ​​of the offset to values ​​from -32 to 31. In this example, the offset information storage unit 621 has a maximum memory size per picture for storing offsets. (Total number of QAOMUs per picture) x (Number of classes) x (Number of bits per offset) = 256 x 16 x 6 (bits) = 24576 (bits) It is sufficient to have the above.

[0170] Therefore, by configuring the offset to have the value range of this example, the memory size required for offset information storage unit 621 can be reduced to approximately 3 / 5 compared to the conventional example.

[0171] Since the amount of offset code included in the coded data #1 can be reduced, coding efficiency can be improved. Also, since the addition of an excessive offset is prevented, appropriate image quality is guaranteed.

[0172] (Example 2 of offset bit count) Next, a second specific example of the number of bits of the offset (sao_offset) according to this embodiment will be described. In this example, an example will be described in which the pixel bit depth is 10 bits, the shift value is set by pattern S2, and the offset value range is set by pattern C2. The offset bit depth is set to 10 bits by pattern S2, and the offset value range is set to 8 bits by pattern C2. The number of bits per offset being 8 bits corresponds to limiting the possible values ​​of the offset to values ​​from -128 to 127, for example. In this example, the offset information storage unit 621 has a maximum memory size per picture for storing offsets. (Total number of QAOMUs per picture) x (Number of classes) x (Number of bits per offset) = 256 x 16 x 8 (bits) = 32768 (bits) It is sufficient to have the above.

[0173] Therefore, by configuring the offset to have the number of bits in this example, the memory size required for offset information storage unit 621 can be reduced to approximately 4 / 5 compared to the conventional example.

[0174] In addition, since the amount of offset code included in the coded data #1 can be reduced, coding efficiency can be improved, and since the addition of an excessive offset is prevented, appropriate image quality is guaranteed.

[0175] Note that by limiting the number of offset bits, the amount of coding for the offset information included in the encoded data can be reduced. However, excessive restriction reduces the benefit of having an adaptive offset filter, and there is a problem that the amount of coding for the residual data (pixel values ​​of the residual image) included in the encoded data increases.

[0176] (Example 3 of offset bit count) Next, a third specific example of the number of bits of the offset (sao_offset) will be described. In either of the patterns C1 and C2 of this example, a different value is set for the number of bits of the offset depending on whether the offset type is an edge offset (offset type = 1 to 4) or a band offset (offset type = 5 to 6). Here, a method is described in which the shift value is set by pattern S2, the offset value range of the edge offset is set by pattern C2, and the offset value range of the band offset is set by pattern C1, taking a case where the pixel bit depth is 10 bits as an example.

[0177] When the pixel bit depth is 10 bits, the offset bit depth is set to 10 bits in pattern S2. An offset belonging to an edge offset offset type (hereinafter, edge offset offset) has a value range set to 8 bits from pattern C2. An offset belonging to an band offset offset type (hereinafter, band offset offset) has a value range set to 6 bits from pattern C1. More generally, when the number of bits of the edge offset offset is N bits and the number of bits of the band offset offset is M bits, the number of bits of the offset is determined so that N≧M is satisfied.

[0178] The memory size that needs to be secured in the offset information storage unit 621 is, in QAOU units, the number of classes of the offset type multiplied by the number of bits of the offset, so by reducing the number of bits of the band offset, whose number of classes of the offset is larger than that of the edge offset, the memory area used for the offset in the offset information storage unit 621 can be effectively utilized.

[0179] In this way, by varying the number of offset bits according to the offset type, it is possible to improve the coding efficiency without requiring an excessive memory size for the offset information storage unit 621. It is possible to make the most effective use of the memory area.

[0180] The threshold value th for limiting the possible offset values ​​is 2 m-1 Greater than 2 m When m is equal to or less than 1, an m-bit fixed-length coding / decoding method can be used as a coding method for coding the offset. Also, a variable-length coding / decoding method such as truncated unary coding, which uses the maximum value as th, or truncated rice coding can be used. Note that the maximum value th is determined by the offset value range supplied from the offset attribute setting unit 613. Also, the video decoding device 1 can decode the offset coded in this manner.

[0181] In the above configuration, the offset attribute setting unit 613 sets the bit depth of the offset, the offset range, and the shift value. The adaptive offset filter information decoding unit 61 decodes the quantized offset having a value within the range of the offset range, and stores each offset in the offset information storage unit 621 having a storage area with a bit width equal to or greater than the offset range. This embodiment is characterized in that the offset range is determined according to the bit depth of the offset. In addition, the bit depth of the offset is determined according to the bit depth of the pixel. Therefore, determining the bit depth of the offset according to the bit depth of the pixel is also a feature of this embodiment.

[0182] Also, the adaptive offset filter information decoding unit 61 in the first embodiment may include a storage unit that stores the decoded offset and an inverse quantization unit that inverse quantizes the offset obtained from the storage unit, and may be configured to omit the inverse quantization process in the offset derivation unit 625. In this case, the storage unit stores an offset limited to an offset value range set by the offset attribute setting unit 613, and the inverse quantization unit shifts to the left according to the shift value set by the offset attribute setting unit 613 to perform the inverse quantization process.

[0183] The offset information decoding unit 611 decodes each offset referenced by the offset addition unit 626, which adds an offset to each pixel value of an input image composed of multiple unit areas, from the encoded data #1, and can also be described as having an offset decoding means that sets an offset value range and a shift value determined according to a pixel bit depth not shown, and decodes an offset that is limited to the offset value range.

[0184] Furthermore, the adaptive offset filter 60 according to this embodiment can also be expressed as an image filter device that adds an offset to each pixel value of an input image composed of a plurality of unit areas, and includes an offset attribute setting unit 613 that refers to offset type designation information included in the encoded data and sets an offset attribute in the unit area to be processed, an offset information decoding unit 611 that decodes an offset having a bit width according to the offset value range included in the set offset attribute, and an offset addition unit 626 that adds the offset to each pixel value of the input image.

[0185] Furthermore, the offset information decoding unit 611 may be configured to include, in addition to the offset decoding means, a determination means for determining, from a plurality of offset types, an offset type to which the unit area to be processed belongs, and an offset decoding means for decoding an offset having a different bit width depending on the offset type determined by the determination means.

[0186] In addition, the offset type designation information may include a bit depth of the pixel values ​​of the input image in each unit area, and the offset information decoding unit 611 may be configured to decode an offset having a bit width corresponding to the bit depth of the pixel values.

[0187] The following describes a specific example of the classification process performed by the class classification unit 624. Of the following classification process examples, it is preferable that the class classification unit 624 performs a classification process corresponding to the classification process in the video encoding device that generates the encoded data #1.

[0188] (Classification Processing Example 1 by Classification Unit 624) A first example of the classification process by the classifying section 624 will be described with reference to FIGS. 10(a) to (d) and FIG.

[0189] (When offset type = 1 to 4 (edge ​​offset)) When the offset type supplied from the offset type derivation unit 623 is any one of 1 to 4, the class classification unit 624 determines whether or not an edge exists in the vicinity of the pixel to be processed, and if an edge exists, the type of edge, and classifies the pixel to be processed into one of a plurality of classes depending on the result of the determination.

[0190] More specifically, first, the classification unit 624 calculates the sign of the difference between the pixel value pic[x] of the pixel x to be processed and the pixel values ​​pic[a] and pic[b] of two pixels a and b that are adjacent to the pixel to be processed or share a vertex with the pixel x to be processed. Sign(pic[x]-pic[a]), and Sign(pic[x]-pic[b]) Here, Sign(z) is calculated as follows: Sign(z)=+1 (when z>0) Sign(z)=0 (when z=0) Sign(z)=-1 (when z<0) In addition, which pixels are used as pixel a and pixel b depends on the offset type, and is defined as follows:

[0191] When offset type = 1 (sao_type_idx = 1) As shown in FIG. 10(a), the pixel adjacent to the left side of the processing target pixel x is pixel a, and the pixel adjacent to the right side of the processing target pixel is pixel b.

[0192] When offset type = 2 (sao_type_idx = 2) As shown in FIG. 10(b), the pixel adjacent to the upper side of the processing target pixel x is designated as pixel a, and the pixel adjacent to the lower side of the processing target pixel is designated as pixel b.

[0193] When offset type = 3 (sao_type_idx = 3) As shown in FIG. 10(c), a pixel sharing the upper left vertex of the processing target pixel x is designated as pixel a, and a pixel sharing the lower right vertex of the processing target pixel is designated as pixel b.

[0194] When offset type = 4 (sao_type_idx = 4) As shown in FIG. 10(d), the pixel sharing the lower left vertex of the processing target pixel x is designated as pixel a, and the pixel sharing the upper right vertex of the processing target pixel is designated as pixel b.

[0195] Fig. 11(a) is a graph showing the magnitude relationship between the pixel value pic[x] of the pixel x to be processed and the pixel values ​​of pixels a or b, and a diagram showing the value of the function Sign according to this magnitude relationship. In the graph shown in Fig. 11(a), the black circle marked with pic[x] indicates the pixel value of the pixel x to be processed, and the black circle without pic[x] indicates the pixel value of the pixel a or b to be processed. The up and down direction in the graph shown in Fig. 11(a) indicates the magnitude relationship of the pixel values.

[0196] Next, the classifying unit 624 derives EdgeType based on Sign(pic[x]-pic[a]) and Sign(pic[x]-pic[b]) using the following formula (1-1).

[0197] EgdeType=Sign(pic[x]-pic[a])+Sign(pic[x]-pic[b])+2 ···(1-1) Fig. 11(b) is a graph showing the magnitude relationship between the pixel value of the pixel x to be processed and the pixel values ​​of pixels a and b, and a diagram showing the value of EdgeType according to the magnitude relationship. In Fig. 11(b), the black circle at the center of each graph indicates the pixel value of the pixel x to be processed, and the black circles at both ends indicate the pixel values ​​of pixels a and b. The up and down directions in the graphs shown in Fig. 11(b) indicate the magnitude relationship of the pixel values.

[0198] Next, the class classification unit 624 derives the class index (class_idx) of the class to which the processing target pixel x should belong based on the derived EdgeType as follows.

[0199] class_idx=EoTbl[EdgeType] Here, EoTbl[EdgeType] is a conversion table used to derive class_idx from EdgeType. A specific example of the conversion table EoTbl is shown in FIG.

[0200] As shown in Fig. 11(d), when there is no edge in the area consisting of the target pixel x, pixel a, and pixel b (hereinafter, also referred to as the flat case), the classification unit 624 classifies the target pixel x into class 0 (class_idx=0). Fig. 11(c) shows the correspondence between each graph shown in Fig. 11(b) and class_idx.

[0201] (When offset type = 5~6 (band offset)) When the offset type supplied from the offset type derivation unit 623 is one of 5 to 6, the class classification unit 624 classifies the pixel value of the processing target pixel x into one of a plurality of classes according to the pixel value pic[x] of the processing target pixel x.

[0202] When offset type = 5 (sao_type_idx = 5) The classifying unit 624 classifies the pixel value pic[x] of the processing target pixel x as follows: (max×1 / 4)≦pic[x]≦(max×3 / 4) satisfies the above, the pixel to be processed is classified into a class other than class 0. In other words, if the pixel value of the pixel to be processed is within the shaded area in FIG. 12(a), the pixel to be processed is classified into a class other than class 0. Note that the above max represents the maximum possible pixel value of the pixel to be processed x, and is, for example, max=255. When max=255, the above condition is 8≦(pic[x] / 8)≦23 It can also be expressed as:

[0203] When offset type = 6 (sao_type_idx = 6) The classifying unit 624 classifies the pixel value pic[x] of the processing target pixel x as follows: pic[x]≦(max×1 / 4) or (max×3 / 4)≦pic[x] satisfies the above, the pixel to be processed is classified into a class other than class 0. In other words, if the pixel value of the pixel to be processed is within the shaded area in FIG. 12(b), the pixel to be processed is classified into a class other than class 0. Note that the above max represents the maximum possible pixel value of the pixel to be processed x, and is, for example, max=255. When max=255, the above condition is (pic[x] / 8)≦7 or 24≦(pic[x] / 8) It can also be expressed as:

[0204] The classification process performed by the classification unit 624 will be described in more detail below.

[0205] When the offset type is any one of 5 and 6, the classification unit 624 derives the class index (class_idx) of the class to which the processing target pixel x should belong as follows.

[0206] class_idx=BoTbl[sao_type_idx][pic[x] / 8] Here, BoTbl[sao_type_idx][pic[x] / 8] is a conversion table used to derive class_idx from the pixel value pic[x] of the pixel x to be processed and sao_type_idx. A specific example of the conversion table BoTbl is shown in Fig. 12. In Fig. 12(c), "BO_1" indicates that sao_type_index=5, and "BO_2" indicates that sao_type_index=6.

[0207] As shown in FIG. 12(c), when sao_type_index=5, if the pixel value pic[x] of the processing target pixel x satisfies 8≦(pic[x] / 8)≦23, the class classification unit 624 classifies the processing target pixel x into one of the classes with class indexes 1 to 16 depending on the magnitude of pic[x].

[0208] On the other hand, when sao_type_index=6, when the pixel value pic[x] of the processing target pixel x satisfies pic[x] / 8)≦7 or 24≦(pic[x] / 8), the class classification unit 624 classifies the processing target pixel x into one of the classes from class index 1 to 16 depending on the magnitude of pic[x].

[0209] Generally, max=2 when the image bit depth is PIC_DEPTH. PIC_DEPTH -1, and pic / 2 instead of pic / 8 in Fig. 12(c). (PIC_DEPTH-5) The classes are classified using

[0210] (Classification process example 2 by the classifying unit 624) Next, a second example of the classification process by the classifying unit 624 will be described.

[0211] In this processing example, the classifying unit 624 derives EdgeType by using the following formula (1-2) instead of formula (1-1). The rest is the same as in the classification processing example 1.

[0212] EgdeType=Sign((pic[x]>>shift)-(pic[a]>>shift))+Sign((pic[x]>>shift)-(pic[b]>>shift))+2 ···(1-2) Here, ">>" represents a right bit shift, and "shift" represents the magnitude of the bit shift. The specific value of "shift" can be determined to have a positive correlation with the bit depth of the pixel value, for example.

[0213] In the classification process according to classification process example 1, even if the gradient of the pixel values ​​is very small, the value of Sign will be non-zero unless the gradient of the pixel values ​​is 0. Therefore, the classification process according to classification process example 1 has an aspect that it is easily affected by noise.

[0214] In this processing example, the pixel value is right-shifted before the difference is taken, so that the EdgeType value is less susceptible to the effects of noise, resulting in improved coding efficiency.

[0215] In this processing example, the following formula (1-3) may be used instead of formula (1-2).

[0216] EdgeType=Sign((pic[x]-pic[a])>>shift)+Sign((pic[x]-pic[b])>>shift))+2 (1-3) In other words, after taking the difference between pixel values, a right bit shift may be performed. By using formula (1-3), the same effect as when formula (1-2) is used can be achieved.

[0217] (Classification process example 3 by the classifying unit 624) Next, a third example of the classification process by the classifying unit 624 will be described.

[0218] In this processing example, the classifying unit 624 changes the definition of the function Sign described in the classification processing example 1 as follows. The rest is the same as in the classification processing example 1.

[0219] Sign(z)=+1 (when z>th) Sign(z)=0 (when -th≦z≦th) Sign(z)=-1 (when z<-th) Here, th is a threshold value having a predetermined value. The specific value of the threshold value th can be determined such that the absolute value thereof has a positive correlation with, for example, the bit depth of the pixel value.

[0220] In this processing example as well, the value of EdgeType is less susceptible to the effects of noise, and high coding efficiency can be obtained.

[0221] (Classification Processing Example 4 by Classification Unit 624) Next, a fourth example of the classification process by the classifying unit 624 will be described.

[0222] In this processing example, the classification unit 624 uses BoTbl[sao_type_idx][pic[x] / 8] shown in FIG. 13 instead of BoTbl[sao_type_idx][pic[x] / 8] shown in FIG.

[0223] 13, in this processing example, when the value of pic[x] / 8 is either 8 or 9, the processing target pixel x is classified into a class whose class index is not 0, even if sao_type_index=5 or sao_type_index=6. Also, when the value of pic[x] / 8 is either 22 or 23, the processing target pixel x is classified into a class whose class index is not 0, even if sao_type_index=5 or sao_type_index=6.

[0224] In this processing example, the pixel value of the pixel to be processed is clipped to MIN if its value is 15 or less (if pic[x] / 8=0 or 1). Also, the pixel value of the pixel to be processed is clipped to MAX if its value is 240 or more (if pic[x] / 8=30 or 31). Here, MIN and MAX are defined as follows: MIN=15, MAX=240 MIN=16, MAX=239 MIN=16, MAX=235 It is preferable to use any combination of the above.

[0225] In classification processing example 1, a processing target pixel that is classified into class 0 when sao_type_index=5 is classified into a class other than class 0 when sao_type_index=6. Also, a processing target pixel that is classified into class 0 when sao_type_index=6 is classified into a class other than class 0 when sao_type_index=5.

[0226] Therefore, in classification process example 1, a problem may occur in which the coding efficiency does not improve as much as expected because the pixel value after adding the offset may differ significantly depending on whether sao_type_index = 5 or sao_type_index = 6. In addition, such a problem may become prominent when the pixel value of the image before the offset filter is such that the value of pic[x] / 8 is either 8 or 9, or the value of pic[x] / 8 is either 22 or 23.

[0227] In this processing example, when the value of pic[x] / 8 is either 8 or 9, the pixel x to be processed is classified into a class whose class index is not 0 even if sao_type_index=5 or sao_type_index=6, and when the value of pic[x] / 8 is either 22 or 23, the pixel x to be processed is classified into a class whose class index is not 0 even if sao_type_index=5 or sao_type_index=6, so that the above problem is unlikely to occur. Therefore, by performing the processing of this processing example, it is possible to improve the coding efficiency.

[0228] In this processing example, when the value of pic[x] / 8 is either 8 or 9, and when the value of pic[x] / 8 is either 22 or 23, the pixel x to be processed is classified into a class whose class index is not 0 even if sao_type_index=5 or sao_type_index=6. However, this does not limit the processing example, and it is sufficient if processing is performed in which the pixel x to be processed is classified into a class whose class index is not 0 when the value of pic[x] / 8 is within a predetermined range, even if sao_type_index=5 or sao_type_index=6.

[0229] In general, when the pixel bit depth of the image is PIC_DEPTH, max=2 PIC_DEPTH -1, and pic / 2 instead of pic / 8 in Figure 13. (PIC_DEPTH-5) The classes are classified using

[0230] In this way, the class classification unit 624 that performs the classification processing of this processing example can also be expressed as classifying the target pixel into an offset class to which an offset is added when the pixel value of the target pixel is within a predetermined range, regardless of whether the offset type to which the unit area including the target pixel belongs is the first or second offset type.

[0231] (Video Encoding Device 2) A video encoding device 2 that generates encoded data #1 by encoding a target image for encoding will be described below with reference to Fig. 14 to Fig. 18(a)-(d). The video encoding device 2 includes, in part, the method adopted in H.264 / MPEG-4.AVC, the method adopted in KTA software, which is a codec jointly developed by VCEG (Video Coding Expert Group), the method adopted in TMuC (Test Model under Consideration) software, which is its successor codec, and the technology adopted in HM (HEVC Test Model) software.

[0232] Fig. 14 is a block diagram showing a configuration of a video coding device 2 according to this embodiment. As shown in Fig. 14, the video coding device 2 includes a transform / quantization unit 21, a variable-length code coding unit 22, a dequantization / inverse transform unit 23, a buffer memory 24, an intra-prediction image generating unit 25, an inter-prediction image generating unit 26, a motion vector detection unit 27, a prediction method control unit 28, a motion vector redundancy elimination unit 29, an adder 31, a subtractor 32, a deblocking filter 33, an adaptive filter 70, and an adaptive offset filter 80. The video coding device 2 is a device that generates coded data #3 by coding a video #10 (a target image to be coded).

[0233] The transform / quantization unit 21 (1) performs DCT (Discrete Cosine Transform) on a block-by-block basis on prediction residuals D obtained by subtracting a predicted image Pred from an image to be coded, (2) quantizes the DCT coefficients obtained by the DCT, and (3) supplies the quantized prediction residuals QD obtained by the quantization to the variable-length code coding unit 22 and the inverse quantization / inverse transform unit 23. The transform / quantization unit 21 (1) selects a quantization step QP to be used during quantization for each tree block, (2) supplies a quantization parameter difference Δqp indicating the magnitude of the selected quantization step QP to the variable-length code coding unit 22, and (3) supplies the selected quantization step QP to the inverse quantization / inverse transform unit 23. Here, the quantization parameter difference Δqp refers to a quantization parameter qp (QP=2 pq / 6 qp′ is the difference value obtained by subtracting the value of the quantization parameter qp′ for the tree block that was previously DCT transformed / quantized from the value of qp′.

[0234] The variable-length code encoding unit 22 generates encoded data #1 by variable-length coding (1) the quantized prediction residual QD and Δqp supplied from the transform / quantization unit 21, (2) the prediction parameters PP supplied from a prediction method control unit 28 described later, and (3) the filter set number, filter coefficient group, area designation information, and on / off information supplied from an adaptive filter 70 described later. The variable-length code encoding unit 22 also encodes the QAOU information supplied from the adaptive offset filter 80, and includes it in the encoded data #3.

[0235] The inverse quantization and inverse transform unit 23 (1) inverse quantizes the quantized prediction residual QD, (2) performs inverse DCT (Discrete Cosine Transform) transformation on the DCT coefficients obtained by the inverse quantization, and (3) supplies the prediction residual D obtained by the inverse DCT transformation to the adder 31. When inverse quantizing the quantized prediction residual QD, the quantization step QP supplied from the transform and quantization unit 21 is used. Note that the prediction residual D output from the inverse quantization and inverse transform unit 23 is the prediction residual D input to the transform and quantization unit 21 plus a quantization error, but for simplicity, a common name will be used here.

[0236] The intra-prediction image generating unit 25 generates a prediction image Pred_Intra for each partition. Specifically, (1) a prediction mode to be used for intra-prediction for each partition is selected, and (2) a prediction image Pred_Intra is generated from the decoded image P using the selected prediction mode. The intra-prediction image generating unit 25 supplies the generated intra-prediction image Pred_Intra to the prediction method control unit 28.

[0237] Furthermore, the intra-prediction image generation unit 25 specifies a prediction index PI for each partition from the prediction mode selected for each partition and the size of each partition, and supplies the prediction index PI to the prediction method control unit .

[0238] Furthermore, the intra-predicted image generating unit 25 supplies the adaptive filter 70 with intra-encoding mode information IEM, which is information indicating the size of the target partition and the prediction mode assigned to the target partition.

[0239] The motion vector detection unit 27 detects a motion vector mv for each partition. Specifically, (1) a filtered decoded image P_FL' to be used as a reference image is selected, and (2) a motion vector mv for the target partition is detected by searching for an area that best approximates the target partition in the selected filtered decoded image P_FL'. Here, the filtered decoded image P_FL' is an image obtained by performing a deblocking process by the deblocking filter 33, an adaptive offset process by the adaptive offset filter 80, and an adaptive filter process by the adaptive filter 70 on an already decoded image that has already been decoded, and the motion vector detection unit 27 can read out the pixel values ​​of each pixel constituting the filtered decoded image P_FL' from the buffer memory 24. The motion vector detection unit 27 supplies the detected motion vector mv to the inter-prediction image generation unit 26 and the motion vector redundancy removal unit 29 together with a reference image index RI that specifies the filtered decoded image P_FL' used as a reference image. In addition, for partitions performing bidirectional prediction (weighted prediction), two filtered decoded images P_FL1' and P_FL2' are selected as reference images, and motion vectors mv1 and mv2 corresponding to each of the two filtered decoded images P_FL1' and P_FL2', as well as reference image indices RI1 and RI2, are supplied to the inter-prediction image generation unit 26 and the motion vector redundancy elimination unit 29.

[0240] The inter-prediction image generating unit 26 generates a motion compensation image mc for each inter-prediction partition. Specifically, using the motion vector mv supplied from the motion vector detecting unit 27, the motion compensation image mc is generated from the filtered decoded image P_FL' specified by the reference image index RI supplied from the motion vector detecting unit 27. Like the motion vector detecting unit 27, the inter-prediction image generating unit 26 can read out the pixel values ​​of each pixel constituting the filtered decoded image P_FL' from the buffer memory 24. The inter-prediction image generating unit 26 supplies the generated motion compensation image mc (inter-prediction image Pred_Inter) to the prediction method control unit 28 together with the reference image index RI supplied from the motion vector detecting unit 27. In addition, for bidirectional prediction (weighted prediction) partitions, (1) a motion compensation image mc1 is generated from a filtered decoded image P_FL1' specified by a reference image index RI1 using a motion vector mv1, (2) a motion compensation image mc2 is generated from a filtered reference image P_FL2' specified by a reference image index RI2 using a motion vector mv2, and (3) an inter-predicted image Pred_Inter is generated by adding an offset value to the weighted average of the motion compensation images mc1 and mc2.

[0241] The prediction method control unit 28 compares the intra-predicted image Pred_Intra and the inter-predicted image Pred_Inter with the image to be coded, and selects whether to perform intra-prediction or inter-prediction. When intra-prediction is selected, the prediction method control unit 28 supplies the intra-predicted image Pred_Intra as a predicted image Pred to the adder 31 and the subtractor 32, and supplies the prediction index PI supplied from the intra-predicted image generation unit 25 as a prediction parameter PP to the variable-length code coding unit 22. On the other hand, when inter-prediction is selected, the prediction method control unit 28 supplies the inter-predicted image Pred_Inter as a predicted image Pred to the adder 31 and the subtractor 32, and supplies the reference image index RI supplied from the inter-predicted image generation unit 26, and the estimated motion vector index PMVI and motion vector residual MVD supplied from a motion vector redundancy elimination unit 29 (described later) as prediction parameters PP to the variable-length code coding unit.

[0242] A prediction residual D is generated by the subtractor 32 by subtracting the prediction image Pred selected by the prediction method control unit 28 from the image to be coded. As described above, the prediction residual D generated by the subtractor 32 is DCT transformed / quantized by the transform / quantization unit 21. Meanwhile, a locally decoded image P is generated by the adder 31 by adding the prediction image Pred selected by the prediction method control unit 28 to the prediction residual D generated by the inverse quantization / inverse transform unit 23. The locally decoded image P generated by the adder 31 passes through a deblocking filter 33, an adaptive offset filter 80, and an adaptive filter 70, and is then stored in the buffer memory 24 as a filtered decoded image P_FL and is used as a reference image in inter prediction.

[0243] The motion vector redundancy removal unit 29 removes redundancy in the motion vector mv detected by the motion vector detection unit 27. Specifically, (1) it selects an estimation method to be used for estimating the motion vector mv, (2) it derives an estimated motion vector pmv according to the selected estimation method, and (3) it generates a motion vector residual MVD by subtracting the estimated motion vector pmv from the motion vector mv. The motion vector redundancy removal unit 29 supplies the generated motion vector residual MVD to the prediction method control unit 28 together with an estimated motion vector index PMVI indicating the selected estimation method.

[0244] When a difference in pixel values ​​between adjacent pixels across a block boundary or a CU boundary in the decoded image P is smaller than a predetermined threshold, the deblocking filter 33 performs a deblocking process on the block boundary or the CU boundary in the decoded image P, thereby smoothing an image near the block boundary or the CU boundary. The image deblocked by the deblocking filter 33 is output to the adaptive offset filter 80 as a deblocked decoded image P_DB.

[0245] The adaptive offset filter 80 generates an offset-filtered decoded image P_OF by performing adaptive offset filtering on the deblocked decoded image P_DB supplied from the deblocking filter 33. The generated offset-filtered decoded image P_OF is supplied to the adaptive filter 70. A specific configuration of the adaptive offset filter 80 will be described later, and therefore will not be described here.

[0246] The adaptive filter 70 generates a filtered decoded image P_FL by performing adaptive filtering on the offset-filtered decoded image P_OF supplied from the adaptive offset filter 80. The filtered decoded image P_FL that has been filtered by the adaptive filter 70 is stored in the buffer memory 24. The filter coefficients used by the adaptive filter 70 are determined so as to reduce the error between the filtered decoded image P_FL and the encoding target image #10. The filter coefficients determined in this manner are coded as filter parameters FP and transmitted to the video decoding device 1.

[0247] (Adaptive offset filter 80) 15 is a block diagram showing the configuration of the adaptive offset filter 80. As shown in FIG. 15, the adaptive offset filter 80 includes an adaptive offset filter information setting unit 81 and an adaptive offset filter processing unit .

[0248] As shown in FIG. 15, the adaptive offset filter information setting unit 81 includes an offset calculation unit 811, an offset shift unit 816, an offset clip unit 812, an offset information selection unit 813, and an offset attribute setting unit 815.

[0249] (Offset calculation unit 811) The offset calculation unit 811 calculates offsets for all offset types and all classes for all QAOMUs up to a predetermined division depth included in the target processing unit (e.g., LCU). Here, the offset types and classes refer to the same ones as those described in the description of the video decoding device 1.

[0250] FIG. 16 is a flowchart showing the flow of processing by offset calculation section 811.

[0251] (Step S201) First, the offset calculation unit 811 starts a first loop in which the QAOMU number of the target QAOMU to be processed is used as a loop variable. For example, in the case of the example shown in Fig. 7(a) to (e), the first loop is a loop from QAOMU number = 0 to QAOMU number = 340.

[0252] (Step S202) Next, the offset calculation unit 811 starts a second loop in which the offset type selectable for the target QAOMU is set as a loop variable. The second loop is a loop from offset type 1 to offset type 6.

[0253] (Step S203) Next, the offset calculation unit 811 starts a third loop with the pixels included in the target QAOMU as units.

[0254] (Step S204) Next, the offset calculation unit 811 classifies the target pixel into one of a plurality of classes. More specifically, when the offset type, which is the second loop variable, is 1 to 4, the target pixel is classified into one of classes 1 to 4. The classification process in this step is the same as any one of classification process examples 1 to 4 by the class classification unit 624 included in the adaptive offset filter 60 in the video decoding device 1.

[0255] In addition, for the target QAOMU, the classification count count[part_idx][sao_type_index][class_idx], which is the number of times a pixel is classified, is calculated for each class, where part_idx represents the QAOMU number.

[0256] (Step S205) Next, the offset calculation unit 811 calculates a difference pixel value at the target pixel by taking the difference between the pixel value of the deblocked decoded image P_DB at the target pixel and the pixel value of the encoding target image #10 at the target pixel. More specifically, when the position of the target pixel is (x, y), P_DB(x, y)-Org(x, y) is calculated. Here, P_DB(x, y) represents the pixel value of the deblocked decoded image P_DB at the target pixel, and Org(x, y) represents the pixel value of the encoding target image #10 at the target pixel.

[0257] (Step S206) This step is the end of the third loop. When this step is completed, the pixel difference values ​​have been calculated for all pixels included in the target QAOMU.

[0258] (Step S207) Next, the offset calculation unit 811 calculates an offset by dividing the sum of the difference pixel values ​​for each pixel included in the target QAOMU by the number of classifications of the class. More specifically, the offset calculation unit 811 calculates the offsets offset[part_idx][sao_type_idx][class_idx] for the target QAOMU, target offset type, and target class using the following formula.

[0259] offset[part_idx][sao_type_idx][class_idx] = Σ(P_DB(x, y)-Org(x, y)) / count[part_idx][sao_type_idx][class_idx] Here, the symbol Σ indicates that the sum is taken for pixels classified into the object class specified by class_idx in the object QAOMU specified by part_idx and the object offset type specified by sao_type_idx.

[0260] (Step S208) This step is the end of the second loop.

[0261] (Step S209) This step is the end of the first loop.

[0262] Through the above process, the offset calculation unit 811 calculates offsets for all offset types and all classes for all QAOMUs up to a predetermined division depth included in the target LCU. For example, in the case of the example shown in FIG. 7(a) to (e), the offset calculation unit 811 calculates the following in total: ((total number of QAOMUs with partition depth 0) + ... + (total number of QAOMUs with partition depth 4)) x ((number of EO offset types) x (number of EO classes) + (number of BO offset types) x (number of BO classes)) = (1 + 4 + 16 + 64 + 256) x ((4 x 4) + (2 x 16)) = 16368 (units) Here, the number of bits of each offset is, for example, 10 bits.

[0263] The offset calculation unit 811 supplies the offset information, which is made up of the offset calculated by the above processing, the offset type, the class, and QAOU structure information indicating the division structure of the QAOU, to the offset shift unit 816.

[0264] The video encoding device 2 may be configured to encode a flag indicating which classification process example was performed in step S204, and the adaptive offset filter 60 included in the video decoding device 1 may be configured to refer to the flag and perform the same classification process as the classification process indicated by the flag. Alternatively, the video encoding device 2 and the video decoding device 1 may be configured to perform the same classification process that is predetermined without using such a flag.

[0265] (Offset shift unit 816) The offset shift unit 816 quantizes each offset included in the offset information supplied from the offset calculation unit 811. The quantization converts the offset from the pixel bit depth precision to the offset bit depth precision by bit-shifting the offset to the right. Note that the shift amount in the shift process is determined by a shift value supplied by an offset attribute setting unit 815, which will be described later.

[0266] (Offset clip part 812) The offset clip unit 812 performs clip processing on the offset supplied from the offset shift unit 816 using either clip processing 1 or clip processing 2 as shown below, in order to limit the offset value range supplied from the offset attribute setting unit 815 described later.

[0267] (Clip Processing 1) The offset clip unit 812 performs clipping processing on each offset included in the offset information supplied from the offset shift unit 816. The offset clip unit 812 clips each offset supplied from the offset shift unit 816 to a value from −8 to 7, for example, to express each offset in 4 bits. Each clipped offset is supplied to an offset information selection unit 813. As with the video decoding device 1, the bit width to be clipped is set according to the bit depth of the image and the bit depth of the offset.

[0268] In this way, by clipping each offset, it is possible to reduce the memory size of a memory (not shown) in which each offset is stored. Also, since the amount of code of the offset included in the coded data #1 can be reduced, it is possible to improve coding efficiency. Also, since the addition of an excessive offset is suppressed, appropriate image quality is guaranteed.

[0269] (Clip Processing 2) Furthermore, the offset clipping unit 812 may be configured to set the clipping range of each offset supplied from the offset shifting unit 816 to a different value depending on the offset type.

[0270] For example, if the offset type is edge offset, the number of offset bits is 8 bits, and if the offset type is band offset, the number of offset bits is 4 bits. More generally, when the number of offset bits is N bits when the offset type is edge offset, and the number of offset bits is M bits when the offset type is band offset, the number of offset bits is determined so that N>M is satisfied.

[0271] By thus varying the number of bits of the offset depending on the offset type, it is possible to improve the coding efficiency without requiring an excessive memory size for the memory for storing each offset.

[0272] The threshold value th for limiting the possible offset values ​​is 2 m-1 Greater than 2 m When m is less than 1, m-bit fixed length coding can be used as a coding method for coding the offset. More specifically, truncated unary coding or truncated rice coding with the maximum value th can be used. Note that the maximum value th is determined by the offset value range supplied from the offset attribute setting unit 815.

[0273] This embodiment also includes clipping processing obtained by combining the above clipping processing 1 and 2. The adaptive offset filter 80 may be configured without including the offset clip unit 812.

[0274] (Offset information selection unit 813) The offset information selection unit 813 determines a combination of offset type, class, and offset that reduces the RD cost (Rate-Distortion cost) and a corresponding QAOU partition structure, and supplies QAOU information indicating the determined offset type, class, offset, and the corresponding QAOU partition structure to the variable-length code encoding unit 22. In addition, the offset information selection unit 813 supplies the determined offset to the adaptive offset filter processing unit 82 for each QAOU or each QAOMU.

[0275] The process of offset information selection unit 813 will be described in more detail below with reference to Figures 17 and 18. Figure 17 is a flowchart showing the flow of the process by offset information selection unit 813.

[0276] (Step S301) First, the offset information selection unit 813 starts a first loop in which the QAOMU number of the target QAOMU to be processed is used as a loop variable.

[0277] (Step S302) Next, the offset information selection unit 813 starts a second loop in which the offset types selectable for the target QAOMU are used as loop variables. The second loop is a loop from offset type 1 to offset type 6.

[0278] (Step S303) Next, the offset information selection unit 813 calculates the pairwise square error between the offset-filtered decoded image P_OF in the target QAOMU and the encoding target image #10 for the target offset type.

[0279] (Step S304) This step is the end of the second loop.

[0280] (Step S305) This step is the end of the first loop. When the first and second loops are completed, the squared errors for all offset types are calculated for each QAOMU.

[0281] (Step S306) Next, the offset information selection unit 813 determines a QAOU partitioning structure that reduces the RD cost from among the QAOU partitioning structures that divide the target processing unit (for example, LCU) into QAOUs.

[0282] A specific example of the processing performed by the offset information selection unit 813 in this step will be described below with reference to FIGS. 18(a) to (e).

[0283] First, the offset information selection unit 813 calculates the RD cost when the partition depth is set to 0 and the RD cost when the partition depth is set to 1 (FIGS. 18(a) and 18(b)). In FIG. 18(a), it is assumed that the RD cost when the partition depth is 1 is smaller than the RD cost when the partition depth is 0 (FIG. 18(c)).

[0284] Next, the offset information selection unit 813 calculates the RD cost when the division depth is set to 2 (FIG. 18(d)).

[0285] Next, the offset information selection unit 813 compares the RD cost of the QAOMU of partition depth 1 with the RD cost of the QAOMU of partition depth 2 included in the QAOMU of partition depth 1, and if the RD cost of the QAOMU of partition depth 2 is smaller, it updates the QAOMU of partition depth 1 to the QAOMU of partition depth 2 (Figure 18 (e)). This process is repeated until the maximum partition depth is reached. This determines a QAOU partition structure with a smaller RD cost.

[0286] (Offset attribute setting unit 815) The offset attribute setting unit 815 determines the offset bit depth using the pixel bit depth (not shown) as input. The offset value range and shift value are set based on the determined offset bit depth. The offset value range is supplied to the adaptive offset filter processing unit 82, and the shift value is supplied to the offset shift unit 816. The setting of the offset value range and shift value is the same process as that of the offset attribute setting unit 613 described above, and therefore will not be described here.

[0287] (Adaptive offset filter processing unit 82) The adaptive offset filter processing unit 82 adds an offset supplied from the offset information selection unit 813 to each pixel of the deblocked decoded image P_DB in the target QAOU. The adaptive offset filter processing unit 82 outputs an image obtained by performing processing on all QAOUs included in the deblocked decoded image P_DB as an offset-filtered decoded image P_OF. Note that the configuration of the adaptive offset filter processing unit 82 is the same as that of the adaptive offset filter processing unit 62, and therefore a description thereof will be omitted here. Each offset stored in an offset information storage unit included in the adaptive offset filter processing unit 82 (not shown) is limited to an offset value range set by an offset attribute setting unit 815.

[0288] [Embodiment 2] In embodiment 1, sao_offset[sao_curr_depth][ys][xs][i] included in encoded data #1 is syntax representing the specific value of the offset added to each pixel included in the target QAOU in offset filter processing using an adaptive offset filter.

[0289] On the other hand, the inventor has found that the amount of coding data can be further reduced by predictively coding the offset value used in the offset filter process, i.e., by coding the offset residual calculated using the offset value and a predicted value of the offset value.

[0290] In this embodiment, a video decoding device that performs offset filtering using a predictively coded offset, and a video coding device that predictively codes the offset used in the offset filtering will be described with reference to Fig. 19 to Fig. 21. Note that the description of the parts that have already been described in the first embodiment will be omitted.

[0291] (encoded data) The coded data according to this embodiment includes an offset residual sao_offset_residual[sao_curr_depth][ys][xs][i] instead of sao_offset[sao_curr_depth][ys][xs][i] included in the coded data #1 according to the embodiment 1. The other configurations of the coded data according to this embodiment are the same as those of the coded data #1 according to the embodiment 1. Hereinafter, the coded data according to this embodiment may be referred to as coded data #3.

[0292] (sao_offset_residual) The offset residual sao_offset_residual[sao_curr_depth][ys][xs][i] is a weighted difference value between the offset value added to each pixel included in the target QAOU in offset filter processing by the adaptive offset filter of this embodiment and the predicted value of the offset value, and is also expressed as sao_offset_residual[sao_type_idx][class_idx].

[0293] When the offset added to the target pixel included in the target QAOU is expressed as Offset[sao_type_idx][class_idx], the offset residual sao_offset_residual[sao_type_idx][class_idx] is sao_offset_residual[sao_type_idx][class_idx] = Offset[sao_type_idx][class_idx] - a*pred_offset[merge_tbl[sao_type_idx]][class_idx] Here, a is a weighting coefficient by which the predicted value pred_offset is multiplied, and merge_tbl is a function that takes sao_type_idx as an argument. Specific examples of a and merge_tbl will be described later, and therefore will not be described here.

[0294] (Video Decoding Device) The video decoding device according to this embodiment includes an adaptive offset filter 60' instead of the adaptive offset filter 60 included in the video decoding device 1 according to Embodiment 1. Other configurations of the video decoding device according to this embodiment are similar to those of the video decoding device 1 according to Embodiment 1.

[0295] 19 is a block diagram showing a configuration of an adaptive offset filter 60' according to this embodiment. As shown in FIG. 19, the adaptive offset filter 60' includes an offset information decoding unit 611' instead of the offset information decoding unit 611 included in the adaptive offset filter 60.

[0296] (Offset information decoding unit 611') The offset information decoding unit 611' refers to the QAOU information included in the coded data #3 and decodes the offset information OI included in the QAOU information. In addition, the offset information decoding unit 611' uses the offset residual sao_offset_residual[sao_type_idx][class_idx] obtained by decoding the offset information OI and the predicted value pred_offset[merge_tbl[sao_type_idx]][class_idx] to obtain an offset Offset[sao_type_idx][class_idx] used in the adaptive offset filter process. Offset[sao_type_idx][class_idx] =a*pred_offset[merge_tbl[sao_type_idx]][class_idx] + sao_offset_residual[sao_type_idx][class_idx] The offset [sao_type_idx][class_idx] calculated by the above is stored in the offset information storage unit 621. Here, pred_offset [merge_tbl [sao_type_idx] [class_idx] is a predicted value of Offset [sao_type_idx] [class_idx]. merge_tbl [sao_type_idx] is a table to which an index is given for sao_type_idx = 1 to 6, and one or more sao_type_idx can be regarded as the same group.

[0297] (Example 1 of pred_offset) A first specific example of pred_offset[merge_tbl[sao_type_idx]][class_idx] will be described. In this example, the predicted value pred_offset[merge_tbl[sao_type_idx]][class_idx] is expressed as follows: pred_offset[merge_tbl[sao_type_idx]][class_idx] = Offset'[sao_type_idx][class_idx] Here, Offset'[sao_type_idx][class_idx] is a decoded offset, and represents an offset associated with the offset type index sao_type_idx and the class index class_idx.

[0298] In this way, in this example, the offset Offset'[sao_type_idx][class_idx], which is a decoded offset and is associated with the offset type index sao_type_idx and the class index class_idx, is used as the predicted value of Offset[sao_type_idx][class_idx].

[0299] (Example 2 of pred_offset) A second specific example of pred_offset[merge_tbl[sao_type_idx]][class_idx] will be described. In this example, the predicted value pred_offset[merge_tbl[sao_type_idx]][class_idx] is expressed as follows: pred_offset[merge_tbl[sao_type_idx]][class_idx] = (pred_offset'[merge_tbl[sao_type_idx]][class_idx]*W1 + Offset'[sao_type_idx][class_idx]*W2) >> log2(W1+W2) Here, pred_offset'[merge_tbl[sao_type_idx]][class_idx] represents a predicted value used to calculate the decoded offset Offset'[sao_type_idx][class_idx]. In addition, "*" represents an operation symbol for multiplication, and ">>" represents a right bit shift. In addition, W1 and W2 represent weighting coefficients, and for example, W1=3 and W2=1 can be taken. The specific values ​​of W1 and W2 may be determined so as to increase the coding efficiency.

[0300] As is clear from the above formula, in this example, pred_offset' and Offset' are referenced to obtain pred_offset, pred_offset'' and Offset'' are referenced to obtain pred_offset', and so on, the decoded predicted value and offset are referenced recursively, so that a plurality of decoded offsets contribute to pred_offset. Therefore, excessive fluctuation of the predicted value is suppressed. As a result, even in a case where an inappropriate predicted value is calculated due to the influence of noise, for example, the influence of such an inappropriate predicted value can be suppressed, and therefore the coding efficiency can be improved.

[0301] (Example 3 of pred_offset) A third specific example of pred_offset[merge_tbl[sao_type_idx]][class_idx] will be described. In this example, the predicted value pred_offset[merge_tbl[sao_type_idx]][class_idx] is expressed as follows: pred_offset[merge_tbl[sao_type_idx]][class_idx] = clip3(-th, th, pred_offset[merge_tbl[sao_type_idx]][class_idx]) Here, clip3(A, B, C) indicates that the value C is clipped with the lower limit A and the upper limit B. The argument pred_offset[merge_tbl[sao_type_idx]] of clip3 is defined as in the above specific example 1 or 2. The threshold th is defined as follows depending on the bit depth bit_depth of the pixel value, for example.

[0302] th = 4 (bit_depth = 8) th = 8 (bit_depth > 8) In this way, in this example, by using the predicted value clipped at the upper and lower limits, too large or too small predicted values ​​are not generated, so that the coding efficiency can be improved. Also, the absolute values ​​of the upper and lower limits are set to be large when the number of bits of the pixel value is large, so that appropriate clipping processing can be performed according to the bit depth of the pixel value, and deterioration of image quality can be prevented.

[0303] (Example 1 of merge_tbl) Fig. 20(a) is a table showing a first specific example of the function merge_tbl[sao_type_idx]. As shown in Fig. 20(a), the merge_tbl[sao_type_idx] according to this example has no value when sao_type_idx=0, and has values ​​of 0 to 5 when sao_type_idx=1 to 6. Therefore, the merge_tbl[sao_type_idx] according to this example is as follows: merge_tbl[sao_type_idx] = sao_type_idx - 1 It can also be expressed as:

[0304] By using merge_tbl[sao_type_idx] according to this example, the offset information decoding unit 611′ determines the predicted value pred_offset individually for each sao_type_idx and each class_idx, so that the amount of code of the offset residual sao_offset_residual can be reduced.

[0305] (Specific example 2 of merge_tbl) Fig. 20(b) is a table showing a second specific example of the function merge_tbl[sao_type_idx]. As shown in Fig. 20(b), merge_tbl[sao_type_idx] according to this example takes a value of 0 for edge offset (sao_type_idx=1 to 4) and takes a value of 1 to 2 for band offset (sao_type_idx=5 to 6).

[0306] For example, if sao_type_idx=1 (merge_tbl[sao_type_idx=1]=0) and class_idx=1 are specified when calculating the previous offset, and sao_type_idx=2 (merge_tbl[sao_type_idx=2]=0) and class_idx=1 are specified when calculating the next offset, the predicted value used when calculating the next offset will be the same as the predicted value used when calculating the previous offset.

[0307] By using merge_tbl[sao_type_idx] according to this example, the offset information decoding unit 611′ performs the following processing for each of the cases where an edge offset is specified and where a band offset is specified.

[0308] When an edge offset is specified A predicted value of an offset to be decoded is calculated from a decoded offset of the same class as the class associated with the offset to be decoded. Here, as long as the offset to be decoded and the predicted value are of the same class, the offset types may be different. Therefore, a predicted value set for calculating an offset of a certain offset type can be used for calculating an offset of a different offset type from the certain offset, so that the process for setting the predicted value is reduced.

[0309] If a band offset is specified A predicted value of the offset to be decoded is calculated from an offset of the same offset type and class as the offset type and class associated with the offset to be decoded.

[0310] By using merge_tbl[sao_type_idx] according to this example, it is possible to calculate an appropriate predicted value while reducing the amount of processing.

[0311] (Example of coefficient a) The weighting coefficient a by which the predicted value pred_offset is multiplied may be 1 regardless of the offset type, or may be a different value depending on the offset type.

[0312] for example, a=1 (for edge offset) a=0.5 (for band offset) More generally, if the coefficient a when an edge offset is specified is expressed as a(edge), and the coefficient a when a band offset is specified is expressed as a(band), then a(edge)>a(band) It is sufficient to use one that satisfies the above.

[0313] The inventor found that the correlation between the decoded offset and the offset to be decoded when the edge offset is specified is greater than the correlation between the decoded offset and the offset to be decoded when the band offset is specified. In the above specific example, the influence of the correlation between the decoded offset and the offset to be decoded can be appropriately incorporated, so that the amount of code of the offset residual is reduced.

[0314] (Video Encoding Device) The video coding device according to this embodiment includes an adaptive offset filter 80' instead of the adaptive offset filter 80 included in the video coding device 2 according to embodiment 1. Other configurations of the video coding device according to this embodiment are similar to those of the video coding device 2 according to embodiment 1.

[0315] Fig. 21 is a block diagram showing a configuration of an adaptive offset filter 80' according to this embodiment. As shown in Fig. 21, the adaptive offset filter 80' includes an offset residual derivation unit 814 in addition to the components included in the adaptive offset filter 80.

[0316] (Offset residual derivation unit 814) The offset residual derivation unit 814 calculates an offset residual by taking the difference between the offset supplied from the offset information selection unit 813 and a predicted value of the offset. The offset residual is coded by the variable-length code coding unit 22 as a part of the QAOU information.

[0317] The predicted value set by the offset residual derivation unit 814 is similar to the predicted value set by the offset information decoding unit 611' included in the video decoding device according to this embodiment, and therefore a description thereof will be omitted here.

[0318] (Additional Note 1) The present invention can also be described as follows.

[0319] The image filter device of the present invention is an image filter device that adds an offset to each pixel value of an input image composed of a plurality of unit areas, and is characterized in that it comprises an offset attribute setting means that refers to offset type designation information included in encoded data and sets an offset attribute in the unit area to be processed, an offset decoding means that decodes an offset having a bit width corresponding to the offset value range included in the set offset attribute, and a filter means that adds the offset to each pixel value of the input image.

[0320] According to the image filter device configured as described above, the offset attribute setting means refers to offset type designation information included in the encoded data and sets an offset attribute for the unit area to be processed, and the offset decoding means decodes an offset having a bit width according to the offset value range included in the set offset attribute, thereby effectively reducing the memory size for storing the offset.

[0321] Therefore, according to the above configuration, it is possible to perform appropriate offset filtering while reducing the memory size for storing offsets.

[0322] The offset type designation information may be determined for each of the input images, or for each of the unit areas, or for each of a predetermined set of the input images, or for each of a predetermined set of the unit areas.

[0323] It is also preferable that the offset type designation information includes a bit depth of pixel values ​​of the input image in each unit area, and the offset decoding means decodes an offset having a bit width corresponding to the bit depth of the pixel values.

[0324] According to the above configuration, the offset decoding means decodes an offset having a bit width corresponding to the bit depth of the pixel value, thereby effectively reducing the memory size for storing the offset.

[0325] In addition, the offset type designation information may include a bit depth of pixel values ​​of the input image in each unit area, and the offset decoding means may be configured to decode an offset having a bit width capable of expressing a range of values ​​corresponding to the bit depth.

[0326] According to the above configuration, the offset decoding means decodes an offset having a bit width capable of expressing a range of values ​​corresponding to the bit depth of the pixel values ​​of the input image in each unit area, thereby effectively reducing the memory size for storing the offset.

[0327] The bit width that can express the range of values ​​according to the bit depth above refers to the bit width when the values ​​included in the range of the range are expressed in binary. For example, if the range is -2 3 ~2 3 When it is -1, the expressible bit width is 4 bits.

[0328] It is also preferable that the decoded offset is a quantized value, and that the filter means add a value obtained by dequantizing the offset using a parameter included in the offset attribute to each of the pixel values.

[0329] According to the above configuration, the decoded offset is a quantized value, and the filter means adds a value obtained by dequantizing the offset using a parameter included in the offset attribute to each pixel value, so that an offset according to the parameter included in the offset attribute is added to each image value.

[0330] Therefore, according to the above configuration, it is possible to improve the coding efficiency while reducing the memory size for storing the offsets.

[0331] The offset type designation information may include a shift value of a pixel value, and the filter means may add a value obtained by dequantizing the offset using the shift value in place of the offset.

[0332] According to the above configuration, the offset type designation information includes a shift value of a pixel value, and the filter means adds a value obtained by dequantizing the offset using the shift value in place of the offset, so that an offset corresponding to the shift value of a pixel value can be obtained. Therefore, it is possible to improve coding efficiency while reducing the memory size for storing the offset.

[0333] Note that the shift value of a pixel value refers to the difference between the pixel bit depth and the bit depth of the offset, and inverse quantizing the offset using the shift value refers to bit-shifting the offset to the left by the shift value to convert from the offset bit depth to the pixel bit depth.

[0334] It is also preferable that the offset type designation information is determined for the input image.

[0335] According to the above configuration, since the offset type designation information is determined for the input image, the image filter device can perform appropriate offset processing for the input image.

[0336] Furthermore, an offset decoding device according to the present invention is an offset decoding device that decodes each offset referenced by an image filter that adds an offset to each pixel value of an input image, and is characterized in that it comprises: offset residual decoding means that decodes each offset residual from encoded data; predicted value derivation means that derives a predicted value of each offset from an already decoded offset; and offset calculation means that calculates each offset from the predicted value derived by the predicted value derivation means and the offset residual decoded by the offset residual decoding means.

[0337] The offset decoding device configured as described above includes offset residual decoding means which decodes each offset residual from encoded data, predicted value derivation means which derives a predicted value of each offset from an already decoded offset, and offset calculation means which calculates each offset from the predicted value derived by the predicted value derivation means and the offset residual decoded by the offset residual decoding means. Therefore, it is possible to appropriately decode offsets from encoded data with a smaller amount of code compared to the case where each offset is encoded as is.

[0338] It is also preferable that the input image is composed of a plurality of unit areas, the offset residual decoding means decodes each offset residual in association with an offset type determined for each unit area and an offset class determined for each pixel, and the predicted value derivation means derives a predicted value of each offset from a decoded offset associated with the same offset type and offset class as the offset.

[0339] According to the above configuration, since a predicted value of each offset is derived from a decoded offset associated with the same offset type and offset class as the offset, prediction accuracy is improved. Therefore, according to the above configuration, offsets can be appropriately decoded from encoded data with a small amount of code.

[0340] It is also preferable that the input image is composed of a plurality of unit areas, the offset residual decoding means decodes each offset residual in association with an offset type determined for each unit area and an offset class determined for each pixel, and the predicted value derivation means derives the predicted value of each offset from a decoded offset associated with a first offset type group and the same offset class as the offset when the offset type associated with the offset belongs to a first offset type group, and from a decoded offset associated with the same offset type and offset class as the offset when the offset type associated with the offset belongs to a second offset type group.

[0341] According to the above configuration, when the offset type associated with the offset belongs to a first offset type group, the predicted value of each offset is derived from a decoded offset associated with the same offset type as the offset, and when the offset type associated with the offset belongs to a second offset type group, the predicted value is derived from a decoded offset associated with the same offset type and offset class as the offset, so that it is possible to improve prediction accuracy while reducing the amount of processing. Therefore, according to the above configuration, it is possible to appropriately decode offsets from encoded data with a small amount of code while reducing the amount of processing.

[0342] The above-mentioned first offset type refers to, for example, one in which each pixel in a unit area associated with the first offset type is classified into one of a plurality of classes, for example, depending on the state of an edge in the vicinity of the pixel, and the above-mentioned second offset type refers to one in which each pixel in a unit area associated with the second offset type is classified into one of a plurality of classes, for example, depending on the pixel value of the pixel.

[0343] It is also preferable that the offset calculation means calculates each offset as a linear function of the predicted value derived by the predicted value derivation means and the offset residual decoded by the offset residual decoding means, and that the coefficient by which the predicted value is multiplied is different between a case where the offset type associated with the offset belongs to a first offset type group and a case where the offset type associated with the offset belongs to a second offset type group.

[0344] According to the above configuration, the coefficient by which the predicted value is multiplied differs between when the offset type associated with the offset belongs to a first offset type group and when the offset type associated with the offset belongs to a second offset type group, so that the offset can be calculated using a more appropriate coefficient depending on the offset type, thereby improving coding efficiency.

[0345] It is also preferable that the predicted value derivation means derives the predicted value of each offset by taking a weighted average of a decoded offset and the predicted value of the decoded offset.

[0346] According to the above configuration, the predicted value of each offset is derived by taking a weighted average of the decoded offset and the predicted value of the decoded offset, so that the predicted value of each offset is contributed by a plurality of decoded offsets. Therefore, excessive fluctuation of the predicted value is suppressed. As a result, even if an inappropriate predicted value is calculated due to the influence of noise, the influence of the inappropriate predicted value can be suppressed, so that the coding efficiency can be improved.

[0347] It is also preferable that the predicted value derivation means includes clipping means for clipping each derived predicted value at an upper limit value and a lower limit value according to the bit depth of each pixel value in the input image.

[0348] According to the above configuration, each derived predicted value is clipped at an upper limit and a lower limit according to the bit depth of each pixel value in the input image, so that too large or too small predicted values ​​are not generated, thereby improving the coding efficiency.

[0349] Furthermore, an image filter device according to the present invention is an image filter device that acts on an input image, and is characterized in that it comprises: calculation means for calculating a difference value between a pixel value of a target pixel in the input image and a pixel value of a pixel surrounding the target pixel; bit shift means for bit-shifting the pixel value referred to by the calculation means or the difference value calculated by the calculation means by a predetermined shift value to the right; classification means for classifying the target pixel into one of a plurality of offset classes depending on the magnitude relationship between the difference value bit-shifted to the right by the bit shift means and 0; and offset means for adding an offset associated with the offset class classified by the classification means to the pixel value of the target pixel.

[0350] According to the image filter device configured as described above, the target pixel is classified into one of a plurality of offset classes according to the magnitude relationship between the right bit-shifted difference value and 0, and an offset associated with the offset class classified by the classification means is added to the pixel value of the target pixel, so that the class classification process is less susceptible to the effects of noise and coding efficiency is improved.

[0351] It is also preferable that the predetermined shift value has a positive correlation with the bit depth of the pixel value of the target pixel.

[0352] According to the above configuration, since the predetermined shift value has a positive correlation with the bit depth of the pixel value of the target pixel, it is possible to more effectively improve the coding efficiency.

[0353] Furthermore, an image filter device according to the present invention is an image filter device that acts on an input image, and is characterized by comprising: a calculation means for calculating a difference value between a pixel value of a target pixel in the input image and pixel values ​​of pixels surrounding the target pixel; a classification means for classifying the target pixel into one of a plurality of offset classes depending on the magnitude relationship between the difference value calculated by the calculation means and predetermined first and second threshold values; and an offset means for adding an offset associated with the offset class classified by the classification means to the pixel value of the target pixel.

[0354] The image filter device configured as described above classifies the target pixel into one of a plurality of offset classes depending on the magnitude relationship between the difference value calculated by the calculation means and predetermined first and second threshold values, and adds an offset associated with the offset class classified by the classification means to the pixel value of the target pixel, so that the class classification process is less susceptible to the effects of noise and encoding efficiency is improved.

[0355] It is also preferable that the absolute values ​​of the first and second thresholds have a positive correlation with the bit depth of the pixel value of the target pixel.

[0356] According to the above configuration, the absolute values ​​of the first and second thresholds have a positive correlation with the bit depth of the pixel value of the target pixel, so that it is possible to more effectively improve the coding efficiency.

[0357] In addition, an image filter device according to the present invention is an image filter device that acts on an input image composed of a plurality of unit areas, and includes a determination means for determining, from first and second offset types, an offset type to which a target unit area including a target pixel in the input image belongs; a classification means for classifying the target pixel into either an offset class that does not add an offset or a plurality of offset classes to which an offset is added, depending on the offset type to which the target unit area belongs and the pixel value of the target pixel; and an offset means for adding, to the pixel value of the target pixel, an offset associated with the offset type to which the target unit area belongs and the offset class classified by the classification means, wherein, when the pixel value of the target pixel is within a predetermined range, the classification means classifies the target pixel into an offset class to which an offset is added, regardless of whether the offset type to which the unit area including the target pixel belongs is the first or second offset type.

[0358] According to the image filter device configured as above, when the pixel value of the target pixel is within a predetermined range, the target pixel is classified into an offset class to which an offset is added regardless of whether the offset type to which the unit area including the target pixel belongs is the first or second offset type, so that block noise can be effectively removed. Therefore, according to the configuration, it is possible to improve the coding efficiency.

[0359] Furthermore, an image filter device according to the present invention is an image filter device that adds an offset to each pixel value of an input image composed of a plurality of unit areas, and is characterized in comprising a determination means that determines an offset type to which a unit area to be processed belongs among a plurality of offset types, an offset encoding means that determines an offset having a different bit width according to the offset type and encodes the offset, and a filter means that adds the determined offset to each pixel value of the input image.

[0360] According to the image filter device configured as above, an offset type to which a unit region to be processed belongs is determined from among a plurality of offset types, offsets having different bit widths are determined according to the determined offset type, the determined offsets are added to each pixel of the input image, and the determined offsets are encoded.

[0361] Therefore, according to the above configuration, it is possible to perform appropriate offset filtering while reducing the memory size for storing offsets. Also, according to the above configuration, the amount of coding data is reduced, thereby improving coding efficiency.

[0362] Furthermore, an offset coding device according to the present invention is an offset coding device that codes each offset referenced by an image filter that adds an offset to each pixel value of an input image, and is characterized in comprising: predicted value derivation means that derives a predicted value of each offset from an already-coded offset; offset residual calculation means that calculates an offset residual from each offset and the predicted value derived by the predicted value derivation means; and offset residual coding means that codes the offset residual calculated by the offset residual calculation means.

[0363] The offset coding device configured as described above is provided with predicted value derivation means for deriving a predicted value of each offset from an already-encoded offset, offset residual calculation means for calculating an offset residual from each offset and the predicted value derived by the predicted value derivation means, and offset residual coding means for coding the offset residual calculated by the offset residual calculation means, so that the amount of code of encoded data can be reduced.

[0364] Furthermore, the data structure of the encoded data according to the present invention is a data structure of encoded data referenced by an image filter that adds an offset to each pixel value of an input image composed of a plurality of unit areas, and includes offset type designation information that designates the offset type to which each unit area belongs, and an offset having a different bit width depending on the offset type, and the image filter refers to the offset type designation information included in the encoded data, determines the offset type to which the unit area to be processed belongs, and decodes the offset having a different bit width depending on the determined offset type.

[0365] The coded data configured as above includes offsets having different bit widths depending on the offset type, so that the amount of code of the coded data is reduced. Also, the image filter that decodes the coded data refers to the offset type designation information, determines the offset type to which the unit area to be processed belongs, and decodes the offsets having different bit widths depending on the determined offset type, so that it is possible to perform appropriate offset filter processing while reducing the memory size for storing the offsets.

[0366] The offset type designation information may be determined for each of the input images, or for each of the unit regions, or for each of a predetermined set of the input images, or for each of a predetermined set of the unit regions.

[0367] Third Embodiment First, the offset information OI in this embodiment will be described with reference to Fig. 23. Fig. 23(a) is a diagram showing the syntax of the offset information OI (denoted as "sao_offset_param()" in Fig. 23(a)).

[0368] 23(a), the offset information OI includes a parameter "sao_type_idx[sao_curr_depth][ys][xs]". In addition, if the parameter "sao_type_idx[sao_curr_depth][ys][xs]" is not "0", the offset information OI includes a parameter "sao_offset[sao_curr_depth][ys][xs][i]".

[0369] (sao_curr_depth, ys, xs) "sao_curr_depth", which is an argument to "sao_type_idx" and "sao_offset", is a parameter that represents the division depth of the QAOU, and "ys" and "xs" are parameters that respectively represent the y-direction position and x-direction position of the QAOU (or the QAOMU described below).

[0370] The manner in which the QAOU is divided according to the value of "sao_curr_depth" is as described above with reference to FIG.

[0371] FIG. 23(b) is a diagram showing the syntax of QAOU information (denoted as "sao_split_param()" in FIG. 23(b)). As shown in the syntax in FIG. 23(b), if the division depth "sao_curr_depth" is smaller than the maximum value set by a predetermined "saoMaxDepth", the parameter "sao_split_flag" selects whether the QAOU is further divided. If it is divided, the "sao_split_param()" of the next hierarchical depth is recursively called. If the division depth reaches the maximum value ("sao_curr_depth" is not smaller than "saoMaxDepth"), "0" is set to "sao_split_flag[sao_curr_depth][ys][xs]" and no further division is performed.

[0372] FIG. 44 shows another example of the syntax of the offset information and the QAOU information.

[0373] Figure 44(a) shows the syntax of offset information. It has the same structure as Figure 23(a), but "component", which is a value indicating a color component, is added to the argument of "sao_offset_param()" and the subscripts of the arrays of "sao_split_flag", "sao_type_idx", and "sao_offset". This makes it possible to perform different QAOU divisions for each color component, such as luminance and chrominance, and to apply different offsets.

[0374] Fig. 44(b) shows the syntax of the QAOU information. As with Fig. 44(a), this is the syntax in Fig. 23(b) with the color component "component" added as an argument.

[0375] FIG. 44(c) shows the syntax of the entire adaptive offset filter that calls the syntax of FIG. 44(a) and FIG. 44(b). The parameter “sao_flag” is a flag that selects whether or not to apply the adaptive offset filter, and the parameters related to the subsequent adaptive offset filter are used only when the flag is true. When the flag is true, the syntax “sao_split_param()” and “sao_offset_param()” are called for each color component by specifying the top layer. Since it is the top layer, the arguments given to each syntax are sao_curr_depth=0, ys=0, and xs=0. In addition, the value of “component” is 0 for luminance (Y), 1 for chrominance (Cb), and 2 for chrominance (Cr) to distinguish each color component. Note that the value of component may be other values ​​as long as the color components to be processed can be distinguished.

[0376] For chrominance (Cb) and chrominance (Cr), the corresponding flags “sao_flag_cb” and “sao_flag_cr” are used to select whether or not to apply an adaptive offset filter. If an adaptive offset filter is not applied, the QAOU information and offset information corresponding to that color component are not stored.

[0377] In the syntax of Fig. 44 above, the argument "component" is added, so in the explanation using Fig. 23 above, the arguments [sao_curr_depth][ys][xs] are replaced with [sao_curr_depth][ys][xs][component] for processing. The same applies to the following explanations.

[0378] (Video Decoding Device 1') Next, a video decoding device 1' according to this embodiment will be described with reference to Fig. 22 and Fig. 24 to Fig. 29. Note that components having the same functions as those in the above-mentioned embodiment will be denoted by the same reference numerals, and the description thereof will be omitted.

[0379] Like the video decoding device 1, the video decoding device 1' includes, as a part thereof, the method adopted in H.264 / MPEG-4.AVC, the method adopted in KTA software which is a codec jointly developed by VCEG (Video Coding Expert Group), the method adopted in TMuC (Test Model under Consideration) software which is its successor codec, and the technology adopted in HM (HEVC TestModel) software. Also, the video decoding device 1' differs in that an adaptive offset filter 60' is provided instead of the adaptive offset filter 60 in the video decoding device 1, but the other configurations are similar.

[0380] (Adaptive offset filter 60') Next, details of the adaptive offset filter 60' will be described with reference to Fig. 22. Fig. 22 is a block diagram showing the configuration of the adaptive offset filter 60'. As shown in Fig. 22, the adaptive offset filter 60' includes an adaptive offset filter information decoding unit 61' and an adaptive offset filter processing unit 62'.

[0381] As shown in FIG. 22, the adaptive offset filter information decoding unit 61 ′ includes an offset information decoding unit 611 and a QAOU structure decoding unit 612 .

[0382] The offset information decoding unit 611 refers to the QAOU information included in the coded data #1 and decodes the offset information OI included in the QAOU information. Also, the offset information decoding unit 611 supplies the values ​​of “sao_type_idx[sao_curr_depth][ys][xs][component]” and “sao_offset[sao_curr_depth][ys][xs][i]” obtained by decoding the offset information OI to the offset information storage unit 621 in association with the arguments (sao_curr_depth, ys, xs) and (sao_curr_depth, ys, xs, i), respectively.

[0383] More specifically, the offset information decoding unit 611 decodes the code from the encoded data #1, converts the decoded code into a value of "sao_type_idx", and supplies the decoded code to the offset information storage unit 621 in association with an argument. Here, the offset information decoding unit 611 changes the code decoding method and the conversion from the code to the value of "sao_type_idx" depending on the hierarchical depth of the QAOU to be processed. These conditions such as the hierarchical depth of the QAOU are called parameter conditions. The decoding of offset information according to general parameter conditions will be described later with reference to Figs. 41 and 42.

[0384] The code decoding method may use different maximum values ​​when the hierarchical depth of the QAOU to be processed is less than a threshold and when the hierarchical depth of the QAOU to be processed is equal to or greater than a threshold, or may use the maximum value only when the hierarchical depth of the QAOU to be processed is equal to or greater than a threshold. Also, different binarizations may be used when the hierarchical depth of the QAOU to be processed is less than a threshold and when the hierarchical depth of the QAOU to be processed is equal to or greater than a threshold. Furthermore, different contexts may be used when the hierarchical depth of the QAOU to be processed is less than a threshold and when the hierarchical depth of the QAOU to be processed is equal to or greater than a threshold.

[0385] For example, if the hierarchical depth of the QAOU to be processed is smaller than a threshold, the code may be decoded using variable length coding (ue(v)), and if the hierarchical depth of the QAOU to be processed is equal to or greater than the threshold, the code may be decoded using truncated coding (te(v)) according to the number of offset types. Also, if the number of offset types is a power of 2, the code may be decoded using fixed length coding. If the number of offset types is four, they can be expressed in 2 bits, so fixed length coding can be performed in 2 bits.

[0386] Furthermore, the offset information decoding unit 611 converts the decoded code into the value of "sao_type_idx" using conversion tables 801 and 802 as shown in Fig. 25. Conversion table 801 in Fig. 25(a) shows two conversion patterns. These two conversion patterns are used depending on the hierarchical depth of the QAOU to be processed. That is, the offset information decoding unit 611 uses conversion table 801A when the hierarchical depth of the QAOU to be processed is smaller than a threshold, and uses conversion table 801B when the hierarchical depth of the QAOU to be processed is equal to or greater than the threshold.

[0387] 25(a), offset type "EO_0" ("sao_type_idx" = 1) is associated with code "1," offset type "EO_1" ("sao_type_idx" = 2) is associated with code "2," and offset type "EO_2" ("sao_type_idx" = 3) is associated with code "3." The same applies to codes 4 to 6.

[0388] 25(a), offset type "EO_0" ("sao_type_idx" = 1) is associated with code "1," offset type "BO_0" ("sao_type_idx" = 5) is associated with code "2," and offset type "BO_1" ("sao_type_idx" = 6) is associated with code "3." Codes 4 to 6 are not used in conversion table 801B.

[0389] In this way, the conversion table 801A includes all offset types used in the adaptive offset (SAO), whereas the conversion table 801B includes only some of the offset types used in the adaptive offset (SAO). Therefore, when the hierarchical depth of the QAOU to be processed is equal to or greater than the threshold, only some of the offset types can be used.

[0390] This is because in deeper layers, the area of ​​the QAOU becomes smaller, so the characteristics of the pixel values ​​in the QAOU become more uniform, and appropriate offsets can be derived without using many types. This also makes it possible to reduce the number of offset types used, thereby reducing the required memory amount, and by shortening the data length of the encoded data indicating the offset type, the encoding efficiency can be improved.

[0391] If the correspondence between codes and offset types in conversion table 801B is the same as the correspondence between codes and offset types in conversion table 801A, a configuration using only conversion table 801A is also possible.

[0392] The conversion table 802 in Fig. 25(b) shows various examples of conversion tables similar to the conversion table 801B that can be used instead of the conversion table 801B. The types of each conversion table are limited, just like the conversion table 801B. Also, blank spaces indicate that the corresponding code is not used.

[0393] The conversion table 802A is an example using only edge offsets and does not include band offsets. The number of edge offsets is usually 4, and the number of band offsets is usually 16. Thus, the number of edge offsets is smaller than the number of band offsets. Therefore, by restricting the use of band offsets, the amount of memory used to hold offsets can be reduced. In particular, when the hierarchy is deep, the offsets increase, so the amount of memory increases. Also, when the hierarchy is deep, the selection rate of band offsets decreases. Therefore, by using the hierarchy as a parameter condition, when the hierarchy is deep, the conversion table 802A having only edge offsets is used, and when the hierarchy is shallow, the conversion table (e.g. 801A) having edge offsets and band offsets is used, it is possible to reduce the amount of memory without reducing the encoding efficiency. Also, it is possible to omit cost calculation of unnecessary options in the encoding device, so that the amount of processing can be reduced.

[0394] Conversion table 802B is an example that uses only band offsets and does not include edge offsets. Band offsets are characterized by the fact that the amount of calculation required is smaller than that of edge offsets, and that line memory for holding reference pixels is not required because pixels surrounding the target pixel are not used. Therefore, the above effects can be obtained by using conversion table 802B according to parameter conditions. Conversion table 802C is an example that uses only one band offset "BO_0" ("sao_type_idx"=5).

[0395] Also, conversion table 802D is an example using one edge offset and two band offsets like conversion table 801B. Specifically, this is an example using offset types "EO_0" ("sao_type_idx" = 1), "BO_0" ("sao_type_idx" = 5), and "BO_1" ("sao_type_idx" = 6), and associating short codes (small code numbers) with band offset types preferentially. When the hierarchy is shallow, the band offset selection rate is higher than when the hierarchy is deep. Therefore, by using the hierarchy as a parameter condition, and using conversion table 802D when the hierarchy is shallow, and associating frequently used types with short codes, it is possible to improve coding efficiency.

[0396] In addition to the examples shown in Fig. 25, the correspondence between codes and offset types can be changed according to parameter conditions. An offset type different from any edge offset or band offset in conversion table 801A can be used alone or together with other offset types according to conditions such as hierarchical depth. Specific examples include an offset type having both EO and BO characteristics, an edge offset that detects edges at horizontal sample positions different from the conventional edge offset "EO_0", and a band offset with a band allocation different from the conventional band offsets "BO_0" and "BO_1", which will be described in another embodiment below.

[0397] The QAOU structure decoding unit 612 determines the partition structure of the QAOU by decoding “sao_split_flag[sao_curr_depth][ys][xs]” contained in the QAOU information, and supplies the QAOU structure information representing the determined partition structure of the QAOU to the offset information storage unit 621.

[0398] The adaptive offset filter 60' may also include an offset attribute setting unit 613 as described below. The offset attribute setting unit 613 determines an offset shift value. The offset of the encoded data is encoded with an offset bit depth (also referred to as SAO_DEPTH) that is less accurate than the pixel bit depth (also referred to as PIC_DEPTH). That is, the offset in the encoded data is quantized. The shift value indicates the amount of bit shift required for inverse quantization. The offset attribute setting unit 613 also determines the offset bit depth and the offset value range. Here, the offset bit depth is determined from a pixel bit depth (also referred to as PIC_DEPTH) (not shown) that is input to the offset attribute setting unit 613. The pixel bit depth indicates the value range of pixel values ​​constituting the input image of the adaptive offset filter 60' in bit width. When the pixel bit depth is N bits, the pixel value ranges from 0 to 2. N Range is -1.

[0399] The bit depth and shift value of SAO are calculated using the following formulas, but other values ​​may be used depending on the parameter conditions, as described later.

[0400] SAO_DEPTH = MIN(PIC_DEPTH, 10) Shift value = PIC_DEPTH - MIN(PIC_DEPTH, 10) As one configuration, the offset value range (here, the maximum value) is determined by the following formula:

[0401] Offset range=2 SAO_DEPTH-K -1 Here, K is a predetermined constant (described later).

[0402] As shown in FIG. 22, the adaptive offset filter processing unit 62 ′ includes an offset information storage unit 621 , a QAOU control unit 622 , an offset type derivation unit 623 , a class classification unit 624 , an offset derivation unit 625 , and an offset addition unit 626 .

[0403] The offset information storage unit 621 manages and stores the offset type specified for each QAOU and the specific values ​​of the offset for each class selectable for that offset type based on the QAOU structure information, "sao_type_idx[sao_curr_depth][ys][xs]", and "sao_offset[sao_curr_depth][ys][xs][i]", and is configured with a map memory and a list memory.

[0404] The map memory and the list memory will be described with reference to Fig. 24. Fig. 24 is a diagram showing an example of what is stored in the map memory and the list memory, Fig. 24(a) is a diagram for explaining an example of a QAOU index stored in the map memory, and Fig. 24(b) is a diagram for explaining an example of information stored in the list memory.

[0405] The map memory 601 stores QAOU indexes (described later) that are assigned to each offset minimum unit (also referred to as QAOMU: Quad Adaptive Offset Minimum Unit) determined according to the division depth. FIG. 24(a) shows each QAOMU with a division depth of 3 that constitutes the target processing unit (e.g., LCU) and the QAOU index assigned to each QAOMU. Note that in FIG. 24(a), indexes 0 to 9 are simply assigned to the QAOU without considering the division depth of the QAOU. In the example shown in FIG. 24(a), the QAOU specified by the QAOU index=I is represented as QAOUI. Also, thin lines in FIG. 24(a) indicate the boundaries of the QAOMU, and thick lines indicate the boundaries of the QAOU.

[0406] As shown in Figure 24(a), QAOU0 is composed of four QAOMUs, and these four QAOMUs are assigned a QAOU index of 0. On the other hand, QAOU3 is composed of one QAOMU, and this QAOMU is assigned a QAOU index of 3. In this way, the map memory stores the QAOU index assigned to each QAOMU.

[0407] Furthermore, in the list memory 602, for each QAOU index, an offset type associated with the QAOU index and a specific value of an offset for each class selectable in the offset type are stored in association with each other.

[0408] A specific description will be given with reference to Fig. 24(b). Fig. 24(b) shows offset types associated with each of QAOU indexes 0 to 9 and offsets related to each class that can be selected for each offset type. "xxx" in Fig. 24(b) represents specific numerical values ​​that may differ from one another.

[0409] Also, "BO_1" in FIG. 24(b) represents the offset type specified by "sao_type_idx"=5. Also, "EO_1" represents the offset type specified by "sao_type_idx"=1. In this way, edge offsets that are offset types specified by "sao_type_idx"=1, 2, 3, 4 are also written as "EO_1, 2, 3, 4", respectively, and band offsets that are offset types specified by "sao_type_idx"=5, 6 are also written as "BO_1, 2", respectively.

[0410] 24(b), when the offset type is a band offset, the offsets stored in the list memory for that offset type are a total of 16, offset 1 to offset 16. Here, offset 1 to offset 16 refer to the values ​​specified by "sao_offset[sao_curr_depth][ys][xs][1]" to "sao_offset[sao_curr_depth][ys][xs]

[16] ", respectively, when the value of "sao_type_idx[sao_curr_depth][ys][xs]" is "5" or "6".

[0411] On the other hand, when the offset type is an edge offset, the offsets stored in the list memory for that offset type are a total of four, offsets 1 to 4. Here, offsets 1 to 4 refer to the values ​​specified by "sao_offset[sao_curr_depth][ys][xs][1]" to "sao_offset[sao_curr_depth][ys][xs][4]" respectively when the value of "sao_type_idx[sao_curr_depth][ys][xs]" is any of "1, 2, 3, 4". Note that in the case of an edge offset, nothing is stored in offsets 5 to 16.

[0412] Each QAOMU is assigned a QAOMU number, which allows each QAOMU to be distinguished from others. In the following, the QAOMU number is N Q QAOMU is QAOMUN Q It can also be written as:

[0413] In addition, in this embodiment, the accuracy of the offset may be different depending on the layer of the QAOU to be processed. The shift value used for inverse quantization also differs depending on the accuracy of the offset. In this case, the accuracy of the offset and the shift value are derived using the pixel bit depth PIC_DEPTH.

[0414] As an example of the accuracy of the offset, for example, when the depth of the QAOU hierarchy is smaller than the threshold, the accuracy of the offset, the shift value, SAO_DEPTH = MIN(PIC_DEPTH, 10) Shift value = PIC_DEPTH - MIN(PIC_DEPTH, 10) The accuracy of the offset when the depth of the QAOU hierarchy is equal to or greater than the threshold is defined as SAO_DEPTH = MIN(PIC_DEPTH, 8) Shift value = PIC_DEPTH - MIN(PIC_DEPTH, 8) In this way, the offset precision can be made different.

[0415] As described above, the offset precision (offset bit depth, SAO_DEPTH) and the pixel bit depth (PIC_DEPTH) that expresses the range of pixel values ​​constituting the input image in bit width are closely related to each other in terms of quantization error. The bit depth of the output image of the adaptive offset filter 60 is the pixel bit depth PIC_DEPTH, and SAO_DEPTH is the bit depth of the offset added to the pixel, so even if an offset with precision exceeding the pixel bit depth is used, it will be discarded in the output process, and there is no point in setting SAO_DEPTH to exceed PIC_DEPTH. Conversely, if SAO_DEPTH is smaller than PIC_DEPTH, the input image can only be corrected coarser than the precision (PIC_DEPTH) that can be corrected by the filter, and the filter effect is reduced.

[0416] Therefore, by limiting the quantized offset to an offset value range that can be expressed with a certain bit width, it is possible to limit the bit width for storing the quantized offset in the offset information storage unit 621. This makes it possible to obtain the effect of reducing memory size compared to the case where no limit is imposed. On the other hand, excessively narrowing the offset value range reduces the effect of the offset in correcting the distortion of the decoded image, and the distortion of the decoded image cannot be removed even by the offset addition process, resulting in a decrease in coding efficiency.

[0417] Therefore, by setting the offset value range to an optimal range so that the coding efficiency does not decrease, it is possible to reduce the memory usage while maintaining the effect of the filter.

[0418] Then, the maximum bit length representing the offset range value is CLIP_BIT, and the offset range is calculated as CLIP_BIT = SAO_DEPTH-K. CLIP_BIT-1 ~2 CLIP_BIT-1 When K is set to −1, it has been discovered through experiments by the inventors that when K=4, there is no decrease in coding efficiency even if the offset range is restricted by the offset value range.

[0419] When the pixel bit depth is 8, the offset bit depth SAO_DEPTH is also 8, and CLIP_BIT = 8 - K = 4. The fact that one offset can be stored in 4 bits means that in software that handles 8-bit bytes as units, one offset can be packed and stored in one byte, making it possible to easily reduce memory size.

[0420] If the precision of the offset to be set differs depending on the hierarchical level of the QAOU to be processed, the offset information storage unit 621 secures the offset storage area by switching the unit size according to the hierarchical depth of the QAOU to be processed.

[0421] Specifically, when the depth of the hierarchy of the QAOU to be processed is less than the threshold, the accuracy of the offset is n A Bits (e.g., n A = 8 or 6), the accuracy of the offset when the hierarchical depth of the QAOU to be processed is equal to or greater than the threshold B Bits (n A >n B , e.g., n B =n A When the offset information storage unit 621 is set to n−2, the offset information storage unit 621 allocates an area for storing an offset when the hierarchical depth of the QAOU to be processed is less than the threshold. AThe area to store the offset when the hierarchical depth of the QAOU to be processed is equal to or greater than the threshold is allocated in bits. B It is allocated in bit units.

[0422] If the hierarchical depth of the QAOU to be processed is equal to or greater than the threshold, when reading or writing an offset from the offset information storage unit 621, the following is executed: A -n B ) bits rounded down, when reading (n A -n B It is preferable to have a configuration in which input and output are performed by rounding up the offset by 1.5 bits, because this configuration eliminates the need to perform processing in other modules taking into account differences in offset precision.

[0423] In addition, when the number of required classes changes depending on the hierarchical depth of the QAOU to be processed, the offset information storage unit 621 switches the area of ​​the list memory to be secured according to the number of required classes. For example, as described later, consider the case of a band offset that is classified into 16 classes when the hierarchical depth of the QAOU to be processed is smaller than a threshold, and is classified into 8 classes when the hierarchical depth of the QAOU to be processed is equal to or greater than the threshold. In this case, the memory capacity required to secure classes when the hierarchical depth of the QAOU to be processed is equal to or greater than the threshold is half that when the hierarchical depth of the QAOU to be processed is smaller than the threshold. Therefore, the offset information storage unit 621 switches the size of the list memory to be secured when the hierarchical depth of the QAOU to be processed is equal to or greater than the threshold, to half that when the hierarchical depth of the QAOU to be processed is smaller than the threshold.

[0424] The QAOU control unit 622 controls each unit included in the adaptive offset filter processing unit 62'. The QAOU control unit 622 also refers to the QAOU structure information, divides the deblocked decoded image P_DB into one or more QAOUs, and scans each QAOU in a predetermined order. The QAOU control unit 622 also supplies the offset type derivation unit 623 with a QAOMU number indicating the target QAOMU to be processed.

[0425] The offset type derivation unit 623 refers to the map memory and list memory of the offset information storage unit 621, and derives the offset type specified by the QAOMU number supplied from the QAOU control unit 622. In addition, it supplies the derived offset type to the classifying unit 624.

[0426] The class classification unit 624 classifies each pixel included in the target QAOU into one of a plurality of classes selectable in the offset type supplied from the offset type derivation unit 623. In addition, the class classification unit 624 supplies the offset type and a class index indicating the class into which each pixel has been classified to the offset derivation unit 625. Note that the specific classification process by the class classification unit 624 will be described later, and therefore will not be described here.

[0427] The offset derivation unit 625 refers to the list memory of the offset information storage unit 621, and derives an offset specified by the offset type and class index supplied from the classifying unit 624 for each pixel included in the target QAOU. It further includes an offset inverse shift unit (not shown) that bit-shifts the offset to the left by the shift value set by the offset attribute setting unit 613. The offset inverse shift unit inversely quantizes the offset so as to match the bit depth of the offset with the pixel bit depth. By performing such inverse quantization, it is possible to add the pixel value and the offset at the same bit depth in the addition process of the offset addition unit 626 described later. The inversely quantized offset for each pixel is supplied to the offset addition unit 626.

[0428] The offset addition unit 626 adds, to each pixel of the deblocked decoded image P_DB in the target QAOU, the offset supplied from the offset derivation unit 625. The offset addition unit 626 outputs an image obtained by performing processing on all QAOUs included in the deblocked decoded image P_DB as an offset-filtered decoded image P_OF.

[0429] Next, the classification process by the classifying unit 624 will be described with reference to Figs. 26 to 29. Note that, in the following, a case where a condition such as the depth of the QAOU hierarchy is used as a parameter condition will be described, but the configuration of a classifying unit according to general parameter conditions will be described later with reference to Fig. 34.

[0430] Figure 26 is a diagram for explaining offset processing by adaptive offset filter 60, in which (a) is a graph showing the magnitude relationship between the pixel value pic[x] of the pixel x to be processed and the pixel value of pixel a or b, and the value of the function Sign corresponding to this magnitude relationship, (b) is a graph showing the magnitude relationship between the pixel value of the pixel x to be processed and the pixel values ​​of pixels a and b, and the value of EdgeType corresponding to this magnitude relationship, (c) shows the correspondence between each graph shown in (b) and class_idx, and (d) to (f) show conversion tables showing the conversion from EdgeType to class_idx.

[0431] FIG. 27 is a diagram for explaining offset processing by the adaptive offset filter 60, in which (a) shows a schematic diagram of class classification when “sao_type_idx”=5, and (b) shows a schematic diagram of class classification when “sao_type_idx”=6.

[0432] Figure 28 is a diagram for explaining offset processing by the adaptive offset filter 60, where (a) shows a schematic diagram of class classification when the hierarchical depth of the QAOU to be processed is less than a threshold, and (b) shows a schematic diagram of class classification when the hierarchical depth of the QAOU to be processed is greater than or equal to the threshold.

[0433] Figure 29 is a diagram showing an example of class classification when a band offset is specified, where (a) shows an example of class classification when the hierarchical depth of the QAOU to be processed is less than a threshold, and (b) shows an example of class classification when the hierarchical depth of the QAOU to be processed is greater than or equal to the threshold.

[0434] (When offset type = 1 to 4 (edge ​​offset)) When the offset type supplied from the offset type derivation unit 623 is any one of 1 to 4, the processing of the classifying unit 624 is as described above with reference to FIG.

[0435] Therefore, as shown in Fig. 26(a), in terms of the magnitude relationship between the pixel value pic[x] and the pixel value of pixel a or b, if the pixel value pic[x] is smaller than the pixel value of pixel a or b, Sign(pic[x]-pic[a])=-1, if they are the same, Sign(pic[x]-pic[a])=0, and if they are larger, Sign(pic[x]-pic[a])=1. Note that in Fig. 26(a), a black circle with pic[x] attached indicates the pixel value of the processing target pixel x, and a black circle without pic[x] attached indicates the pixel value of the processing target pixel a or b. Also, the up and down direction in Fig. 26(a) indicates the magnitude of the pixel values.

[0436] Next, the classifying unit 624 derives EdgeType based on Sign(pic[x]-pic[a]) and Sign(pic[x]-pic[b]) using the following formula (1-1).

[0437] EgdeType=Sign(pic[x]-pic[a])+Sign(pic[x]-pic[b])+2 ···(1-1) As a result, as shown in FIG. 26(b), when the pixel values ​​of both pixels a and b are larger than the pixel value pic[x], EdgeType=0. When the pixel value of one of pixels a and b is larger than the pixel value pic[x] and the pixel value of the other is the same, EdgeType=1. When the pixel value of one of pixels a and b is smaller than the pixel value pic[x] and the pixel value of the other is the same, EdgeType=3. When the pixel values ​​of both pixels a and b are smaller than the pixel value pic[x], EdgeType=4. When the pixel value of one of pixels a and b is smaller than the pixel value pic[x] and the pixel value of the other is larger than the pixel value pic[x], or when the pixel values ​​of both pixels a and b are the same than the pixel value pic[x], EdgeType=2.

[0438] In Fig. 26(b), the black circle at the center of each graph indicates the pixel value of the processing target pixel x, and the black circles on both ends indicate the pixel values ​​of pixels a and b. Also, the up and down direction in Fig. 26(b) indicates the magnitude of the pixel values.

[0439] Next, the class classification unit 624 derives the class index (class_idx) of the class to which the processing target pixel x should belong based on the derived EdgeType as follows.

[0440] class_idx=EoTbl[EdgeType] Here, EoTbl[EdgeType] is a conversion table used to derive class_idx from EdgeType. Specific examples of the conversion table EoTbl are shown in Figs. 26(d) to (f).

[0441] A conversion table 1001X shown in FIG. 26(d) is a conversion table in the case where the conversion table to be used is not changed depending on the hierarchical depth of the QAOU to be processed.

[0442] In addition, conversion table 1001A shown in Figure 26(e) and conversion table 1001B shown in Figure 26(f) are conversion tables in which the conversion table to be used is changed depending on the hierarchical depth of the QAOU to be processed, and conversion table 1001A is used when the hierarchical depth of the QAOU to be processed is smaller than a threshold, and conversion table 1001B is used when the hierarchical depth of the QAOU to be processed is equal to or greater than the threshold.

[0443] When the conversion table to be used is not changed depending on the hierarchical depth of the QAOU to be processed, as shown in conversion table 1001X, the classification unit 624 classifies the processing target pixel x into class 0 ("class_idx"=0) when there is no edge in the area consisting of the processing target pixel x, pixel a, and pixel b (hereinafter also referred to as the flat case), that is, when EdgeType=2. Also, it classifies EdgeType=0, 1, 3, 4 into class_idx=1, 2, 3, 4, respectively.

[0444] In addition, when the conversion table to be used is not changed depending on the hierarchical depth of the QAOU to be processed, and the hierarchical depth of the QAOU to be processed is smaller than the threshold value, as shown in the conversion table 1001A, the classifying unit 624 classifies the processing target pixel x into class 0 ("class_idx"=0) when there is no edge in the area consisting of the processing target pixel x, pixel a, and pixel b (hereinafter also referred to as the flat case), that is, when EdgeType=2. Also, EdgeType=0, 1, 3, and 4 are classified into class_idx=1, 3, 4, and 2, respectively. In addition, when the hierarchical depth of the processing target QAOU is equal to or greater than the threshold value, as shown in the conversion table 1001B, the classifying unit 624 classifies the processing target pixel x into class 0 ("class_idx"=0) when there is no edge in the area consisting of the processing target pixel x, pixel a, and pixel b (hereinafter also referred to as the flat case), that is, when EdgeType=2. Moreover, EdgeType=0, 1 is classified as class_idx=1, and EdgeType=3, 4 is classified as class_idx=2. Therefore, in conversion table 1001B, one class_idx corresponds to a plurality of EdgeTypes.

[0445] It is considered that conversion table 1001A may be the same as conversion table 1001X. However, when two conversion tables are used depending on the layer, if the same table defined in conversion table 1001X is used as conversion table 1001A, different Edgetypes will be assigned to the same class_idx between conversion table 1001A and conversion table 1001B, and the processing will change depending on the layer. For example, when class_idx=2, if the layer depth is smaller than the threshold (when conversion table 1001X is used), Edgetype=1, and if the layer depth is equal to or greater than the threshold (when conversion table 1001B is used), Edgetype=4, and the processing will change depending on the layer.

[0446] Therefore, by configuring the conversion table 1001A as shown in FIG. 26(e), the processing does not change between the conversion table 1001A and the conversion table 1001B, that is, between hierarchical levels.

[0447] (When offset type = 5~6 (band offset)) When the offset type supplied from the offset type derivation unit 623 is 5 or 6, the class classification unit 624 classifies the pixel value of the processing target pixel x into one of a plurality of classes according to the pixel value pic[x] of the processing target pixel x.

[0448] When offset type = 5 (sao_type_idx = 5) The classifying unit 624 classifies the pixel value pic[x] of the processing target pixel x as follows: (max×1 / 4)≦pic[x]≦(max×3 / 4) satisfies the above, the pixel to be processed is classified into a class other than class 0. In other words, if the pixel value of the pixel to be processed is within the shaded range in FIG. 27(a), the pixel to be processed is classified into a class other than class 0. Note that the above max represents the maximum possible value of the pixel value of the pixel to be processed x, and is, for example, max=255. When max=255, the above condition can also be expressed as 8≦(pic[x] / 8)≦23 or 4≦(pic[x] / 16)≦11.

[0449] When offset type = 6 (sao_type_idx = 6) The classifying unit 624 classifies the pixel value pic[x] of the processing target pixel x as follows: pic[x]≦(max×1 / 4) or (max×3 / 4)≦pic[x] satisfies the above, the pixel to be processed is classified into a class other than class 0. In other words, if the pixel value of the pixel to be processed is within the shaded range in FIG. 27(b), the pixel to be processed is classified into a class other than class 0. Note that the above max represents the maximum possible pixel value of the pixel to be processed x, and is, for example, max=255. When max=255, the above condition can also be expressed as (pic[x] / 8)≦7 or 24≦(pic[x] / 8), or (pic[x] / 16)≦3 or 12≦(pic[x] / 16).

[0450] The classification process performed by the classification unit 624 will be described in more detail below.

[0451] When the offset type is any one of 5 to 6, the classification unit 624 derives the class index (class_idx) of the class to which the processing target pixel x should belong, according to the hierarchical depth of the processing target QAOU, as follows: - When the hierarchical depth of the QAOU to be processed is less than the threshold class_idx=EoTbl[sao_type_idx][pic[x] / >>BoRefBit32] ·When the depth of the hierarchy of the QAOU to be processed is equal to or greater than the threshold value class_idx = EoTbl[sao_type_idx][pic[x] / >>BoRefBit16] Here, EoTbl[sao_type_idx][pic[x] / >>BoRefBit32] and EoTbl[sao_type_idx][pic[x] / >>BoRefBit16] are conversion tables used to derive class_idx from the pixel value pic[x] of the pixel x to be processed and sao_type_idx. BoRefBit32 and BoRefBit16 are values derived from PIC_DEPTH - 5 and PIC_DEPTH - 4, respectively, when the image bit depth is PIC_DEPTH, and are the quantization of the pixel value into 32 levels or 16 levels. The quantized pixel value is also described as pixquant. Right-shifting by BoRefBit32 and BoRefBit16 corresponds to division by 1 << BoRefBit32 and 1 << BoRefBit16. When the bit depth of the pixel value is 8 bits, the quantization widths are 1 << (8 - 5) = 8 and 1 << (8 - 4) = 16, respectively. Hereinafter, the case where the quantization width is 8 will be described. Also, this quantization width is referred to as the class width.

[0452] The class classification unit 624 performs class classification by changing the class width for classification according to the depth of the hierarchy of the QAOU to be processed. For example, as shown in FIG. 28(a), when the depth of the hierarchy of the QAOU to be processed is less than the threshold value, the class classification unit 624 classifies the pixel values into 32 with the class width of "8" and performs class classification. Also, as shown in FIG. 28(b), when the depth of the hierarchy of the QAOU to be processed is equal to or greater than the threshold value, the class classification unit 624 classifies the pixel values into 16 with the class width of "16" and performs class classification.

[0453] Next, a specific example of the conversion table EoTbl is shown in Fig. 29. In each of (a) and (b) of Fig. 29, "BO_1" indicates that "sao_type_index" = 5, and "BO_2" indicates that "sao_type_index" = 6. Furthermore, the conversion table 1301 shown in (a) of Fig. 29 is a conversion table used when the hierarchical depth of the QAOU to be processed is smaller than a threshold, and the conversion table 1302 shown in (b) of Fig. 29 is a conversion table used when the hierarchical depth of the QAOU to be processed is equal to or greater than a threshold.

[0454] When the hierarchical depth of the QAOU to be processed is smaller than the threshold and “sao_type_index”=5, as shown in FIG. 29(a), the class classification unit 624 classifies a pixel x to be processed whose pixel value pic[x] satisfies 8≦(pic[x] / 8)≦23 into one of the classes from class index 1 to 16 depending on the magnitude of pic[x].

[0455] In addition, when “sao_type_index”=6, the class classification unit 624 classifies a processing target pixel x whose pixel value pic[x] satisfies pic[x] / 8)≦7 or 24≦(pic[x] / 8) into one of the classes from class index 1 to 16 depending on the magnitude of pic[x].

[0456] In addition, when the hierarchical depth of the QAOU to be processed is equal to or greater than the threshold and “sao_type_index”=5, as shown in FIG. 29(b), the class classification unit 624 classifies the pixel x to be processed, whose pixel value pix[x] satisfies 4≦(pix[x] / 16)≦11, into one of the class indexes 1 to 8 depending on the magnitude of pix[x].

[0457] In addition, when “sao_type_index”=6, the class classification unit 624 classifies a processing target pixel x whose pixel value pix[x] satisfies pix[x] / 16)≦3 or 12≦(pix[x] / 16) into one of the class indexes 1 to 8 depending on the magnitude of pix[x]. (Configuration of offset information decoding unit 611) 41 is a block diagram of an offset information decoding unit 611 that changes the offset type to be used according to the type of parameter (parameter condition) of the adaptive offset and / or changes the number of classifications of classes according to the parameter condition of the adaptive offset. The offset information decoding unit 611 is composed of an adaptive offset type decoding unit 6111, a used offset type selecting unit 6112, an offset type decoding unit 6113, an adaptive offset decoding unit 6114, a used offset number selecting unit 6115, and an offset decoding unit 6116. Note that the parameter conditions are parameters other than values ​​calculated from pixel values, and include the hierarchical level and offset type described above, as well as the block size (QAOU size), color components (components), and QP described in the appended notes below.

[0458] The adaptive offset type decoding unit 6111 is a means for adaptively decoding the offset type in accordance with the parameter conditions from the QAOU information in the encoded data, and includes a use offset type selection unit 6112 and an offset type decoding unit 6113 .

[0459] The offset type selection unit 6112 is a means for selecting an offset type to be used according to a parameter condition. One of the parameter conditions is the hierarchy as described above. The number of offset types to be used when the hierarchy is shallow is smaller than the number of offsets to be used when the hierarchy is deep. The offset type selection unit 6112 inputs a conversion table such as that described in FIG. 25 to the offset type decoding unit 6113 according to the parameter condition. The offset type selection unit 6112 also inputs the maximum number of usable offset types to the offset type decoding unit 6113.

[0460] The offset type decoding unit 6113 decodes the offset type from the encoded data according to the input conversion table and maximum number. If the maximum number of offset types is N, the range of possible codes is limited to N, from 0 to N-1, so the number of bits required for encoding the code can be reduced. For example, if the maximum number is 2m-1 Greater than 2 m When the length is less than or equal to m bits, fixed-length coding of m bits can be used. Also, truncated unary coding with a maximum value of N-1 or truncated rice coding can be used.

[0461] The adaptive offset decoding unit 6114 is a means for adaptively decoding an offset from QAOU information in the encoded data according to a parameter condition, and includes a number of offsets used selection unit 6115 and an offset decoding unit 6116. The number of offsets used selection unit 6115 is a means for selecting the maximum number of offsets used and the precision of the offsets according to the parameter condition. One of the parameter conditions is a hierarchy, and the number of offsets used when the hierarchy is shallow is greater than the number of offsets used when the hierarchy is deep. For example, when the hierarchy is shallow, the number of offsets can be 16 as shown in FIG. 29(a), and when the hierarchy is deep, the number of offsets can be 8 as shown in FIG. 29(b). Also, as described above, it is possible to change the precision (bit depth) of the offset according to the hierarchy. The number of offsets used selection unit 6115 inputs the maximum number of offsets and the precision of the offset to the offset decoding unit 6116. The offset decoding unit 6116 decodes the offset according to the maximum number of offsets and the precision of the offset. When the number of offsets is reduced, the amount of offset code is reduced. Also, similarly to the offset type, once the precision of each offset is determined, the range of possible codes for encoding the offset is also limited, making it possible to reduce the number of bits required for encoding the code.

[0462] FIG. 42( a ) is a block diagram showing the configuration of the offset type selection unit 6112 .

[0463] The offset type to be used selection unit 6112 includes an offset type conversion table selection unit 6117 , a first offset type conversion table storage unit 6118 , and a second offset type conversion table storage unit 6119 .

[0464] The offset type conversion table selection unit 6117 selects, according to the parameter condition, a conversion table provided in the first offset type conversion table storage unit 6118 or a conversion table provided in the second offset type conversion table storage unit 6119. In the above example, 801A is the conversion table provided in the first offset type conversion table storage unit 6118, and 801B corresponds to the conversion table provided in the second offset type conversion table storage unit 6119.

[0465] FIG. 42(b) is a block diagram showing another configuration of the use offset type selection unit.

[0466] The offset type to be used selection unit 6112' includes an offset type conversion table selection unit 6117', an edge offset type and band offset type conversion table storage unit 6118', and a horizontal edge offset type and band offset type conversion table storage unit 6119'.

[0467] (Configuration of Classification Unit 624) FIG. 43 is a block diagram showing the configuration of the classifying unit 624. As shown in FIG.

[0468] The classifying unit 624 is made up of an adaptive edge offset classifying unit 6241 , a used edge offset class selecting unit 6242 , an edge offset class classifying unit 6243 , an adaptive band offset class classifying unit 6244 , a used band offset class / class width selecting unit 6245 , and a band offset class classifying unit 6246 .

[0469] The classification unit 624 classifies each pixel into a class according to the parameter condition and the offset type. If the offset type indicates an edge offset, the pixel is classified by an adaptive edge offset classification unit 6241, and if the offset type indicates a band offset, the pixel is classified by an adaptive band offset classification unit 6244.

[0470] The adaptive edge offset class classification unit 6241 is a means for adaptively classifying pixels into classes according to parameter conditions, and includes a used edge offset class selection unit 6242 and an edge offset class classification unit 6243. The used edge offset class selection unit 6242 selects the type of class to be used. The used edge offset class selection unit 6242 inputs a classification method for pixel values ​​to the edge offset class classification unit 6243. Specifically, it inputs a derivation method for an intermediate value EdgeType that is temporarily derived when classifying pixel values, and a conversion table used to derive class_idx from EdgeType. An example of a derivation method for an intermediate value EdgeType is described using FIG. 26(a) and FIG. 26(b), and this is called a basic edge classification method. An edge classification method such as that described later in FIG. 33 can also be used. One of the parameter conditions is the hierarchy as described above, and the number of classes to be used when the hierarchy is shallow is selected to be greater than the number of classes to be used when the hierarchy is deep. Examples of the conversion table used to derive class_idx from EdgeType are 1001A and 1001B. Also, a method of switching the edge derivation method depending on the color component is appropriate. In this case, for luminance, it is appropriate to use a basic edge classification method that uses edge classification methods for horizontal, vertical, and diagonal directions, and for chrominance, it is appropriate to use a horizontal edge classification method that limits the range of reference pixels used in edge classification to the horizontal direction as seen from the target pixel in order to reduce line memory. The edge offset class classification unit 6243 classifies pixels based on a given classification method and a conversion table used to derive class_idx from EdgeType.

[0471] The adaptive band offset class classifier 6244 is a means for adaptively classifying pixels into classes according to parameter conditions, and includes a used band offset class / class width selector 6245 and a band offset class classifier 6246. The used band offset class / class width selector 6245 inputs the pixel value classification method to the band offset class classifier 6246. Specifically, the class width, which is the quantization width used when classifying pixel values into intermediate values, and the conversion table used to derive class_idx from the intermediate value are input. One of the parameter conditions is the hierarchy as already described. When the hierarchy is shallow, the width of the class used is smaller than the width of the class used when the hierarchy is deep. The band offset class classifier 6246 classifies pixel values into classes according to the input class width and the conversion table used to derive class_idx from the intermediate value. The input class width may be not the class width itself but an integer corresponding to the class width. For example, when 1<<BoRefBit32 is used as the class width, the number of bits BoRefBit32 used for pixel quantization, which is the logarithm to the base 2, or a value for obtaining the number of bits used for pixel quantization from the bit depth of the pixel, for example, if BoRefBit32 = PIC_DEPTH - 5, 5 may be used instead of the class width.

[0472] In this way, the encoded data is adaptively decoded according to parameter conditions, and pixel class classification is performed.

[0473] As described above, in this embodiment, the number of class classifications is changed according to the depth of the hierarchy of the QAOU to be processed. More specifically, when the depth of the hierarchy of the QAOU to be processed is large, it is classified into a smaller number of classes compared to when it is small.

[0474] Generally, since the required memory amount is Memory amount = Offset data length × Number of classes × Number of QAOU, the amount of memory used can be reduced by reducing the number of classes.

[0475] Furthermore, in deeper layers, the area of ​​the QAOU becomes smaller and the characteristics of the pixel values ​​within the QAOU become closer to uniform, so the effect of the offset does not change much even if the number of classes is reduced.

[0476] In this embodiment, the accuracy of the offset is changed according to the hierarchical depth of the QAOU to be processed. More specifically, when the hierarchical depth of the QAOU to be processed is large, the accuracy of the offset is lowered compared to when the hierarchical depth is small. This reduces the amount of code when the hierarchical depth of the QAOU to be processed is large.

[0477] Since the number of pixels included in a QAOU in a deeper layer is smaller, the quantization error of the entire QAOU is smaller than that in a higher layer. Therefore, even if the offset precision is lowered in a deeper layer, the effect of the offset does not change much.

[0478] (Video Encoding Device 2') Next, a video encoding device 2' that generates encoded data #1 by encoding a target image to be encoded will be described with reference to Figs. 30 and 31. Like the video encoding device 2, the video encoding device 2' includes, as a part thereof, the method adopted in H.264 / MPEG-4.AVC, the method adopted in KTA software, which is a codec jointly developed by VCEG (Video Coding Expert Group), the method adopted in TMuC (Test Model under Consideration) software, which is its successor codec, and the technology adopted in HM (HEVC Test Model) software.

[0479] Also, the video encoding device 2' differs in that an adaptive offset filter 80' is provided instead of the adaptive offset filter 80 in the video encoding device 2, but the other configurations are the same.

[0480] (Adaptive offset filter 80') Next, the adaptive offset filter 80' will be described with reference to Fig. 30. Fig. 30 is a block diagram showing the configuration of the adaptive offset filter 80'. As shown in Fig. 30, the adaptive offset filter 80' includes an adaptive offset filter information setting unit 81' and an adaptive offset filter processing unit 82'.

[0481] As shown in FIG. 30, the adaptive offset filter information setting unit 81 ′ includes an offset calculation unit 811 , an offset clipping unit 812 , and an offset information selection unit 813 .

[0482] (Offset calculation unit 811) The offset calculation unit 811 calculates offsets for all offset types and all classes that exist according to the hierarchy of the QAOU to be processed, for all QAOUs up to a predetermined division depth included in the target processing unit (e.g., LCU). Here, the offset types and classes refer to the same ones as those described in the description of the video decoding device 1.

[0483] The offset calculation unit 811 supplies the offset information, which is made up of the offset calculated by the above processing, the offset type, the class, and QAOU structure information indicating the division structure of the QAOU, to the offset clip unit 812.

[0484] (Offset clip part 812) The offset clip unit 812 performs clip processing on the offset supplied from the offset calculation unit 811 using either clip processing 1 or clip processing 2 as described below.

[0485] (Clip Processing 1) The offset clipping unit 812 clips each offset supplied from the offset calculation unit 811 to a value between −8 and 7, for example, to express each offset in 4 bits. Each clipped offset is supplied to an offset information selection unit 813. The bit width to be clipped is set according to the bit depth of the image and the bit depth of the offset, similar to the video decoding device 1.

[0486] In this way, by clipping each offset, it is possible to reduce the memory size of a memory (not shown) in which each offset is stored. Also, since the amount of code of the offset included in the coded data #1 can be reduced, it is possible to improve coding efficiency. Also, since the addition of an excessive offset is suppressed, appropriate image quality is guaranteed.

[0487] (Clip Processing 2) Furthermore, the offset clipping unit 812 may be configured to set the clipping range of each offset supplied from the offset calculation unit 811 to a different value depending on the offset type.

[0488] For example, if the offset type is edge offset, the number of offset bits is 8 bits, and if the offset type is band offset, the number of offset bits is 4 bits. More generally, when the number of offset bits is N bits when the offset type is edge offset, and the number of offset bits is M bits when the offset type is band offset, the number of offset bits is determined so that N>M is satisfied.

[0489] By thus varying the number of bits of the offset depending on the offset type, it is possible to improve the coding efficiency without requiring an excessive memory size for the memory for storing each offset.

[0490] The threshold value th for limiting the possible offset values ​​is 2 m-1 Greater than 2 mWhen th is equal to or less than 1, m-bit fixed-length coding can be used as a coding scheme for coding the offset. More specifically, truncated unary coding with th as the maximum value or truncated rice coding can be used.

[0491] This embodiment also includes clipping processing obtained by combining the above clipping processing 1 and 2. The adaptive offset filter 80' may not include the offset clip unit 812.

[0492] In addition, the offset clip unit 812 switches the clip range according to the hierarchical depth of the QAOU to be processed in accordance with the accuracy of the offset. Specifically, when the hierarchical depth of the QAOU to be processed is smaller than a threshold, the offset accuracy is n A bits, if the hierarchical depth of the QAOU to be processed is equal to or greater than the threshold, n B In this case, if the hierarchical depth of the QAOU to be processed is smaller than the threshold, the offset clip unit 812 sets the clip range to -2 nA / 2 ~2 nA / 2 If the hierarchical depth of the QAOU to be processed is equal to or greater than the threshold, the clip range is set to -2. nA / 2 ~2 nA / 2 -1, and then the lower (n A -n B ) bit is set to “0.” This allows the offset information selection unit 813, which performs processing using the offset clipped by the offset clipping unit 812, to perform processing without taking into account the accuracy of the offset.

[0493] (Offset information selection unit 813) The offset information selection unit 813 determines a combination of offset type, class, and offset that reduces the RD cost (Rate-Distortion cost) and a corresponding QAOU partition structure, and supplies the determined offset type, class, offset, and QAOU information indicating the corresponding QAOU partition structure to the variable-length code encoding unit 22. In addition, the offset information selection unit 813 supplies the determined offset to the adaptive offset filter processing unit 82 for each QAOU.

[0494] The process of the offset information selection unit 813 will be described in more detail below with reference to FIG.

[0495] Fig. 31 is a diagram showing an overview of calculating the squared error for each offset type for a QAOU with a QAOU index of "x". As shown in Fig. 31, the offset information selection unit 813 calculates the squared error for each offset type for all QAOMUs. Then, the offset type for which the calculated squared error is the smallest is set as the offset type for the QAOU. In this way, the offset types are determined for all QAOMUs (QAOMU numbers 0 to 340).

[0496] Next, the offset information selection unit 813 calculates the RD cost when the division depth is set to 0 and the RD cost when the division depth is set to 1. The specific calculation method is as described above with reference to FIG.

[0497] (Adaptive offset filter processing unit 82') The adaptive offset filter processing unit 82' adds an offset supplied from the offset information selection unit 813 to each pixel of the deblocked decoded image P_DB in the target QAOU. The adaptive offset filter processing unit 82' outputs an image obtained by performing processing on all QAOUs included in the deblocked decoded image P_DB as an offset-filtered decoded image P_OF. Note that the configuration of the adaptive offset filter processing unit 82' is the same as that of the adaptive offset filter processing unit 62', and therefore a description thereof will be omitted here.

[0498] Also, a configuration using an adaptive clip type may be used. That is, an adaptive clip (AC) type may be provided as one of the offset types. In this case, there are three offset types: EO, BO, and AC. In the adaptive clip type (AC), an offset is not used, and a pixel value is corrected using clipping with a lower limit value c1 and an upper limit value c2. Examples of the lower limit value c1 and the upper limit value c2 are, for example, c1=16 and c2=235.

[0499] By using an adaptive clip type, offset encoding processing and memory for holding a large number of offsets are not required. If the lower and upper limits of a clip are given adaptively instead of fixed values, the lower and upper limits may be encoded. In this case, it is preferable to encode the difference from an appropriate fixed value, for example, the lower limit is an offset from 16, and the upper limit is an offset from 235. In particular, since the upper limit is a large value, by doing so, the encoding efficiency is not reduced.

[0500] (Additional Note 2) As described above, "depending on the hierarchical depth of the QAOU" can also mean "depending on the size of the QAOU." In other words, the number of SAO types, the number of classes, and the offset precision may be varied depending on the size of the QAOU.

[0501] For example, when the size of the QAOU is less than N×N pixels, one or more of the number of SAO types, the number of classes, and the offset accuracy may be limited more than when the size of the QAOU is N×N pixels or more. A specific example of N is N=64 (the size of the LCU).

[0502] In addition, examples of limiting the number of SAO types include limiting EO to horizontal and vertical types, limiting BO to one type (conversion table 802C), using only EO without using BO (conversion table 802A), using only either horizontal type EO or BO (conversion table 801B, conversion table 802D), etc. The conversion tables shown in parentheses are used in the offset information decoding unit 611. Note that the number of SAO classes and offset accuracy can be limited in the same manner as in the above-mentioned embodiment.

[0503] In this way, by limiting the type of SAO, the number of types, the number of classes, and the offset precision, it is possible to reduce memory usage and the processing load. In particular, when limiting the type of type to BO, or when limiting EO to horizontal and vertical types, it is possible to reduce the processing load of the classifying unit of the encoding device and the decoding device. In addition, when limiting the number of types, it is possible to omit cost calculation for selecting the optimal type, and therefore it is possible to reduce the processing load of the encoding device. By limiting the type of type to EO, and limiting the number of classes and offset precision, it is possible to reduce the amount of memory usage for accumulating offsets. In addition, by limiting the type of type to horizontal types EO and BO, it is possible to reduce temporary memory such as line memory that holds reference pixels used for classifying, and it is also possible to shorten the delay while waiting for the decoding of the reference pixels. In addition, limiting the number of classes of BO is more effective in reducing memory usage than limiting the number of classes of EO.

[0504] Such restrictions can be realized by adaptively changing the operation of the offset information decoding unit 611 and the class classification unit 624 in accordance with the parameter condition when the parameter condition is the size of the QAOU using the configuration of the means shown in Figures 41 to 43.

[0505] (Appendix 3) In addition, the size of a QAOU (hereinafter also referred to as a luma unit) that adds an offset to a luma value may be made different from the size of a QAOU (hereinafter also referred to as a chroma unit) that adds an offset to a chroma value, and the type of SAO type, number of types, number of classes, offset accuracy, and maximum hierarchy of division may be more limited in the chroma unit than in the luma unit.

[0506] For example, in a data format in which the resolution of luminance and chrominance is different and the resolution of chrominance is lower than that of luminance, such as the 4:2:0 format in a YUV image format, the division of chrominance does not need to be as fine as that of luminance. Therefore, by restricting the type of SAO, the number of types, the number of classes, the offset accuracy, and the maximum level of division for the chrominance unit, it is possible to reduce memory usage and the processing load. Note that the maximum level of the chrominance unit can be restricted, for example, by making the maximum depth of the SAO tree structure shallower than that of the luminance unit.

[0507] Such restrictions can be realized by adaptively changing the operation of the offset information decoding unit 611 and the class classification unit 624 in accordance with the parameter condition when the parameter condition is the size of the QAOU using the configuration of the means shown in Figures 41 to 43.

[0508] In particular, when performing SAO processing on both luminance and chrominance, it is necessary to provide a temporary memory such as a line memory that holds reference pixels used for classifying each color component. Since the importance of the luminance component is lower than that of the chrominance component, the line memory for the chrominance component can be reduced by setting the parameter condition to the chrominance component and limiting the types to horizontal types EO and BO. In this case, the offset information decoding unit 611 uses conversion tables such as conversion table 801B and conversion table 802D.

[0509] It is also particularly effective to set the parameter condition to the color difference component and reduce the offset precision compared to the luminance component. For example, the offset precision (and shift value) of the luminance component is SAO_DEPTH = MIN(PIC_DEPTH,AY) Shift value = PIC_DEPTH - MIN(PIC_DEPTH, AY) In the case of the chrominance component, the offset precision and shift value are SAO_DEPTH = MIN(PIC_DEPTH, THC) Shift value = PIC_DEPTH - MIN(PIC_DEPTH, AC) When calculating the above, it is appropriate to use the variables AY and AC that control the precision, AY>AC. For example, AY=10 or 9, and AC=8. In this case, the offset precision of the luminance component is greater than that of the chrominance components. The shift value of the luminance component is smaller than that of the chrominance components.

[0510] (Appendix 4) Also, when the value of the quantization parameter qp of a CU (qp value) is equal to or greater than a threshold, the type of SAO, the number of types, the number of classes, the offset precision, and the maximum division hierarchy may be restricted. The qp value may be the initial QP value of the picture, or, when the QAOU to be processed by the SAO is along the boundary between an LCU or a CU, the qp value of the CU at the position corresponding to the upper left coordinate or center coordinate of the QAOU may be used.

[0511] The higher the qp value, the lower the image quality of the predicted image, and the more difficult it becomes to perform detailed classification and offset correction, so even if the type, number of types, number of classes, offset accuracy, and maximum division hierarchy of SAO are restricted, the effect on image quality is small. Therefore, if the number of SAO types, number of classes, offset accuracy, and maximum division hierarchy are restricted when the qp value is equal to or greater than a threshold, it is possible to reduce memory usage and processing load without affecting image quality.

[0512] Such a restriction can be realized by adaptively changing the operations of the offset information decoding unit 611 and the class classification unit 624 in accordance with the parameter condition when the parameter condition is QP, using the configuration of the means shown in FIGS.

[0513] (Appendix 5) Also, for pictures of a particular type, such as B pictures or non-reference pictures (IDR pictures), the type of SAO, the number of types, the number of classes, the offset precision, and the maximum division hierarchy may be restricted.

[0514] For I-pictures and P-pictures, the image quality of the picture has a large effect on subsequent pictures, so it is necessary to maintain high SAO accuracy. On the other hand, for other pictures, the picture quality does not have a large effect on subsequent pictures, so the type of SAO, the number of types, the number of classes, the offset accuracy, and the maximum division hierarchy may be restricted for these pictures (B-pictures, IDR pictures). This can reduce memory usage and processing load.

[0515] Such restrictions can be realized by adaptively changing the operation of the offset information decoding unit 611 and the class classification unit 624 in accordance with the parameter condition when the parameter condition is the type of picture using the configuration of the means shown in Figures 41 to 43.

[0516] (Additional Note 6) Also, the number of SAO types, the number of classes, the offset precision, and the maximum division hierarchy may be limited according to the position of the QAOU on the screen.

[0517] For example, at the edge of the screen, the number of SAO types, the number of classes, the offset precision, and the maximum level of division may be limited.

[0518] Since the center of the screen is the area that attracts the user's attention, lowering the image quality will directly result in a decrease in subjective image quality. However, even if the image quality is lowered at the periphery of the screen, the subjective image quality does not decrease as much as near the center of the screen.

[0519] Furthermore, if the SAO type, which requires sample points beyond the screen edge, is not used, the boundary determination process can be reduced.

[0520] Such restrictions can be realized by adaptively changing the operation of the offset information decoding unit 611 and the class classification unit 624 in accordance with the parameter condition when the parameter condition is the position of the QAOU using the configuration of the means shown in Figures 41 to 43.

[0521] Fourth embodiment Another embodiment of the present invention will be described below with reference to Fig. 32. This embodiment differs from the above-described embodiment in that EO and BO are switched according to the pixel value.

[0522] The inventors of the present application discovered that in pixels, EO is more effective than BO in the mid-tone range, and BO is more effective in other value ranges (low pixel value range and high pixel value range). Therefore, in this embodiment, BO is used in the low pixel value range and high pixel value range, and EO is used in the mid-tone range.

[0523] A specific description will be given with reference to Fig. 32. Fig. 32 is a diagram for explaining a configuration for switching between EO and BO depending on a pixel value, where (a) is a diagram for explaining the outline of a configuration for switching between EO and BO depending on a pixel value, (b) is a diagram for explaining specific switching values, and (c) is a diagram showing the contents of a list memory stored in offset information storage unit 621.

[0524] 32(a), in this embodiment, BO is used when the pixel value (brightness value) is near 0 and near 255, and EO is used in other value ranges. That is, the SAO type is classified according to the pixel value, and an offset according to the pixel value is added when the pixel value is near 0 and near 255, and an offset according to the edge type is added in other areas.

[0525] As a result of the experiments conducted by the inventors, it was found that EO and BO are biased in the range of pixel values ​​used. In other words, it was found that EO is more prevalent in the mid-tone range. This means that in the low and high pixel value ranges, the pixel value tends to have a greater effect on the error than the type of edge.

[0526] Therefore, in this embodiment, EO and BO are determined according to the pixel value range, and one SAO type uses the part where the error correction effect is high in both EO and BO. This improves the coding efficiency. In addition, the number of SAO types and the total number of classes can be reduced, so that the memory usage and the processing load can be reduced.

[0527] Next, the present embodiment will be described in more detail. First, unlike the above-mentioned embodiment, the present embodiment uses 1 to 4 for "sao_type_idx", which are defined as follows. "sao_type_idx" = 1: (BO_EO_0) corresponds to EO_0 + BO in the first embodiment "sao_type_idx" = 2: (BO_EO_1) corresponds to EO_1 + BO in the first embodiment "sao_type_idx" = 3: (BO_EO_2) corresponds to EO_2 + BO in the first embodiment "sao_type_idx" = 4: (BO_EO_3) corresponds to EO_3 + BO in the first embodiment Moreover, EO_0 to EO_3 and BO_0, BO_1 in the first embodiment are not used.

[0528] Then, as shown in Fig. 32(b), the classification unit 624 performs classification using BoTbl[BO_1] in the conversion table 1301 in Fig. 29. Then, if the pixel value is ¼ or less of max or ¾ or more of max, the classification unit 624 performs classification using the conversion table 1301.

[0529] If the pixel value is between 1 / 4 and 3 / 4 of max, the edge type is determined according to EO_0 to EO_3 described in the above embodiment, and classification is performed.

[0530] This allows the classifying unit 624 to classify classes by switching between BO and EO depending on pixel values.

[0531] Further, offset calculation unit 811 calculates the offset in the same manner as the offset calculation method described in the above embodiment, except that in this embodiment, the number of offset types is four, and therefore the calculation is performed with this point in mind.

[0532] In addition, the list memory 2101 stored in the offset information storage unit 621 stores the QAOU index, the offset type, and the specific value of the offset for each class selectable in the offset type, in association with each other, as shown in FIG. 32(c).

[0533] In addition, when using an offset type having both the characteristics of EO and BO, the configuration may be such that first, class classification is performed using EO, and a specific class (a class having a flat edge) is further classified using BO. Alternatively, the configuration may be such that EO and BO described in the embodiment are used together as they are. Furthermore, the configuration may be such that the number of types and classes is reduced by changing whether or not to provide an offset type having both the characteristics of EO and BO depending on the condition described in the above embodiment (whether or not the depth of the QAOU hierarchy is smaller than a threshold value).

[0534] Fifth embodiment Another embodiment of the present invention will be described below with reference to Figures 33, 34, and 42. This embodiment differs from the above-described embodiment in that the pixels used for edge determination in EO classification are limited to only pixels present in the horizontal direction of the pixel to be determined.

[0535] In the above embodiment, the pixels above and below the target pixel are also used in determining the edge, so a line buffer is required to store the pixel values ​​of the pixels above the target pixel (pixels that have already been offset).

[0536] Therefore, if the pixels used for edge determination are limited to those existing in the horizontal direction of the target pixel, the amount of memory used for the line buffer can be reduced because there is no need to refer to the upper direction.In addition, since there is no need to determine the upper boundary (such as the edge of the screen), the amount of processing can be reduced.

[0537] Furthermore, when only pixels existing in the horizontal direction are used, it is possible to reuse the results of processing up to the previous time, so the amount of processing can be further reduced. This will be explained with reference to Fig. 33(a). Fig. 33(a) shows pixels existing in the horizontal direction, and x0 to x3 are the pixel values ​​of each pixel.

[0538] First, for a pixel with a pixel value of x1, the difference between the pixel on both sides is as follows: s1=sign(x1-x0)-sign(x2-x1) Next, for the pixel with pixel value x2, the difference between the pixel on either side is calculated as follows. s2=sign(x2-x1)-sign(x3-x2) Here, s1 and s2 are values ​​used to classify the edges at x1 and x2.

[0539] In this way, when calculating s2, the sign(x2-x1) used in s1 is reused, and the amount of processing can be reduced accordingly.

[0540] Next, the details of this embodiment will be described. In this embodiment, there are two types of EO. That is, only when "sao_type_idx"=1,2, it is set as edge offset (EO). When "sao_type_idx"=1 (EO_0), the type of edge is derived by comparing with the pixels on the left and right of the target pixel (FIG. 33(b)), and when "sao_type_idx"=2 (EO'_0), the type of edge is derived by comparing with the pixels located two pixels away on the left and right of the target pixel (FIG. 33(c)).

[0541] It should be noted that the value is a band offset (BO) when "sao_type_idx"=3 or 4. This can be achieved by replacing the cases in the first embodiment where "sao_type_idx"=5 or 6 is 3 or 4, respectively.

[0542] When "sao_type_idx"=2, the type of edge is determined using pixels one pixel away from both sides of the target pixel in order to facilitate detection of edges with shallow angles relative to the horizontal. As shown in FIG. 33(d), when the pixels on both sides of the target pixel are compared, the difference in pixel values ​​may be small and the pixel may not be determined to be an edge. Even in such a case, if the difference with the pixel one pixel away is taken, it becomes possible to detect edges with shallow angles relative to the horizontal.

[0543] Also, instead of using the pixels on either side of the target pixel, the difference between the pixel on the left and the pixel one pixel further to the right (FIG. 34(a)) may be used, or vice versa (FIG. 34(b)). This is particularly effective when the reference pixel is at the edge of the screen.

[0544] Furthermore, the configuration may be such that only pixels in the horizontal direction are used are limited. For example, only in the case of a picture that has little influence on other pictures such as a B picture or a non-reference picture, pixels in the horizontal direction only are referred to, and in other cases, the configuration may be the same as that of the above-mentioned embodiment 1, or in the vicinity of a screen edge or slice boundary, pixels in the horizontal direction only are referred to, and in other cases, the configuration may be the same as that of the above-mentioned embodiment. In this way, it is possible to reduce the memory usage and the processing amount. In addition, since no reference is made in the upward direction, the boundary determination processing near the screen edge or slice boundary can also be reduced.

[0545] Alternatively, a flag may be explicitly specified for each picture or block. For example, when there are many horizontal edges (FIG. 34(c)), the performance can be prevented from deteriorating by performing the processing according to the above embodiment.

[0546] In addition, depending on the condition described in the above embodiment (whether the depth of the QAOU hierarchy is smaller than the threshold value or not), it may be changed whether or not to use only pixels existing in the horizontal direction. Furthermore, only when BO and EO are used in combination as described in the above embodiment, it may be configured to use only pixels existing in the horizontal direction for EO.

[0547] Also, the type of edge may be derived depending on whether the difference with respect to pixels existing in the horizontal direction is larger (or smaller) than a threshold value. Specifically, the classes may be classified according to the following formula.

[0548] Sign(z)=+1 (when z>th) Sign(z)=0 (when -th≦z≦th) Sign(z)=-1 (when z<-th) Here, th is a threshold value having a predetermined value.

[0549] Furthermore, in the case of pixel values ​​representing color differences, only pixels existing in the horizontal direction may be used in edge offset.

[0550] The effect of limiting to horizontal can be obtained even when there is only one horizontal edge classification method, for example, only the case of FIG. 33(b). When multiple different horizontal edge classification methods are combined as described above, it is possible to classify edges more precisely than when one horizontal edge classification method, for example, FIG. 33(b), is used. Note that the combination of multiple horizontal edge classification methods is not limited to the case of using two examples by changing the distance between the target pixel and the reference pixel as in FIG. 33(b) and (c), but may also be a configuration using two threshold values ​​used when deriving the edge type. For example, two horizontal edge classification methods may be provided, one for when the threshold value th is 0 and the other for when it is 1.

[0551] The configuration for limiting the cases where only horizontal pixels are used can be realized by adaptively changing the operations of the offset information decoding unit 611 and the class classification unit 624 in accordance with parameter conditions using the configurations of the means shown in Fig. 41 to Fig. 43 of embodiment 1. For example, in the offset information decoding unit 611 in Fig. 41, the types of offset types selected by the offset type selection unit 6112 are limited to only the horizontal edge classification method in accordance with the parameter conditions, and the offset type decoding unit 6113 decodes the limited types of offset types.

[0552] 42(b), the offset type conversion table selection unit 6117' selects a conversion table included in an edge offset type and band offset type storage unit 6118' or a conversion table included in a horizontal edge offset type and band offset type conversion table storage unit 6119' according to parameter conditions. In one example, when the color component is luminance, the conversion table included in the edge offset type and band offset type storage unit 6118' is selected, and when the color component is chrominance, the conversion table included in the horizontal edge offset type and band offset type conversion table storage unit 6119' is selected.

[0553] In the class classification unit 624, an internal offset type to be used selection unit 6112 selects an edge classification method that is limited to only the horizontal edge classification method according to the parameter conditions, and classifies the pixels in an edge offset class classification unit 6243. In one example, when the color component is luminance, the basic edge classification method is selected, and when the color component is chrominance, the horizontal edge classification method is selected.

[0554] Sixth embodiment Another embodiment of the present invention will be described below with reference to Figures 35 to 37. This embodiment differs from the above-described embodiment in that the offset precision is improved or the classes are divided finely near the center of the chrominance pixel values.

[0555] An image has a characteristic that errors near achromatic colors are easily noticeable in terms of subjective image quality. For each chrominance pixel value, the achromatic color is the center of the range, and in the case of an 8-bit bit depth, it is the color of a pixel with a pixel value of 128. Therefore, in the present embodiment, in the case of BO, the offset precision is increased or a smaller class width is used to finely divide the classes near the chrominance pixel value of 128. In the following, the range of values ​​near achromatic colors for chrominance pixel values ​​is referred to as the achromatic value range.

[0556] This makes it possible to increase the offset accuracy in the achromatic range, thereby improving the performance of offset correction in the achromatic range. This reduces the error with the original image in the achromatic range, improving the subjective image quality. Also, by dividing the classes in the achromatic range narrower than other ranges, offset correction in the achromatic range can be performed more precisely. This reduces the error with the original image, improving the subjective image quality.

[0557] In this embodiment, the offset accuracy of the achromatic color range is improved and the class division is subdivided, but the subjective image quality can be improved by subdividing the class division or improving the offset accuracy for values ​​in a range that is likely to affect the subjective image quality. Also, for luminance pixel values, the same effect can be achieved by similarly increasing the offset accuracy or subdividing the class division for pixel value ranges where image quality degradation is likely to be noticeable in terms of subjective image quality.

[0558] First, a case where the offset accuracy is improved will be described with reference to Fig. 35. Fig. 35 is a diagram showing an overview of a case where the offset accuracy is improved. As shown in Fig. 35, when the offset accuracy is improved, the offset accuracy in the vicinity of the chrominance pixel value 128 is set to high accuracy, and the offset accuracy in the other value ranges is set to low accuracy.

[0559] In addition, the offset information storage unit 621 stores n B bits (e.g. n B = 8), and for offsets in the low-precision range, n A bits (e.g. n A If there is enough storage space, n B The bits may be unified, which makes implementation easier.

[0560] In addition, the offset clip unit 812 switches the clipping range between a high-precision range of the offset and a low-precision range of the offset. nA / 2 ~2 nA / 2 -1, and for the offset high precision range, the clip range is -2 nB / 2 ~2 nB / 2 Let's say -1.

[0561] Next, a case where class classification is subdivided will be described with reference to Fig. 36. Fig. 36 is a diagram showing an overview of a case where class classification is subdivided, in which (a) shows a conversion table, and (b) to (d) are diagrams for explaining class division.

[0562] When class classification is to be subdivided, an offset storage area corresponding to "sao_type_idx"=5 is secured in the list memory for chrominance in the offset information storage unit 621 according to the number of classifications.

[0563] In addition, in the case of BO, the class classification unit 624 determines whether the pixel value of the target pixel is in the achromatic color range when deriving a class from the pixel value. Specifically, the class classification is performed using, for example, one of the conversion tables 2501 shown in FIG. 36(a).

[0564] In the conversion table 2501, class classification is performed by "class_idx = BoTbl[sao_type_idx][pic[x] / 4]", and classes are assigned to finer pixel values ​​than in the conversion tables 1301 and 1302 in FIG.

[0565] Also, "BoTbl[BO_0][pix / 4](a)" of the conversion table 2501 shows an example in which the classes are divided into 16, the achromatic value range is subdivided, and the mid-tone range (pix / 4=16-21, 42-47) is coarsened (FIG. 36(b)). Also, "BoTbl[BO_0][pix / 4](b)" shows an example in which the classes are divided into 16, the offset value range is narrowed, and the achromatic value range is subdivided while maintaining the number of classes and the maximum class width (FIG. 36(c)). Also, "BoTbl[BO_0][pix / 4](c)" shows an example in which the classes are divided into 18, the value range near the achromatic colors is subdivided, and the maximum class width is maintained (FIG. 36(d)). As another example, by making the class width of the mid-tone range relatively larger than that of the achromatic range and reducing the total number of classes, it is possible to reduce the offset storage area while maintaining the correction accuracy of the achromatic range as before. This is possible because the frequency of use of offset correction in the mid-tone range is lower than in the achromatic range.

[0566] Further, a configuration may be adopted in which both the improvement of the offset accuracy and the subdivision of the class classification in the achromatic color range are used.

[0567] In addition, the offset precision may be different between pixel values ​​indicating luminance and pixel values ​​indicating chrominance, i.e., the offset precision for pixel values ​​indicating chrominance may be coarser than the offset precision for pixel values ​​indicating luminance.

[0568] (Appendix 7) Also, the offset may be added in a direction approaching achromatic colors (=color difference pixel value 128). That is, the offset adding unit 626 may handle the offset with the opposite sign to the pixel value that is equal to or greater than 128 and the pixel value that is less than 128. For example, when the offset is a, x'=x+a(x<128), x'=xa(x≧128) may be used.

[0569] Furthermore, if the pixel value after adding the offset crosses 128, it may be configured to clip at 128. For example, when the pixel value is "126 (on the green side)" and the offset is "3", instead of adding the offset as is and getting x=126+3=129, it may be configured as x=clip(126+3,0,128)=128. Here, clip(x,y,z) indicates the process of limiting the value of x to y≦x≦z.

[0570] Also, for example, when the pixel value is “129 (red side)” and the offset is “−4”, instead of simply adding the offset and setting x=129+(−4)=125, it is also possible to set x=clip(129+(−4),128,255)=128.

[0571] (Appendix 8) Also, the offset may be added separately for the mid-tone range and other value ranges in the color difference. For example, as shown in FIG. 37(a), class classification may be performed taking into account achromatic pixel values. In the example shown in FIG. 37(a), when "sao_type_idx"=5 (BO_0), a class is assigned to the mid-tone range, and when "sao_type_idx"=6 (BO_1), a class is assigned to value ranges other than the mid-tone range.

[0572] Also, as shown in Fig. 37(b), the classes may be classified asymmetrically into two ranges on either side of the achromatic pixel values. In the example shown in Fig. 37(b), the class width is the same as in the example shown in Fig. 37(a), but the position of the separator is changed.

[0573] Also, as shown in Fig. 37(c) and (d), different class classifications may be used for each chrominance channel (Cr and Cb). In the example shown in Fig. 37(c) and (d), the class width is different between chrominance (Cr) and chrominance (Cb). The class width can be changed by subdividing the number of classes as in the conversion table 2501.

[0574] Also, as shown in FIG. 37(e), for chrominance, a configuration may be adopted in which class classification is performed with only one BO type.

[0575] Seventh embodiment Another embodiment of the present invention will be described below with reference to Figures 38 to 40. This embodiment differs from the above-mentioned embodiment in that, instead of the offset value itself, the offset value is predictively coded, that is, an offset residual calculated using the offset value and a predicted value of the offset value is coded, and "0" is set as the offset predicted value.

[0576] In general, in SAO, the offset is coded according to the class indicating the area classification. If there are no pixels classified into the class (if the class is empty), "0" is coded. Normally, the offsets of neighboring classes and neighboring areas have similar values, so if the offset value that has already been decoded is used as a predicted value and the difference is coded, it is possible to reduce the amount of coding. However, if the class is empty, the amount of coding increases.

[0577] Therefore, in this embodiment, the predicted value of the offset includes "0", one predicted value is selected from multiple predicted value candidates including "0", and the prediction difference is encoded to reduce the amount of coding for the offset.

[0578] This allows an appropriate prediction value to be assigned even if the class is empty, even if there are pixels classified into that class, and the offset is coded, so that the amount of coding can be reduced by predictive coding.

[0579] The configuration of an offset information decoding unit 611' according to this embodiment will be described with reference to Fig. 38. The offset information decoding unit 611' is provided in place of the offset information decoding unit 611 in Fig. 22, and includes an offset residual decoding unit 651, an offset restoration unit 652, and a predicted value derivation unit 653. The predicted value derivation unit 653 includes a predicted candidate flag decoding unit 661, a fixed predicted value calculation unit 662, an encoded predicted value calculation unit 663, and a predicted value candidate selection unit 664.

[0580] The offset residual decoding unit 651 decodes the offset residual from the QAOU information included in the coded data #1, and supplies the decoded offset residual to the offset restoration unit 652.

[0581] The prediction value derivation unit 653 derives a prediction value of the offset. The prediction candidate flag decoding unit 661 decodes the prediction candidate flag from the QAOU information, and supplies the decoded prediction candidate flag to the prediction value candidate selection unit 664. Note that the prediction candidate flag decoding unit 661 may be configured not to decode the prediction candidate flag when the previously decoded offset value is "0".

[0582] The fixed predicted value calculation unit 662 calculates a fixed value (here, “0”) that is an offset to be coded when there is no pixel classified into the class, as a predicted value, and supplies the calculated value to the predicted value candidate selection unit 664 .

[0583] The encoding predicted value calculation unit 663 reads out the already decoded offset from the offset information storage unit 621 , calculates a predicted value, and supplies the calculated value to a predicted value candidate selection unit 664 .

[0584] The prediction value candidate selection unit 664 selects the prediction value candidate from the fixed prediction value calculation unit 662 according to the prediction candidate flag. The prediction value to be supplied to the offset restoration unit 652 is selected from the prediction value supplied thereto and the prediction value supplied from the encoding prediction value calculation unit 663 .

[0585] The offset restoration unit 652 restores the offset (Offset) from the predicted value (pred) supplied from the predicted value candidate selection unit 664 and the offset residual (sao_offset) supplied from the offset residual decoding unit 651 in accordance with the following equation. “Offset”=“pred”+“sao_offset” Here, the reason why the amount of code increases when there is a class with no pixels to be classified (empty class) will be described with reference to FIG. 39. FIG. 39 is a diagram showing an overview of the case when there is an empty class. As shown in FIG. 39, for a non-empty class, the offset of the class is encoded. Then, for each class, the previous offset is used as a predicted value and the difference between the predicted value and the offset is encoded. However, if there is an empty class, the predicted value becomes "0" and the difference value becomes large. This increases the amount of code.

[0586] Therefore, in this embodiment, the prediction candidate flag allows the prediction value to be selected from "0" and "the immediately preceding offset (other than 0)."

[0587] The syntax when a prediction candidate flag is used will be described with reference to Fig. 40. Fig. 40 is a diagram showing syntax 2901 when a prediction candidate flag is used. As shown in Fig. 40, a prediction candidate flag "sao_pred_flag" is added, and if "sao_pred_flag"=0, the offset decoded immediately before (not 0) is set as a predicted value offsetp, and if "sao_pred_flag"=1 or the first offset, the predicted value offsetp=0. In the syntax 2901, "sao_offset_delta" represents a difference d from the predicted value, and "sao_offset[sao_curr_depth][ys][xs]"="offsetp +d".

[0588] Depending on the offset type, the prediction candidate flag may not be used. For example, in the case of EO, the prediction candidate flag may not be used, and in the case of BO, the prediction candidate flag may be used. This is because in BO, empty classes often occur depending on the distribution of pixel values.

[0589] Alternatively, a configuration may be used in which it is determined whether or not to perform predictive coding depending on the offset type. For example, a configuration may be used in which predictive coding is performed in the case of EO and not in the case of BO.

[0590] (Appendix 9) Furthermore, in the above third to seventh embodiments, the intra prediction mode may be used to estimate, restrict, or rearrange the edge offset class for each LCU.

[0591] The edge direction selected as the EO type of SAO is considered to be correlated with the intra prediction mode. Therefore, the intra prediction mode of the CU at the position corresponding to the QAOU of SAO can be used to select the EO class.

[0592] For example, the EO class can be estimated and determined according to the intra prediction mode, candidates for the EO class can be restricted, or the order (index) of the EO classes can be rearranged in order of the likelihood of being selected.

[0593] This can be easily achieved particularly when the size of the block QAOU in each layer of the SAO is the same as the size of the CU, for example, when the image is divided so that the maximum size of the QAOU in the SAO is equal to the LCU size.

[0594] (Appendix 10) In conventional adaptive offset filters, since there are many types (types, classes) of offsets, there is a possibility that a large memory size is required. Therefore, in this embodiment, an image filter device or the like is realized that can reduce block distortion while suppressing an increase in memory size.

[0595] The image filter device of the present invention is an image filter device that adds an offset selected from a plurality of offsets to each pixel value of an input image composed of a plurality of unit areas, and comprises an offset determination means that determines, for each unit area, an offset to be added to the pixel value of a pixel included in the unit area, and a filter means that adds the offset determined by the offset determination means to the pixel value of the pixel included in the unit area, wherein, when the size of the unit area for which the offset is to be determined is smaller than a predetermined size, the offset determination means determines the offset to be added to the pixel value of the pixel included in the unit area from among offsets with a more limited number of selectable offsets than when the size of the unit area is equal to or larger than the predetermined size.

[0596] According to the above configuration, when the size of the unit area for which an offset is to be determined is smaller than a predetermined size, the offset to be added is determined from among offsets that are more limited than when the size of the unit area is equal to or larger than the predetermined size.

[0597] When the size of a unit area is small, the number of pixels contained therein is small, and the pixels are likely to have similar values. Therefore, when the size of a unit area is small, even if the number of selectable offsets is limited, the effect on the image after the offsets are applied is small. Furthermore, by limiting the number of selectable offsets, the required memory capacity can be reduced.

[0598] Therefore, according to the above configuration, it is possible to reduce memory usage while reducing the influence on the image after the offset is applied. Also, by reducing the number of selectable offsets, the amount of coding can be reduced, and coding efficiency can be improved.

[0599] In order to solve the above problems, the image filter device of the present invention is an image filter device that adds an offset to each pixel value of an input image composed of a plurality of unit areas, and includes an offset determination means that determines, for each unit area, an offset to be added to the pixel value of a pixel included in the unit area, and a filter means that adds the offset determined by the offset determination means to the pixel value of the pixel included in the unit area, and is characterized in that when the size of the unit area for which the offset is to be determined is smaller than a predetermined size, the filter means adds an offset with lower accuracy than when the size of the unit area is equal to or larger than the predetermined size.

[0600] According to the above configuration, when the size of the unit area for which an offset is to be determined is smaller than a predetermined size, an offset with lower accuracy is added than when the size of the unit area is equal to or larger than the predetermined size.

[0601] When the size of a unit area is small, the number of pixels contained therein is small, and the pixels are likely to have similar values. Therefore, when the size of a unit area is small, even if the offset precision is made coarse, the effect on quantization error is small. Therefore, the effect on the image after the offset is applied is also small. Furthermore, by making the offset precision coarse, the required memory capacity can be reduced.

[0602] Therefore, according to the above configuration, it is possible to reduce the memory usage while reducing the influence on the image after the offset is applied. Also, by reducing the accuracy of the offset, the amount of code can be reduced and the coding efficiency can be improved.

[0603] The image filter device of the present invention is an image filter device that applies an adaptive offset (SAO: Sample Adaptive Offset) to each pixel value of an input image composed of a plurality of unit areas, and is equipped with an offset determination means that determines, for each unit area, a type of offset to be added to the pixel values ​​of pixels included in each unit area, and a filter means that adds an offset corresponding to the type of offset determined by the offset determination means to the pixel values ​​of the pixels included in the unit area, wherein the offset determination means determines the type of offset to be band offset (BO) when the pixel value of the target pixel to which the offset is to be added is near the maximum and minimum values, and determines the type of offset to be edge offset (EO) for pixels in other value ranges.

[0604] According to the above configuration, when adaptive offsets are applied, a band offset is applied to pixels whose pixel values ​​are near the maximum and minimum values, and an edge offset is applied to pixels in other value ranges.

[0605] In the high and low pixel value ranges, pixel values ​​tend to be more susceptible to errors than edges, so the above configuration can increase the efficiency of error correction and improve coding efficiency.

[0606] Furthermore, by using both band offset and edge offset in one type, the number of types of adaptive offset can be reduced, leading to a reduction in the amount of memory used and the amount of processing.

[0607] In the above configuration, the band offset refers to an offset process in which one of a plurality of offsets is added to the pixel value of a pixel to be processed depending on the magnitude of the pixel value of the pixel to be processed (same below). Also, in the above configuration, the edge offset refers to an offset process in which one of a plurality of offsets is added to the pixel value of a pixel to be processed depending on the difference between the pixel value of the pixel to be processed and the pixel values ​​of the pixels surrounding the pixel to be processed (same below).

[0608] In the image filter device of the present invention, the offset determination means may determine a band offset for pixels whose pixel values ​​are in the ranges from the minimum value to one-quarter of the maximum value, and from three-quarters of the maximum value to the maximum value, and may determine an edge offset for other value ranges.

[0609] According to the above configuration, it is possible to clearly distinguish between pixels to which a band offset is applied and pixels to which an edge offset is applied.

[0610] The image filter device according to the present invention is an image filter device that applies an adaptive offset (SAO: Sample Adaptive Offset) to an input image, and is characterized in that it comprises: a class classification means that performs edge judgment for determining a class when applying an edge offset (EO) by referring only to pixels existing in the horizontal direction of a target pixel; and a filter means that, when applying an edge offset (EO), adds an offset corresponding to the class classified by the class classification means.

[0611] According to the above configuration, since only pixels in the horizontal direction are referenced to determine edges, the amount of memory required can be reduced compared to when pixels in the upper direction are also referenced. Also, since boundary determination in the upper direction is not required, the amount of processing can be reduced.

[0612] In the image filtering device according to the present invention, the classifying means may determine an edge by referring to a pixel located two pixels away from the target pixel in the horizontal direction.

[0613] According to the above configuration, a pixel two pixels away from the target pixel is referenced, so that an edge can be detected even if the edge angle is shallow.

[0614] The image filter device according to the present invention is an image filter device that applies an adaptive offset (SAO: Sample Adaptive Offset) to an input image, and is characterized by comprising: a class classification means that divides the division width of a class near a median value, which is the median value between the maximum and minimum values ​​of a pixel value indicating color difference, more finely than other value ranges to determine a class when applying a band offset (BO); and a filter means that, when applying a band offset (BO), adds an offset corresponding to the class classified by the class classification means.

[0615] According to the above configuration, the division width of the class near the median, which is the center value between the maximum and minimum values ​​of the pixel value indicating the color difference, is made finer than other value ranges, and the class is determined when applying the band offset (BO).

[0616] When the pixel value of the color difference is the median, the pixel is achromatic. Achromatic errors are easily noticeable to humans and degrade the subjective image quality. Therefore, by subdividing the class width near the median as in the above configuration, it is possible to set fine offsets for pixels near the median. This improves the subjective image quality.

[0617] In the image filtering device according to the present invention, when the median value exists between the pixel value before the offset addition and the pixel value after the offset addition, the filtering means may set the pixel value after the offset addition to the median value.

[0618] According to the above configuration, the offset is not added beyond the median. A pixel with a median color difference pixel value is an achromatic color, and the color perceived by the human eye changes on both sides of the median. Therefore, according to the above configuration, it is possible to prevent the color perceived by the human eye from changing due to the addition of the offset.

[0619] The image filter device according to the present invention is an image filter device which applies an adaptive offset (SAO: Sample Adaptive Offset) to an input image, and is characterized by including a filter means which adds an offset with higher precision to pixels whose pixel values ​​indicating color difference are in the vicinity of a median value, which is the median value between the maximum and minimum values, than to pixels in other value ranges.

[0620] According to the above configuration, an offset with higher accuracy is added to pixels whose pixel values ​​indicating color difference are in the vicinity of a median value, which is the median value between the maximum and minimum values, than to pixels in other value ranges.

[0621] When the pixel value of the color difference is the median, the pixel is achromatic. Achromatic errors are easily noticeable to humans and degrade the subjective image quality. Therefore, as in the above configuration, by improving the accuracy of the offset to be added near the median, it is possible to add a fine offset to pixels near the median. This improves the subjective image quality.

[0622] The offset decoding device according to the present invention is an offset decoding device that decodes each offset referenced by an image filter that adds an offset to each pixel value of an input image, and is characterized in that it comprises an offset residual decoding means that decodes each offset residual from encoded data, a predicted value derivation means that derives a predicted value of each offset from an offset that has already been decoded or a predetermined value, and an offset calculation means that calculates each offset from the predicted value derived by the predicted value derivation means and the offset residual decoded by the offset residual decoding means.

[0623] According to the above configuration, since the offset is decoded from the residual, the amount of code can be reduced compared to the case where the offset is encoded as it is. Also, since the predicted value for determining the residual is derived from the decoded offset or a predetermined value, by using only the decoded offset, the amount of code for the difference data can be prevented from becoming larger than that when the difference data is encoded as it is.

[0624] The predetermined value may be, for example, "0."

[0625] The offset coding device according to the present invention is an offset coding device that codes each offset referenced by an image filter that adds an offset to each pixel value of an input image, and is characterized in that it comprises a predicted value derivation means that derives a predicted value of each offset from an already-encoded offset or a predetermined value, an offset residual calculation means that calculates an offset residual from each offset and the predicted value derived by the predicted value derivation means, and an offset residual coding means that codes the offset residual calculated by the offset residual calculation means.

[0626] According to the above configuration, since the offset is decoded from the residual, the amount of code can be reduced compared to the case where the offset is encoded as it is. Also, since the predicted value for determining the residual is derived from the decoded offset or a predetermined value, by using only the decoded offset, the amount of code for the difference data can be prevented from becoming larger than that when the difference data is encoded as it is.

[0627] The data structure of encoded data according to the present invention is a data structure of encoded data referenced by an image filter that adds an offset to each pixel value of an input image composed of a plurality of unit regions, and includes prediction value derivation information indicating whether a prediction value is to be derived from a decoded offset or a predetermined value, and the image filter derives a prediction value and decodes the offset by referring to the prediction value derivation information included in the encoded data.

[0628] According to the above configuration, it is possible to determine whether the predicted value is to be a decoded offset or a predetermined value, based on the predicted value derivation information.

[0629] (Application example) The above-mentioned video decoding device 1 (1') and video encoding device 2 (2') can be mounted and used in various devices that transmit, receive, record, and play videos. Note that the video may be a natural video captured by a camera or the like, or an artificial video (including CG and GUI) generated by a computer or the like.

[0630] First, it will be described with reference to FIG. 45 that the above-mentioned video decoding device 1 and video encoding device 2 can be used for transmitting and receiving videos.

[0631] Fig. 45(a) is a block diagram showing the configuration of a transmitting device A equipped with a video coding device 2. As shown in Fig. 45(a), the transmitting device A includes an encoding unit A1 that obtains encoded data by encoding a video, a modulation unit A2 that obtains a modulated signal by modulating a carrier wave with the encoded data obtained by the encoding unit A1, and a transmitting unit A3 that transmits the modulated signal obtained by the modulation unit A2. The above-mentioned video coding device 2 is used as this encoding unit A1.

[0632] The transmitting device A may further include a camera A4 for capturing moving images, a recording medium A5 for recording moving images, an input terminal A6 for inputting moving images from the outside, and an image processing unit A7 for generating or processing images, as a supply source of moving images to be input to the encoding unit A1. In Fig. 45(a), a configuration in which the transmitting device A includes all of these components is illustrated, but some of them may be omitted.

[0633] The recording medium A5 may record unencoded video, or may record video encoded by a recording encoding method different from the transmission encoding method. In the latter case, a decoding unit (not shown) that decodes the encoded data read from the recording medium A5 according to the recording encoding method may be provided between the recording medium A5 and the encoding unit A1.

[0634] Fig. 45(b) is a block diagram showing the configuration of a receiving device B equipped with a video decoding device 1. As shown in Fig. 45(b), the receiving device B includes a receiving unit B1 that receives a modulated signal, a demodulating unit B2 that obtains coded data by demodulating the modulated signal received by the receiving unit B1, and a decoding unit B3 that obtains a video by decoding the coded data obtained by the demodulating unit B2. The above-mentioned video decoding device 1 is used as this decoding unit B3.

[0635] The receiving device B may further include a display B4 for displaying the moving image, a recording medium B5 for recording the moving image, and an output terminal B6 for outputting the moving image to the outside, as destinations of the moving image output by the decoding unit B3. Although Fig. 45(b) shows an example of a configuration in which the receiving device B includes all of these, some of them may be omitted.

[0636] The recording medium B5 may be for recording unencoded video, or may be encoded using an encoding method for recording that is different from the encoding method for transmission. In the latter case, it is preferable to interpose an encoding unit (not shown) between the decoding unit B3 and the recording medium B5, which encodes the video acquired from the decoding unit B3 according to the encoding method for recording.

[0637] The transmission medium for transmitting the modulated signal may be wireless or wired. The transmission mode for transmitting the modulated signal may be broadcast (here, this refers to a transmission mode in which the destination is not specified in advance) or communication (here, this refers to a transmission mode in which the destination is specified in advance). In other words, the transmission of the modulated signal may be realized by any of wireless broadcast, wired broadcast, wireless communication, and wired communication.

[0638] For example, a broadcasting station (such as broadcasting equipment) / receiving station (such as a television receiver) for terrestrial digital broadcasting is an example of a transmitting device A / receiving device B that transmits and receives modulated signals by wireless broadcasting. Also, a broadcasting station (such as broadcasting equipment) / receiving station (such as a television receiver) for cable television broadcasting is an example of a transmitting device A / receiving device B that transmits and receives modulated signals by cable broadcasting.

[0639] Also, a server (such as a workstation) / client (such as a television receiver, personal computer, smartphone, etc.) of a VOD (Video On Demand) service or video sharing service using the Internet is an example of a transmitter A / receiver B that transmits and receives modulated signals by communication (usually, in a LAN, either wireless or wired is used as the transmission medium, and in a WAN, wired is used as the transmission medium). Here, personal computers include desktop PCs, laptop PCs, and tablet PCs. Smartphones also include multi-function mobile phone terminals.

[0640] The client of the video hosting service has a function to decode the encoded data downloaded from the server and display it on a display, as well as a function to encode the video captured by the camera and upload it to the server. In other words, the client of the video hosting service functions as both the transmitting device A and the receiving device B.

[0641] Next, it will be described with reference to FIG. 46 that the above-mentioned video decoding device 1 and video encoding device 2 can be used for recording and reproducing videos.

[0642] Fig. 46(a) is a block diagram showing the configuration of a recording device C equipped with the above-mentioned video encoding device 2. As shown in Fig. 46(a), the recording device C includes an encoding unit C1 that obtains encoded data by encoding a video, and a writing unit C2 that writes the encoded data obtained by the encoding unit C1 onto a recording medium M. The above-mentioned video encoding device 2 is used as this encoding unit C1.

[0643] The recording medium M may be (1) a type built into the recording device C, such as an HDD (Hard Disk Drive) or SSD (Solid State Drive), (2) a type connected to the recording device C, such as an SD memory card or USB (Universal Serial Bus) flash memory, or (3) a type loaded into a drive device (not shown) built into the recording device C, such as a DVD (Digital Versatile Disc) or BD (Blu-ray Disc: registered trademark).

[0644] The recording device C may further include a camera C3 for capturing moving images, an input terminal C4 for inputting moving images from the outside, a receiving unit C5 for receiving moving images, and an image processing unit C6 for generating or processing images, as a supply source of moving images to be input to the encoding unit C1. In Fig. 46(a), a configuration in which the recording device C includes all of these components is illustrated, but some of them may be omitted.

[0645] The receiving unit C5 may receive unencoded video, or may receive encoded data encoded by an encoding method for transmission that is different from the encoding method for recording. In the latter case, a transmission decoding unit (not shown) that decodes the encoded data encoded by the encoding method for transmission may be interposed between the receiving unit C5 and the encoding unit C1.

[0646] Examples of such a recording device C include a DVD recorder, a BD recorder, and an HD (Hard Disk) recorder (in this case, the input terminal C4 or the receiving unit C5 is the main source of the moving image). Other examples of such a recording device C include a camcorder (in this case, the camera C3 is the main source of the moving image), a personal computer (in this case, the receiving unit C5 or the image processing unit C6 is the main source of the moving image), and a smartphone (in this case, the camera C3 or the receiving unit C5 is the main source of the moving image).

[0647] Fig. 46(b) is a block diagram showing the configuration of a playback device D equipped with the above-mentioned video decoding device 1. As shown in Fig. 46(b), the playback device D includes a reading unit D1 that reads out coded data written to a recording medium M, and a decoding unit D2 that obtains video by decoding the coded data read by the reading unit D1. The above-mentioned video decoding device 1 is used as this decoding unit D2.

[0648] The recording medium M may be (1) a type that is built into the playback device D, such as an HDD or SSD, (2) a type that is connected to the playback device D, such as an SD memory card or USB flash memory, or (3) a type that is loaded into a drive device (not shown) built into the playback device D, such as a DVD or BD.

[0649] Moreover, the playback device D may further include, as destinations of the video output by the decoding unit D2, a display D3 for displaying the video, an output terminal D4 for outputting the video to the outside, and a transmission unit D5 for transmitting the video. Although Fig. 46(b) illustrates a configuration in which the playback device D includes all of these components, some of them may be omitted.

[0650] The transmission unit D5 may transmit unencoded video, or may transmit encoded data encoded by an encoding method for transmission different from the encoding method for recording. In the latter case, it is preferable to interpose an encoding unit (not shown) between the decoding unit D2 and the transmission unit D5, which encodes the video by the encoding method for transmission.

[0651] Examples of such a playback device D include a DVD player, a BD player, and an HDD player (in this case, the output terminal D4 to which a television receiver or the like is connected is the main destination of the video). Other examples of such a playback device D include a television receiver (in this case, the display D3 is the main destination of the video), a desktop PC (in this case, the output terminal D4 or the transmission unit D5 is the main destination of the video), a laptop or tablet PC (in this case, the display D3 or the transmission unit D5 is the main destination of the video), a smartphone (in this case, the display D3 or the transmission unit D5 is the main destination of the video), and a digital signage (also called an electronic billboard or electronic bulletin board, and the display D3 or the transmission unit D5 is the main destination of the video).

[0652] (Software configuration) Finally, each block of the video decoding device 1 (1') and the video encoding device 2 (2'), in particular the variable length code decoding unit 13, the motion vector restoration unit 14, the inter prediction image generation unit 16, the intra prediction image generation unit 17, the prediction method determination unit 18, the inverse quantization and inverse transform unit 19, the deblocking filter 41, the adaptive filter 50, the adaptive offset filter 60 (60'), the transform and quantization unit 21, the variable length code encoding unit 22, the inverse quantization and inverse transform unit 23, the intra prediction image generation unit 25, the inter prediction image generation unit 26, the motion vector detection unit 27, the prediction method control unit 28, the motion vector redundancy elimination unit 29, the deblocking filter 33, the adaptive filter 70, and the adaptive offset filter 80 (80'), may be realized in hardware by a logic circuit formed on an integrated circuit (IC chip), or in software by using a CPU (central processing unit).

[0653] In the latter case, the video decoding device 1 and the video encoding device 2 each include a CPU that executes instructions of a control program that realizes each function, a ROM (read only memory) that stores the program, a RAM (random access memory) that expands the program, and a storage device (recording medium) such as a memory that stores the program and various data. The object of the present invention can also be achieved by supplying a recording medium on which the program code (executable program, intermediate code program, source program) of the control program of the video decoding device 1 and the video encoding device 2, which is software that realizes the above-mentioned functions, is recorded in a computer-readable manner to the video decoding device 1 and the video encoding device 2, and having the computer (or a CPU or MPU (micro processing unit)) read and execute the program code recorded on the recording medium.

[0654] Examples of the recording medium that can be used include tapes such as magnetic tape and cassette tape, magnetic disks such as floppy disks (registered trademark) and hard disks, disks including optical disks such as CD-ROM (compact disc read-only memory), MO disks (magneto-optical disc), MD (Mini Disc), DVD (digital versatile disc), and CD-R (CD Recordable), cards such as IC cards (including memory cards) and optical cards, semiconductor memo...

Claims

1. an offset attribute setting unit that refers to the encoded data and sets an offset value range and a shift value; an offset decoding unit that decodes an offset limited to the set offset value range; a filter processing unit that shifts the offset to the left in accordance with the shift value and adds the left-shifted offset to each pixel value of an input image composed of a plurality of unit areas.

2. the encoded data includes at least a bit depth of pixel values ​​of the input image; 2. The image filtering device according to claim 1, wherein the offset attribute setting section sets the offset value range in accordance with a bit depth of the pixel value.

3. The offset attribute setting unit sets the offset bit depth to a value equal to the bit depth of the pixel value when the bit depth of the pixel value is 10 or less, and sets the offset bit depth to 10 when the bit depth of the pixel value is 11 or more. The offset attribute setting unit sets the offset bit depth to a value equal to the bit depth of the pixel value when the bit depth of the pixel value is 11 or more, and sets the offset bit depth to 10 when the maximum value of the offset value range is -2 （オフセットビット深度－Ｋ－１） From 2 （オフセットビット深度－Ｋ－１） 3. The image filter device according to claim 2, wherein the range is set to −1.

4. 4. The image filter device according to claim 3, wherein the value of K is four.

5. 5. The image filter device according to claim 3, wherein the shift value is set to (a bit depth of a pixel value - (the smaller of the bit depth of a pixel value and 10)).

6. A step of setting an offset value range and a shift value by referring to the encoded data; decoding an offset restricted to the set offset value range; and shifting the offset to the left and adding the left-shifted offset to each pixel value of an input image composed of a plurality of unit areas.

7. the encoded data includes at least a bit depth of pixel values ​​of the input image; 7. The method of claim 6, wherein the offset range is set according to a bit depth of the pixel values.

8. The offset bit depth is set to a value equal to the bit depth of the pixel value if the bit depth of the pixel value is 10 or less, and is set to 10 if the bit depth of the pixel value is 11 or more. The maximum value of the offset value range is −2. （オフセットビット深度－Ｋ－１） From 2 （オフセットビット深度－Ｋ－１） 8. The method of claim 7, wherein the filter is set to be in the range of -1.

9. 9. The method of claim 8, wherein the value of K is four.

10. 10. The filtering method according to claim 8 or 9, wherein the shift value is set to (bit depth of pixel value - (the lesser of bit depth of pixel value and 10)).

11. an offset attribute setting unit that refers to the encoded data and sets an offset value range and a shift value; an offset decoding unit that decodes an offset limited to the set offset value range; a filter processing unit that shifts the offset to the left in accordance with the shift value and adds the left-shifted offset to each pixel value of an input image composed of a plurality of unit areas.

12. the encoded data includes at least a bit depth of pixel values ​​of the input image; The video decoding device according to claim 11, wherein the offset attribute setting unit sets the offset value range in accordance with a bit depth of the pixel value.

13. The offset attribute setting unit sets the offset bit depth to a value equal to the bit depth of the pixel value when the bit depth of the pixel value is 10 or less, and sets the offset bit depth to 10 when the bit depth of the pixel value is 11 or more. The offset attribute setting unit sets the offset bit depth to a value equal to the bit depth of the pixel value when the bit depth of the pixel value is 11 or more, and sets the offset bit depth to 10 when the maximum value of the offset value range is -2 （オフセットビット深度－Ｋ－１） From 2 （オフセットビット深度－Ｋ－１） 13. The video decoding device according to claim 12, wherein the range is set to −1.

14. 14. The video decoding device according to claim 13, wherein the value of K is four.

15. The video decoding device according to claim 13 or 14, wherein the shift value is set to (a bit depth of a pixel value - (the smaller of the bit depth of the pixel value and 10)).

Images