Image processing apparatus and image processing method

By setting the color difference offset parameter set and applying a deblocking filter, the problem of adjusting the filter intensity of brightness components and color difference components in VVC technology is solved, the image quality is improved, and more efficient image encoding and decoding is achieved.

CN118945341APending Publication Date: 2024-11-12SONY GROUP CORP
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411221272.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2019-09-06
Filing Date
2020-09-03
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

In VVC technology, it is difficult to adjust the filter intensity between the brightness component and the color difference component, which makes it difficult to improve the image quality of the decoded image.

Method used

By setting the chromatic aberration offset parameter set, a deblocking filter is applied to the chromatic aberration component of the pixels near the block boundary of the decoded image to generate a bitstream including the chromatic aberration offset parameter set, thereby adjusting the filter intensity.

Benefits of technology

Improve the image quality of the decoded image, solve the problem of adjusting the filter intensity between the brightness component and the color difference component, and enhance the efficiency of image encoding and decoding.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118945341A_ABST
    Figure CN118945341A_ABST
Patent Text Reader

Abstract

The invention relates to an image processing apparatus and an image processing method. An image processing apparatus includes: a parsing unit that parses a plurality of groups of picture unit offset parameter sets according to a bitstream including, as the plurality of groups of picture unit offset parameter sets, a color difference offset parameter set that is set as a picture parameter set and a color difference offset parameter set that is set as a picture header; a decoding unit that decodes the bitstream to generate a decoded image; and a filter unit that applies a deblocking filter to a color difference component of a pixel in the vicinity of a block boundary of the decoded image generated by the decoding unit, using the plurality of sets of picture unit offset parameter sets analyzed by the analysis unit.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] This application is a divisional application of the application with national application number 202080060686.7, international application date September 3, 2020, national entry date February 25, 2022, and invention name “Image processing device and image processing method”. Technical Field

[0002] The present technology relates to an image processing device and an image processing method, and more particularly, to an image processing device and an image processing method capable of improving image quality, for example. Background Art

[0003] JVET (Joint Video Experts Team), a joint standardization organization of ITU-T and ISO / IEC, is standardizing VVC (Versatile Video Coding) as a next-generation image coding method with the aim of further improving coding efficiency compared to H.265 / HEVC.

[0004] Regarding tc and beta which are filter parameters of DF (deblocking filter), for example, techniques for setting one set of offset parameters slice_tc_offset_div2 and slice_beta_offset_div2, respectively, in units of slices are described in NPL 1 and NPL 2.

[0005] According to the techniques described in NPL1 and NPL2, filter parameters tc and beta may be adjusted by offset parameters slice_tc_offset_div2 and slice_beta_offset_div2 in units of slices, DF may be controlled using the adjusted filter parameters, and DF may be applied.

[0006] Citation List

[0007] Non-patent literature

[0008] NPL1: Benjamin Bross, Jianle Chen, Shan Liu, Versatile Video Coding (Draft6), JVET-O2001-v14 (version 14, date 2019-07-31)

[0009] NPL2: Jianle Chen, Yan Ye, Seung Hwan Kim, Algorithm description for Versatile Video Coding and Test Model 6 (VTM 6), JVET-O2002-v1 (version 1, date 2019-08-15) Summary of the invention

[0010] Technical issues

[0011] In the techniques described in NPL1 and NPL2, when a portion of a slice where it is desired to enhance the DF filter strength to suppress block noise and a portion of a slice where it is desired to weaken the DF filter strength to maintain details of an original image to be encoded have different luminance components and different chrominance components, it is difficult to adjust the filter strength between the luminance component and the chrominance component. Therefore, it is difficult to improve the (subjective) image quality of the decoded image.

[0012] The present technology has been proposed in view of such circumstances, and can improve image quality.

[0013] Solution to the problem

[0014] The present technology relates to an image processing device and an image processing method capable of improving image quality. A color difference offset parameter set used when a deblocking filter is applied to a color difference component of a decoded image obtained by decoding a bit stream is set. An image is encoded to generate a bit stream including the color difference offset parameter set. The bit stream is decoded to generate a decoded image. A deblocking filter is applied to the color difference component of pixels near a block boundary of the decoded image using the color difference offset parameter set. The present technology can be applied when encoding and decoding an image.

[0015] The first image processing device of the present technology is an image processing device including the following units: a decoding unit, which decodes a bit stream to generate a decoded image; and a filter unit, which applies a deblocking filter to the color difference component of pixels near the block boundary of the decoded image generated by the decoding unit using a color difference offset parameter set, and the color difference offset parameter set is used when applying the deblocking filter to the color difference component of the decoded image generated by the decoding unit.

[0016] The first image processing method of the present technology is an image processing method comprising the following steps: a decoding step of decoding a bit stream to generate a decoded image; and a filtering step of applying a deblocking filter to a color difference component of pixels near a block boundary of the decoded image generated by the decoding step using a color difference offset parameter set, the color difference offset parameter set being used when applying the deblocking filter to the color difference component of the decoded image generated by the decoding step.

[0017] In the first image processing device and the first image processing method of the present technology, a bit stream is decoded to generate a decoded image. A deblocking filter is applied to the color difference component of pixels near a block boundary of the decoded image using a color difference offset parameter set, which is used when the deblocking filter is applied to the color difference component of the decoded image.

[0018] The second image processing device for pixel data is an image processing device including the following units: a setting unit that sets a color difference offset parameter set, which is used when a deblocking filter is applied to a color difference component of a decoded image obtained by decoding a bit stream; and an encoding unit that encodes the image to generate a bit stream including the color difference offset parameter set set by the setting unit.

[0019] The second image processing method of the present technology is an image processing method comprising the following steps: a setting step of setting a color difference offset parameter set, the color difference offset parameter set being used when a deblocking filter is applied to a color difference component of a decoded image obtained by decoding a bit stream; and an encoding step of encoding the image to generate a bit stream including the color difference offset parameter set set by the setting step.

[0020] In the second image processing device and image processing method of the present technology, a color difference offset parameter set used when a deblocking filter is applied to a color difference component of a decoded image obtained by decoding a bit stream is set. The image is encoded to generate a bit stream including the color difference offset parameter set.

[0021] According to an embodiment of the present disclosure, an image processing device is provided, comprising: a parsing unit, which parses multiple groups of picture unit offset parameter sets based on a bit stream including a color difference offset parameter set set as a picture parameter set and a color difference offset parameter set set as a picture header as multiple groups of picture unit offset parameter sets; a decoding unit, which decodes the bit stream to generate a decoded image; and a filter unit, which applies a deblocking filter to color difference components of pixels near block boundaries of the decoded image generated by the decoding unit using the multiple groups of picture unit offset parameter sets parsed by the parsing unit.

[0022] According to an embodiment of the present disclosure, there is provided an image processing method, comprising: a parsing step of parsing multiple groups of picture unit offset parameter sets according to a bit stream including a color difference offset parameter set set as a picture parameter set and a color difference offset parameter set set as a picture header as multiple groups of picture unit offset parameter sets; a decoding step of decoding the bit stream to generate a decoded image; and a filtering step of applying a deblocking filter to color difference components of pixels near block boundaries of the decoded image generated by the decoding step using the multiple groups of picture unit offset parameter sets parsed by the parsing step.

[0023] According to an embodiment of the present disclosure, an image processing device is provided, comprising: a setting unit, which sets multiple groups of color difference offset parameter sets as picture unit offset parameter sets; and an encoding unit, which encodes an image to generate a bit stream including the picture unit offset parameter set set by the setting unit as a picture parameter set and a picture header.

[0024] According to an embodiment of the present disclosure, an image processing method is provided, comprising: a setting step of setting multiple groups of color difference offset parameter sets as picture unit offset parameter sets; and an encoding step of encoding an image to generate a bit stream including the picture unit offset parameter set set by the setting step as a picture parameter set and a picture header.

[0025] The image processing device may be an independent device or an internal block constituting one device.

[0026] The image processing apparatus can be realized by causing a computer to execute the program. The program can be provided by transmitting via a transmission medium or by recording on a recording medium. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] [ Figure 1 ] Figure 1 is a diagram showing an example of a picture as an image to be encoded (original image).

[0028] [ Figure 2 ] Figure 2 is a diagram showing the relationship between the quantization step size and the filter parameters tc and beta of DF.

[0029] [ Figure 3 ] Figure 3 is a block diagram showing a configuration example of an embodiment of an image processing system to which the present technology is applied.

[0030] [ Figure 4 ] Figure 4 is a block diagram showing a detailed configuration example of the encoder 11.

[0031] [ Figure 5 ] Figure 5 is a flowchart showing an example of encoding processing by the encoder 11 .

[0032] [ Figure 6 ] Figure 6 is a block diagram showing a detailed configuration example of the decoder 51 .

[0033] [ Figure 7 ] Figure 7 : is a flowchart showing an example of the decoding process of the decoder 51 .

[0034] [ Figure 8 ] Figure 8 is a block diagram showing a configuration example of the DF 31a.

[0035] [ Fig. 9 ] Fig. 9 is a block diagram showing a configuration example of the DF 31b.

[0036] [ Fig.10 ] Fig.10is a diagram showing an example of the syntax of a picture parameter set when two sets of offset parameter sets are included in the picture parameter set.

[0037] [ Fig.11 ] Fig.11 is a diagram showing an example of the syntax of a slice header when two sets of offset parameter sets are included in the slice header.

[0038] [ Fig.12 ] Fig.12 is a block diagram illustrating an example of the configuration of one embodiment of a computer to which the present technology is applied. DETAILED DESCRIPTION

[0039] The scope disclosed in this specification is not limited to the content of the implementation method. The disclosures in the following references REF 1 to REF 6, which are already known at the time of filing this application, are also incorporated into this specification by reference. In other words, the disclosures in the following references REF 1 to REF 6 are also used as a basis for determining the support requirements. For example, even if the quadtree block structure, OTBT (quadtree plus binary tree), block structure, and MTT (multi-type tree) block structure are not directly defined in the specific implementation of the present invention, these structures are also considered to be included in the scope of the present disclosure and meet the support requirements of the claims. For example, the same applies to technical terms such as parsing, grammar, and semantics. Even if these technical terms are not directly defined in the specific implementation of the present invention, these technical terms are also considered to be included in the scope of the present disclosure and meet the support requirements of the claims. Similarly, the same applies to technical terms such as deblocking filters and offset parameters (tc, beta). Even if these technical terms are not directly defined in the specific implementation of the present invention, these technical terms are also considered to be included in the scope of the present disclosure and meet the support requirements of the claims.

[0040] REF1: Recommendation ITU-T H.264 (04 / 2017) “Advanced video coding for generic audio visual services”, April 2017

[0041] REF2: Recommendation ITU-T H.265 (02 / 2018) “High efficiency video coding”, February 2018

[0042] REF3: Benjamin Bross, Jianle Chen, Shan Liu, Versatile Video Coding (Draft 6), JVET-O2001-v14 (version 14, date 2019-07-31)

[0043] REF4: Jianle Chen, Yan Ye, Seung Hwan Kim, Algorithm description for Versatile Video Coding and Test Model 6 (VTM 6), JVET-O2002-v1 (version 1, date 2019-08-15)

[0044] REF5: Geert Van der Auwera, Rajan Joshi, Marta Karczewicz: AHG6: Quantization Matrices and Deblocking Filtering, JCTVC-I0280_r1 (Version 2, 2012-04-25)

[0045] REF6: Kazushi Sato: Title: On Deblocking Filter and DC Component of Quantization Matrices, JCTVC-J0186_r1 (version 2, date 2012-07-11)

[0046] <Definition>

[0047] In the present disclosure, the following terms are defined as follows.

[0048] The original image means the original image before encoding.

[0049] The decoded image means an image generated in a decoding process. The decoded image includes an image generated in the middle of a decoding process and an image output after the decoding process is performed.

[0050] The local decoded image means an image generated in local decoding when encoding an image, and includes an image generated midway in local decoding and an image output after local decoding. Hereinafter, the local decoded image is also simply referred to as a decoded image.

[0051] The encoded data means data after encoding an image (texture).

[0052] A bit stream (or coded bit stream, or coded stream) means data including coded data and data in which parameters required for encoding or decoding are encoded.

[0053] Parameter (coding parameter) is a general term for data required for encoding or decoding, and is generally a syntax of a bit stream, a parameter set, etc. Parameter (coding parameter) includes variables used in a derivation process, etc.

[0054] The filter parameters are parameters used when controlling the filter. For example, for the deblocking filter, tc and beta described in REF3 / REF4 correspond to the filter parameters.

[0055] The offset parameter is an offset for adjusting a filter parameter used when controlling the filter. For example, for a deblocking filter, tc_offset and beta_offset described in REF3 / REF4 correspond to the offset parameter.

[0056] An offset parameter set means a group (set) of offset parameters, and corresponds to, for example, a set of (tc_offset, beta_offset).

[0057] A plurality of offset parameter sets correspond to (tc_offset0, beta_offset0), (tc_offset1, beta_offset1), ..., (tc_offset#N, beta_offset#N). For example, two sets of offset parameters correspond to (tc_offset0, beta_offset0) and (tc_offset1, beta_offset1).

[0058] A block means a block (a group of pixels (values)) which is a processing unit when encoding processing is performed.

[0059] The prediction block means a block as a processing unit when intra prediction or inter prediction is performed, and sub blocks are included in the prediction block.

[0060] When a prediction block is unified with a processing unit such as an orthogonal transform block which is a processing unit when performing an orthogonal transform and a coding block which is a processing unit when performing an encoding process, the prediction block (e.g., PU), the orthogonal transform block (e.g., TU), and the coding block (e.g., CU) mean the same block.

[0061] Block boundary means the boundary of a block.

[0062] The pixels located near the block boundary mean pixels located within a range where a deblocking filter is applied to the block boundary.

[0063] In the present disclosure, identification data for identifying a plurality of modes may be set as syntax of a bit stream. In this case, the decoder may perform processing more efficiently by parsing and referring to the identification data.

[0064] <Example of original image>

[0065] Figure 1 is a diagram showing an example of a picture as an image to be encoded (original image).

[0066] Various subjects are captured in the pictures. Figure 1 The picture in shows the flames and smoke of the fire, people, flags, lawn, etc. that make up a campfire scene.

[0067] In a decoded image obtained by encoding and decoding pictures capturing various subjects, portions where block boundaries are obvious and portions where block boundaries are less obvious are mixed according to features of the subject captured in the block, block size, and the like.

[0068] From the perspective of improving the image quality of the decoded image, it is desirable to apply a DF having a relatively strong filtering strength (smoothing degree) to a portion where the block boundary is obvious, and to apply a DF having a relatively weak filtering strength to a portion where the block boundary is less obvious.

[0069] Therefore, in the decoded image, a portion where it is desired to apply a DF having a relatively strong filtering strength and a portion where it is desired to apply a DF having a weak filtering strength are mixed.

[0070] For example, in Figure 1 , the portion where the smoke is captured and the flat portion where the flag is captured correspond to the portion where it is desired to apply a DF with a strong filtering strength.

[0071] On the other hand, the portion where the face of a person is captured and the portion where the texture of grass is captured correspond to portions where the block size may be set small and application of a DF with a weak filtering strength is desired.

[0072] Generally, the larger the quantization parameter, the larger the block size of the (encoding) block tends to be. For blocks with larger block sizes, the block boundary of two adjacent blocks is easily noticeable and DF needs to be applied.

[0073] In VVC, a long-tap filter with a stronger filtering strength is used as one of the DFs. When the conditions for applying the long-tap filter are met, the long-tap filter is applied to a block with a larger block size of 32 or more luminance component pixels or 8 or more color difference component pixels. Therefore, it may be necessary to apply a DF with a stronger filtering strength to a block with a larger block size.

[0074] In order to improve the (subjective) image quality, it is conceivable, for example, to set an offset parameter of the DF that aggressively applies a long tap filter.

[0075] However, the offset parameter is reflected in the DF filter parameters tc and beta in units of slices. Therefore, if there are blocks with relatively small block sizes in a slice, and it is not desirable to actively apply DF in addition to blocks with large block sizes, DF may be applied to blocks with small block sizes as well as blocks with large block sizes. In this case, the decoded image becomes blurred and it is difficult to improve the image quality.

[0076] On the other hand, when the offset parameter is not applied, it is difficult to apply DF to blocks with large block sizes in a slice, and block noise becomes noticeable.

[0077] <Relationship between quantization step size and filter parameters tc and beta>

[0078] Figure 2 is a diagram showing the relationship between the quantization step size and the filter parameters tc and beta of DF.

[0079] When the quantization matrix is ​​applied in image coding, the quantization step size is adjusted. The DF filter parameters tc and beta of VVC are derived based on the average of the quantization parameters QPp and QPq of two adjacent blocks ((QPp+QPq+1)>>1). Therefore, the adjustment of the quantization step size by the quantization matrix is ​​not reflected in the filter parameters tc and beta.

[0080] Figure 2 Neighboring block B1 is shown as two neighboring blocks in which the basic QP (QPp and QPq) is 17, scalingList[] corresponding to the quantization matrix is, for example, 16 as the default scaling, and the quantization step size Qstep is 20.

[0081] in addition, Figure 2 A neighboring block B2 is shown as two neighboring blocks in which the basic QP is 23, the scalingList[] corresponding to the quantization matrix is ​​16 as the default scaling, and the quantization step size Qstep is 40.

[0082] exist Figure 2 In the example, the filter parameters tc and beta (t') of the DF applied to the block boundary (the pixels near it) of the adjacent block B1 are c and β') are 0 and 7 respectively. The DF filter parameters tc and beta applied to the block boundary of the adjacent block B2 are 5 and 13 respectively.

[0083] Now, if scalingList[] corresponding to the quantization matrix is ​​changed from 16 to 32 as the default scaling of the neighboring block B1, the quantization step size Qstep is adjusted to 40, which is the same as the quantization step size Qstep of the neighboring block B2.

[0084] However, after adjusting the quantization step size Qstep to 40, the filter parameters tc and beta applied to the block boundary of the adjacent block B1 remain at 0 and 7, respectively.

[0085] As described above, in the cases of basic QP=23&&default scaling list (scalingList[]=16) and basic QP=17&&scalingList[]=32, the quantization step size is the same, but the derived DF filter parameters may be significantly different.

[0086] That is, for the neighboring block B1 with a basic QP of 17 and a scalingList[] of 32, and the neighboring block B2 with a basic QP of 23 and a scalingList[] of 16 as default scaling, the quantization step size Qstep is the same 40.

[0087] However, the filter parameters tc and beta of DF applied to the block boundary of the neighboring block B1 having a basic QP of 17 and scalingList[] of 32 are 0 and 7, respectively. The filter parameters tc and beta of DF applied to the block boundary of the neighboring block B2 having a basic QP of 23 and scalingList[] of 16 as the default scaling are 5 and 13, respectively. Therefore, the filter parameters tc and beta of DF are significantly different in the neighboring blocks B1 and B2 having the same quantization step size Qstep of 40.

[0088] References REF5 and REF6 proposed techniques for adjusting quantization parameters, but they were not adopted due to increased processing complexity. A simple solution is needed to solve the problem that the adjustment of the quantization step size by the quantization matrix is ​​not reflected in the filter parameters tc and beta.

[0089] The filter parameters tc and beta can be adjusted using the offset parameter. Considering the trend of applying DF to larger blocks, it is required to adjust the filter parameters tc and beta in the offset parameter only for larger blocks (blocks with larger block sizes).

[0090] When the current set of offset parameters is consistently applied to slices, DF (DF with stronger filtering strength) is strongly applied to relatively small blocks (blocks with small block size), and the image may become blurred.

[0091] On the other hand, if no offset parameter is applied, block noise will be evident in larger blocks.

[0092] Therefore, using the current set of offset parameters, that is, using only one set of offset parameters for adjusting the filter parameter tc and the offset parameters for adjusting the filter parameter beta (offset parameter set), it may be difficult to improve the image quality as a whole in a slice where larger blocks and smaller blocks are mixed.

[0093] Therefore, in the present technology, when adjusting the filter parameters tc and beta of DF, multiple sets of offset parameter sets can be set in the processing unit (filter processing unit) (hereinafter, also referred to as the adjustment unit) in which the filter parameters tc and beta are to be adjusted. The set multiple sets of offset parameter sets can be set as the syntax of the bit stream in which the image is encoded.

[0094] As an adjustment unit for adjusting the filter parameters tc and beta by a plurality of offset parameter sets, for example, a picture unit or a slice unit may be used.

[0095] When a picture unit is used as an adjustment unit, multiple groups of offset parameter sets may be set in a picture parameter set or a picture header.

[0096] When a slice unit is used as an adjustment unit, multiple groups of offset parameter sets may be set in a slice header.

[0097] When a picture unit is used as an adjustment unit, multiple sets of offset parameter sets may be set in both a picture parameter set and a slice. When multiple sets of offset parameter sets are set for both a picture parameter set and a slice, if the offset parameters of the picture parameter set and the slice are different, the multiple sets of offset parameter sets set in the picture parameter set may be preferably used to adjust the filter parameters tc and beta.

[0098] As the number of groups of the offset parameter sets to be set in the adjustment unit, any number of groups may be used, such as 2, 3, 4 or more. When two groups of offset parameter sets are used as the number of groups of the offset parameter sets to be set in the adjustment unit, processing may be simplified and the size of the offset parameter sets may be reduced.

[0099] A current offset parameter set, ie, an offset parameter for a block boundary to which DF is applied, may be selected from a plurality of offset parameter sets according to encoding parameters.

[0100] When selecting the current offset parameter set according to the encoding parameter, a threshold value of the encoding parameter may be set as a selection criterion for selecting the current offset parameter set according to the encoding parameter.

[0101] As an encoding parameter for selecting the current offset parameter set, for example, a block size may be used.

[0102] In this case, for example, a first offset parameter set for adjusting filter parameters tc and beta of DF applied to block boundaries of blocks whose block size is equal to or greater than a threshold, and a second offset parameter set for adjusting filter parameters tc and beta of DF applied to block boundaries of blocks whose block size is not equal to or greater than the threshold may be set.

[0103] For blocks whose block size is equal to or greater than the threshold, the first offset parameter set may be selected as the current offset parameter set. For blocks whose block size is not equal to or greater than the threshold, the second offset parameter set may be selected as the current offset parameter set.

[0104] The first offset parameter set, the second offset parameter set and the threshold of the block size may be set in the syntax of the bitstream.

[0105] As a plurality of groups of offset parameter sets, for example, an offset parameter set for each of a luminance component and a color difference component of a decoded image may be set.

[0106] When there are multiple color difference components, one set of offset parameter sets may be set commonly for the multiple color difference components, or one set of offset parameter sets may be set individually for each of the multiple color difference components.

[0107] When an offset parameter set is set for each of a luminance component and a color difference component of a decoded image, if one set of offset parameter sets is set for each of a plurality of color difference components, one set of offset parameter sets is set for each component of the decoded image.

[0108] For example, when the decoded image has YCbCr components (when the color space of the decoded image is YCbCr), one set of offset parameter sets may be set for each component of YCbCr (i.e., each of the luminance component Y, the color difference component Cb (Cb component), and the color difference component Cr (Cr component)). In this case, three sets of offset parameter sets are set.

[0109] In addition, for example, when the decoded image has a component of YUV (when the color space of the decoded image is YUV), a set of offset parameter sets may be set for each component of YUV (ie, each of the luminance component Y, the color difference component U (U component), and the color difference component V (V component)). In this case, three sets of offset parameter sets are set.

[0110] When a total of two sets of offset parameter sets are set, one set of offset parameter sets is used for the luminance component, and one set of offset parameter sets is used for the chrominance component. In the application of DF, the offset parameter set of the luminance component is used as the current offset parameter set of the luminance component from the two sets of offset parameter sets, and the offset parameter set of the chrominance component is used as the current offset parameter set of the chrominance component.

[0111] In the application of DF, when a total of three sets of offset parameter sets are set, one set of offset parameter sets (luminance offset parameter set) is used for the luminance component Y, one set of offset parameter sets (first color difference offset parameter set) is used for the color difference component (first color difference component) Cb or U, and one set of offset parameter sets (second color difference offset parameter set) is used for the color difference component (second color difference component) Cr or V, the offset parameter set of the luminance component Y is used as the current offset parameter set of the luminance component Y from the three sets of offset parameter sets. The offset parameter set of the color difference component Cb or U is used as the current offset parameter set of the color difference component Cb or U, and the offset parameter set of the color difference component Cr or V is used as the current offset parameter set of the color difference component Cr or V.

[0112] In the application of DF, when a total of two sets of offset parameter sets are used, one set of offset parameter sets is used for the luminance component and one set of offset parameter sets is used for the chrominance component, and when the part where the DF filtering strength is desired to be enhanced to suppress block noise and the part where the DF filtering strength is desired to be weakened to maintain the details of the original image to be encoded have different luminance components and different chrominance components, the filtering strength between the luminance component and the chrominance component can be adjusted and the image quality of the decoded image can be improved.

[0113] In the application of DF, when a total of three sets of offset parameter sets are used, one set of offset parameter sets is used for the luminance component Y, one set of offset parameter sets is used for the color difference component Cb or U, and one set of offset parameter sets is used for the color difference component Cr or V, and when the part where the DF filtering strength is desired to be enhanced to suppress block noise and the part where the DF filtering strength is desired to be weakened to maintain the details of the original image to be encoded have different luminance components and different color difference components, the filtering strength between the components can be adjusted and the image quality of the decoded image can be improved.

[0114] <Image processing system using this technology>

[0115] Figure 3 is a block diagram showing a configuration example of an embodiment of an image processing system to which the present technology is applied.

[0116] The image processing system 10 has an image processing device as an encoder 11 and an image processing device as a decoder 51 .

[0117] The encoder 11 encodes the original image to be encoded supplied thereto, and outputs an encoded bit stream obtained by the encoding. The encoded bit stream is supplied to the decoder 51 via a recording medium or a transmission medium (not shown).

[0118] The decoder 51 decodes the encoded bit stream supplied thereto, and outputs a decoded image obtained by the decoding.

[0119] <Configuration Example of Encoder 11>

[0120] Figure 4 It is shown Figure 3 A block diagram of a detailed configuration example of the encoder 11 is shown.

[0121] In the block diagrams described below, in order to avoid complicating the diagrams, description of lines that provide information (data) necessary for the processing of each block is appropriately omitted.

[0122] exist Figure 4 In the embodiment, the encoder 11 has an A / D conversion unit 21, a screen rearrangement buffer 22, a calculation unit 23, an orthogonal transform unit 24, a quantization unit 25, a lossless encoding unit 26, and an accumulation buffer 27. The encoder 11 includes an inverse quantization unit 28, an inverse orthogonal transform unit 29, a calculation unit 30, a frame memory 32, a selection unit 33, an intra prediction unit 34, a motion prediction / compensation unit 35, a prediction image selection unit 36, and a rate control unit 37. The encoder 11 has a DF 31a, an adaptive offset filter 41, and an ALF (Adaptive Loop Filter) 42.

[0123] The A / D conversion unit 21 A / D converts the original image (encoding target) of the analog signal into the original image of the digital signal, supplies the original image of the digital signal to the screen rearrangement buffer 22, and stores the original image of the digital signal therein. When the original image of the digital signal is supplied to the encoder 11, the encoder 11 may be configured not to provide the A / D conversion unit 21.

[0124] The screen rearrangement buffer 22 rearranges frames of the original image from the display order in the encoding (decoding) order according to GOP (Group of Picture), and supplies the original image to the calculation unit 23 , the intra prediction unit 34 , and the motion prediction / compensation unit 35 .

[0125] The calculation unit 23 subtracts the predicted image supplied from the intra prediction unit 34 or the motion prediction / compensation unit 35 via the predicted image selection unit 36 ​​from the original image from the screen rearrangement buffer 22 , and supplies the residual (prediction residual) obtained by the subtraction to the orthogonal transformation unit 24 .

[0126] The orthogonal transform unit 24 performs an orthogonal transform such as a discrete cosine transform or a Karhunen-Loeve transform on the residual supplied from the calculation unit 23 , and supplies an orthogonal transform coefficient obtained by the orthogonal transform to the quantization unit 25 .

[0127] The quantization unit 25 quantizes the orthogonal transform coefficient supplied from the orthogonal transform unit 24. The quantization unit 25 sets a quantization parameter based on a target value of a code amount (target code amount value) supplied from the rate control unit 37, and performs quantization of the orthogonal transform coefficient. The quantization unit 25 supplies the encoded data, which is the quantized orthogonal transform coefficient, to the lossless encoding unit 26.

[0128] The lossless encoding unit 26 encodes the quantized orthogonal transform coefficient, which is the encoded data from the quantization unit 25 , by a predetermined lossless encoding method.

[0129] The lossless encoding unit 26 acquires encoding information necessary for decoding by the decoding device 170 from each block in the encoding information related to the prediction encoding by the encoder 11 .

[0130] Here, examples of encoding information include prediction modes of intra-frame prediction or inter-frame prediction, motion information such as motion vectors, target code values, quantization parameters, picture types (I, P, B), and encoding parameters such as filter parameters of DF 31a and adaptive offset filter 41.

[0131] The prediction mode may be acquired from the intra prediction unit 34 and the motion prediction / compensation unit 35. The motion information may be acquired from the motion prediction / compensation unit 35. The filter parameters of the DF 31a and the adaptive offset filter 41 may be acquired from the DF 31a and the adaptive offset filter 41, respectively.

[0132] The lossless coding unit 26 encodes the coding information by a variable length coding or arithmetic coding method such as CAVLC (context adaptive variable length coding) or CABAC (context adaptive binary arithmetic coding) and other lossless coding methods to generate a coded bit stream including (multiplexing) the encoded coding information and the coded data from the quantization unit 25, and provides the coded bit stream to the accumulation buffer 27.

[0133] The accumulation buffer 27 temporarily accumulates the encoded bit stream supplied from the lossless encoding unit 26. The encoded bit stream accumulated in the accumulation buffer 27 is read out and transmitted at a predetermined timing.

[0134] The encoded data of the orthogonal transform coefficient quantized in the quantization unit 25 is supplied to the inverse quantization unit 28 and the lossless encoding unit 26. The inverse quantization unit 28 performs inverse quantization on the quantized orthogonal transform coefficient by a method corresponding to the quantization of the quantization unit 25, and supplies the orthogonal transform coefficient obtained by the inverse quantization to the inverse orthogonal transform unit 29.

[0135] The inverse orthogonal transform unit 29 performs inverse orthogonal transform on the orthogonal transform coefficient supplied from the inverse quantization unit 28 by a method corresponding to the orthogonal transform process performed by the orthogonal transform unit 24 , and supplies a residual obtained as a result of the inverse orthogonal transform to the calculation unit 30 .

[0136] The calculation unit 30 adds the predicted image provided from the intra prediction unit 34 or the motion prediction / compensation unit 35 via the predicted image selection unit 36 ​​and the residual provided from the inverse orthogonal transform unit 29, and thereby obtains and outputs (a part of) a decoded image obtained by decoding the original image.

[0137] The decoded image output by the calculation unit 30 is supplied to the DF 31 a or the frame memory 32 .

[0138] The frame memory 32 temporarily stores the decoded image supplied from the calculation unit 30 and the decoded image (filtered image) supplied from the ALF 42 and to which the DF 31a, the adaptive offset filter 41, and the ALF 42 are applied. The decoded image stored in the frame memory 32 is supplied to the selection unit 33 as a reference image for generating a predicted image at a necessary timing.

[0139] The selection unit 33 selects a supply destination of the reference image supplied from the frame memory 32. When the intra prediction unit 34 performs intra prediction, the selection unit 33 supplies the reference image supplied from the frame memory 32 to the intra prediction unit 34. When the motion prediction / compensation unit 35 performs inter prediction, the selection unit 33 supplies the reference image supplied from the frame memory 32 to the motion prediction / compensation unit 35.

[0140] The intra prediction unit 34 performs intra prediction (intra-screen prediction) using the original image supplied from the screen rearrangement buffer 22 and the reference image supplied from the frame memory 32 via the selection unit 33. The intra prediction unit 34 selects an optimal intra prediction mode based on a predetermined cost function (e.g., RD ​​cost, etc.), and supplies a predicted image generated from the reference image in the optimal intra prediction mode to the predicted image selection unit 36. In addition, as described above, the intra prediction unit 34 appropriately supplies the prediction mode indicating the intra prediction mode selected based on the cost function to the lossless encoding unit 26 and the like.

[0141] The motion prediction / compensation unit 35 performs motion prediction (inter-frame prediction) using the original image supplied from the screen rearrangement buffer 22 and the reference image supplied from the frame memory 32 via the selection unit 33. The motion prediction / compensation unit 35 performs motion compensation according to the motion vector detected by the motion prediction, and generates a predicted image. The motion prediction / compensation unit 35 performs inter-frame prediction in a plurality of inter-frame prediction modes prepared in advance, and generates a predicted image from the reference image.

[0142] The motion prediction / compensation unit 35 selects an optimal inter prediction mode based on a predetermined cost function of a predicted image obtained for each of a plurality of inter prediction modes. The motion prediction / compensation unit 35 supplies the predicted image generated in the optimal inter prediction mode to the predicted image selection unit 36 ​​.

[0143] The motion prediction / compensation unit 35 supplies the prediction mode indicating the inter prediction mode selected based on the cost function and motion information such as a motion vector required for decoding the encoded data encoded in the inter prediction mode to the lossless encoding unit 26 .

[0144] The predicted image selection unit 36 ​​selects a supply source of the predicted image to be supplied to the computing unit 23 and the computing unit 30 from the intra prediction unit 34 and the motion prediction / compensation unit 35 , and supplies the predicted image supplied from the selected supply source to the computing unit 23 and the computing unit 30 .

[0145] The rate control unit 37 controls the rate of the quantization operation in the quantization unit 25 so that overflow or underflow does not occur based on the code amount of the encoded bit stream accumulated in the accumulation buffer 27. That is, the rate control unit 37 sets the target code amount of the encoded bit stream and provides the target code amount to the quantization unit 25 so that overflow and underflow of the accumulation buffer 27 do not occur.

[0146] The DF 31 a applies DF to the decoded image from the calculation unit 30 as necessary, and supplies the decoded image to which DF is applied (filtered image) or the decoded image to which DF is not applied to the adaptive offset filter 41 .

[0147] The adaptive offset filter 41 applies the adaptive offset filter to the decoded image from the DF 31 a as necessary, and supplies the decoded image (filtered image) to which the adaptive offset filter is applied or the decoded image to which the adaptive offset filter is not applied to the ALF 42 .

[0148] The ALF 42 applies ALF to the decoded image from the adaptive offset filter 41 as necessary, and supplies the decoded image to which ALF is applied or the decoded image to which ALF is not applied to the frame memory 32 .

[0149] In the encoder 11 configured as described above, the calculation unit 23, the orthogonal transform unit 24, the quantization unit 25, and the lossless encoding unit 26 function as an encoding unit that encodes an image using a filtered image to generate an encoded bit stream.

[0150] <Encoding Processing>

[0151] Figure 5 It is shown Figure 4 A flowchart of an example of encoding processing of encoder 11.

[0152] Figure 5 The order of steps in the encoding process shown is for convenience of explanation, and the steps in the actual encoding process are appropriately executed in parallel in the necessary order. This also applies to the processing described later.

[0153] In the encoder 11 , in step S11 , the A / D conversion unit 21 A / D-converts the original image and supplies the converted original image to the screen rearrangement buffer 22 , and the process proceeds to step S12 .

[0154] In step S12 , the screen rearrangement buffer 22 accumulates the original images from the A / D conversion unit 21 , and rearranges and outputs the original images in the encoding order, and the process proceeds to step S13 .

[0155] In step S13, the intra prediction unit 34 performs intra prediction processing in intra prediction mode, and the process proceeds to step S14. In step S14, the motion prediction / compensation unit 35 performs inter motion prediction processing for performing motion prediction and motion compensation in inter prediction mode, and the process proceeds to step S15.

[0156] In the intra prediction process of the intra prediction unit 34 and the inter motion prediction process of the motion prediction / compensation unit 35 , cost functions of various prediction modes are calculated, and a predicted image is generated.

[0157] In step S15, the predicted image selection unit 36 ​​determines the optimal prediction mode based on the cost function obtained by the intra prediction unit 34 and the motion prediction / compensation unit 35. The predicted image selection unit 36 ​​selects and outputs the predicted image in the optimal prediction mode among the predicted image generated by the intra prediction unit 34 and the predicted image generated by the motion prediction / compensation unit 35, and the process proceeds from step S15 to step S16.

[0158] In step S16, the calculation unit 23 calculates the residual between the target image to be encoded (which is the original image output by the picture rearrangement buffer 22) and the predicted image output by the predicted image selection unit 36, and provides the residual to the orthogonal transformation unit 24, and the processing proceeds to step S17.

[0159] In step S17 , the orthogonal transform unit 24 performs an orthogonal transform on the residual from the calculation unit 23 , supplies the resultant orthogonal transform coefficient to the quantization unit 25 , and the process proceeds to step S18 .

[0160] In step S18 , the quantization unit 25 quantizes the orthogonal transform coefficient from the orthogonal transform unit 24 , and supplies the quantized coefficient obtained by the quantization to the lossless encoding unit 26 and the inverse quantization unit 28 , and the process proceeds to step S19 .

[0161] In step S19, the inverse quantization unit 28 inversely quantizes the quantization coefficient from the quantization unit 25, and supplies the obtained orthogonal transformation coefficient to the inverse orthogonal transformation unit 29, and the process proceeds to step S20. In step S20, the inverse orthogonal transformation unit 29 performs an inverse orthogonal transformation on the orthogonal transformation coefficient from the inverse quantization unit 28, and supplies the obtained residual to the calculation unit 30, and the process proceeds to step S21.

[0162] In step S21, the calculation unit 30 adds the residual from the inverse orthogonal transform unit 29 and the predicted image output by the predicted image selection unit 36, and generates a decoded image corresponding to the original image which is the target of the residual calculation in the calculation unit 23. The calculation unit 30 supplies the decoded image to the DF 31a, and the process proceeds from step S21 to step S22.

[0163] In step S22, the DF 31a applies DF to the decoded image from the calculation unit 30 and supplies the resulting filtered image to the adaptive offset filter 41, and the process proceeds to step S23.

[0164] In step S23 , the adaptive offset filter 41 applies the adaptive offset filter to the filtered image from the DF 31 a and supplies the resulting filtered image to the ALF 42 , and the process proceeds to step S24 .

[0165] In step S24 , the ALF 42 applies ALF to the filtered image from the adaptive offset filter 41 and supplies the resulting filtered image to the frame memory 32 , and the process proceeds to step S25 .

[0166] In step S25, the frame memory 32 stores the filtered image supplied from the ALF 42, and the process proceeds to step S26. The filtered image stored in the frame memory 32 is used as a reference image which is a source from which a predicted image is generated in steps S13 and S14.

[0167] In step S26, the lossless encoding unit 26 encodes the encoded data, which is the quantization coefficient from the quantization unit 25, and generates an encoded bit stream including the encoded data. The lossless encoding unit 26 encodes the following items as necessary, and includes them in the encoded bit stream: a quantization parameter used for quantization in the quantization unit 25, a prediction mode obtained by the intra-frame prediction process in the intra-frame prediction unit 34, a prediction mode and motion information obtained in the inter-frame motion prediction process in the motion prediction / compensation unit 35, and encoded information such as filter parameters of the DF 31a and the adaptive offset filter 41.

[0168] The lossless encoding unit 26 supplies the encoded bit stream to the accumulation buffer 27, and the process proceeds from step S26 to step S27.

[0169] In step S27, the accumulation buffer 27 accumulates the encoded bit stream from the lossless encoding unit 26, and the process proceeds to step S28. The encoded bit stream accumulated in the accumulation buffer 27 is appropriately read out and transmitted.

[0170] In step S28, the rate control unit 37 controls the rate of the quantization operation in the quantization unit 25 based on the code amount (occurrence code amount) of the encoded bit stream accumulated in the accumulation buffer 27 so that overflow or underflow does not occur, and the encoding process ends.

[0171] <Configuration Example of Decoder 51>

[0172] Figure 6 It is shown Figure 3 A block diagram of a detailed configuration example of the decoder 51 is shown.

[0173] exist Figure 6 In the embodiment, the decoder 51 includes an accumulation buffer 61, a lossless decoding unit 62, an inverse quantization unit 63, an inverse orthogonal transform unit 64, a calculation unit 65, a picture rearrangement buffer 67, and a D / A conversion unit 68. The decoder 51 includes a frame memory 69, a selection unit 70, an intra prediction unit 71, a motion prediction / compensation unit 72, and a selection unit 73. The decoder 51 has a DF 31b, an adaptive offset filter 81, and an ALF 82.

[0174] The accumulation buffer 61 temporarily accumulates the encoded bit stream transmitted from the encoder 11 , and supplies the encoded bit stream to the lossless decoding unit 62 at a predetermined timing.

[0175] The lossless decoding unit 62 receives the encoded bit stream from the accumulation buffer 61 and Figure 4 The coded bit stream is decoded using a method corresponding to the coding method of the lossless coding unit 26.

[0176] The lossless decoding unit 62 supplies the quantized coefficients which are the encoded data included in the decoding result of the encoded bit stream to the inverse quantization unit 63 .

[0177] The lossless decoding unit 62 has a function of performing parsing. The lossless decoding unit 62 parses necessary encoding information included in the decoding result of the encoded bit stream, and provides the encoding information to the intra prediction unit 71, the motion prediction / compensation unit 72, the DF 31b, the adaptive offset filter 81 and other necessary blocks.

[0178] The inverse quantization unit 63 is connected with Figure 4 The quantized coefficients as the encoded data from the lossless decoding unit 62 are inversely quantized by a method corresponding to the quantization method of the quantization unit 25 in , and the orthogonal transformation coefficients obtained by the inverse quantization are supplied to the inverse orthogonal transformation unit 64 .

[0179] The inverse orthogonal transform unit 64 is connected with Figure 4 The orthogonal transform unit 24 performs inverse orthogonal transform on the orthogonal transform coefficient supplied from the inverse quantization unit 63 by a method corresponding to the orthogonal transform method of the orthogonal transform unit 24, and supplies the obtained residual to the calculation unit 65.

[0180] The calculation unit 65 is supplied with the residual from the inverse orthogonal transform unit 64 , and the predicted image from the intra prediction unit 71 or the motion prediction / compensation unit 72 via the selection unit 73 .

[0181] The calculation unit 65 adds the residual from the inverse orthogonal transform unit 64 and the predicted image from the selection unit 73 to generate a decoded image, and supplies the decoded image to the DF 31b.

[0182] The screen rearrangement buffer 67 temporarily stores the decoded image supplied from the ALF 82 , rearranges the arrangement of frames (pictures) of the decoded image from the encoding (decoding) order to the display order, and supplies the decoded image to the D / A conversion unit 68 .

[0183] The D / A conversion unit 68 performs D / A conversion on the decoded image supplied from the screen rearrangement buffer 67, and outputs the D / A-converted decoded image to a display (not shown) for display. When the device connected to the decoder 51 receives an image of a digital signal, the decoder 51 may be configured not to provide the D / A conversion unit 68.

[0184] The frame memory 69 temporarily stores the decoded image supplied from the ALF 82. The frame memory 69 supplies the decoded image to the selection unit 70 as a reference image for generating a predicted image at a predetermined timing or based on an external request of the intra prediction unit 71, the motion prediction / compensation unit 72, or the like.

[0185] The selection unit 70 selects a supply destination of the reference image supplied from the frame memory 69. When decoding an image encoded by intra prediction, the selection unit 70 supplies the reference image supplied from the frame memory 69 to the intra prediction unit 71. When decoding an image encoded by inter prediction, the selection unit 70 supplies the reference image supplied from the frame memory 69 to the motion prediction / compensation unit 72.

[0186] The intra prediction unit 71 performs the prediction on the basis of the prediction mode included in the encoding information supplied from the lossless decoding unit 62. Figure 4 In the intra prediction mode used in the intra prediction unit 34 of FIG. 71 , intra prediction is performed using the reference image supplied from the frame memory 69 via the selection unit 70. The intra prediction unit 71 supplies the predicted image obtained by the intra prediction to the selection unit 73.

[0187] The motion prediction / compensation unit 72 generates a prediction mode in accordance with the prediction mode included in the encoding information supplied from the lossless decoding unit 62. Figure 4 In the inter prediction mode used in the motion prediction / compensation unit 35, inter prediction is performed using the reference image supplied from the frame memory 69 via the selection unit 70. Inter prediction is performed using motion information and the like included in the encoding information supplied from the lossless decoding unit 62 as necessary.

[0188] The motion prediction / compensation unit 72 supplies the predicted image obtained by the inter prediction to the selection unit 73 .

[0189] The selection unit 73 selects the predicted image supplied from the intra prediction unit 71 or the predicted image supplied from the motion prediction / compensation unit 72 , and supplies the predicted image to the calculation unit 65 .

[0190] DF 31b applies DF to the decoded image from the calculation unit 65 according to the filter parameters etc. included in the encoding information provided from the lossless decoding unit 62, and provides the decoded image (filtered image) to which DF is applied or the decoded image (filtered image) to which DF is not applied to the adaptive offset filter 81.

[0191] According to the filter parameters included in the encoding information provided from the lossless decoding unit 62, the adaptive offset filter 81 applies the adaptive offset filter to the decoded image from the DF 31b as needed, and provides the decoded image (filtered image) to which the adaptive offset filter is applied or the decoded image to which the adaptive offset filter is not applied to the ALF 82.

[0192] The ALF 82 applies ALF to the decoded image from the adaptive offset filter 81 as necessary, and supplies the decoded image to which ALF is applied or the decoded image to which ALF is not applied to the screen rearrangement buffer 67 and the frame memory 69 .

[0193] In the decoder 51 configured as described above, the lossless decoding unit 62, the inverse quantization unit 63, the inverse orthogonal transform unit 64, and the calculation unit 65 function as a decoding unit that decodes an encoded bit stream to generate an image (decoded image).

[0194] <Decoding Process>

[0195] Figure 7 It is shown Figure 6 A flowchart of an example of a decoding process of the decoder 51.

[0196] In the decoding process, in step S51 , the accumulation buffer 61 temporarily accumulates the encoded bit stream transmitted from the encoder 11 , and appropriately supplies the encoded bit stream to the lossless decoding unit 62 , and the process proceeds to step S52 .

[0197] In step S52 , the lossless decoding unit 62 receives and decodes the encoded bit stream supplied from the accumulation buffer 61 , and supplies the quantization coefficient as encoded data included in the decoded result of the encoded bit stream to the inverse quantization unit 63 .

[0198] The lossless decoding unit 62 parses the encoding information included in the decoding result of the encoded bit stream. The lossless decoding unit 62 supplies necessary encoding information to the intra prediction unit 71, the motion prediction / compensation unit 72, the DF 31b, the adaptive offset filter 81, and other necessary blocks.

[0199] The process proceeds from step S52 to step S53, and the intra prediction unit 71 or the motion prediction / compensation unit 72 performs an intra prediction process or an inter motion prediction process for generating a predicted image based on the reference image supplied from the frame memory 69 via the selection unit 70 and the encoding information supplied from the lossless decoding unit 62. The intra prediction unit 71 or the motion prediction / compensation unit 72 supplies the predicted image obtained by the intra prediction process or the inter motion prediction process to the selection unit 73, and the process proceeds from step S53 to step S54.

[0200] In step S54 , the selection unit 73 selects the predicted image supplied from the intra prediction unit 71 or the motion prediction / compensation unit 72 , and supplies the predicted image to the calculation unit 65 , and the process proceeds to step S55 .

[0201] In step S55 , the inverse quantization unit 63 inversely quantizes the quantized coefficient from the lossless decoding unit 62 , and supplies the resultant orthogonal transform coefficient to the inverse orthogonal transform unit 64 , and the process proceeds to step S56 .

[0202] In step S56 , the inverse orthogonal transform unit 64 performs inverse orthogonal transform on the orthogonal transform coefficient from the inverse quantization unit 63 , and supplies the resultant residual to the calculation unit 65 , and the process proceeds to step S57 .

[0203] In step S57, the calculation unit 65 generates a decoded image by adding the residual from the inverse orthogonal transform unit 64 and the predicted image from the selection unit 73. The calculation unit 65 supplies the decoded image to the DF 31b, and the process proceeds from step S57 to step S58.

[0204] In step S58, the DF 31b applies the DF to the decoded image from the calculation unit 65 according to the filter parameters included in the encoding information supplied from the lossless decoding unit 62, and supplies the resulting filtered image to the adaptive offset filter 81, and the process proceeds to step S59.

[0205] In step S59, the adaptive offset filter 81 applies the adaptive offset filter to the filtered image from the DF 31b according to the filter parameters included in the encoding information supplied from the lossless decoding unit 62, and supplies the resulting filtered image to the ALF 82, and the process proceeds to step S60.

[0206] The ALF 82 applies ALF to the filtered image from the adaptive offset filter 81 and supplies the resulting filtered image to the screen rearrangement buffer 67 and the frame memory 69, and the process proceeds to step S61.

[0207] In step S61, the frame memory 69 temporarily stores the filtered image supplied from the ALF 82, and the process proceeds to step S62. The filtered image (decoded image) stored in the frame memory 69 is used as a reference image which is a source from which a predicted image is generated in the intra prediction process or the inter motion prediction process in step S53.

[0208] In step S62 , the screen rearrangement buffer 67 rearranges the filtered images supplied from the ALF 82 in display order, and supplies the rearranged filtered images to the D / A conversion unit 68 , and the process proceeds to step S63 .

[0209] In step S63, the D / A conversion unit 68 performs D / A conversion on the filtered image from the picture rearrangement buffer 67, and the decoding process ends. The filtered image (decoded image) after D / A conversion is output and displayed on a display (not shown).

[0210] <Configuration example of DF 31a>

[0211] Figure 8 is a block diagram showing a configuration example of DF 31a of the encoder 11.

[0212] Here, hereinafter, for example, for DF, it is assumed that a total of two sets of offset parameter sets are set, one set of offset parameter sets for the luminance component and one set of offset parameter sets for the chrominance component. Then, from the two sets of offset parameter sets, the offset parameter set for the luminance component is used as the current offset parameter set for the luminance component, and the offset parameter set for the chrominance component is used as the current offset parameter set for the chrominance component.

[0213] In Figure 8 DF 31a has a setting unit 111 and a filter unit 112.

[0214] The setting unit 111 sets the filter parameters tc and bata of DF applied to the block boundaries of the blocks of the decoded image, and sets an offset parameter set for adjusting the filter parameters tc and beta.

[0215] For the offset parameter set, the setting unit 111 sets a total of two sets of offset parameter sets, one set of offset parameter sets for the luminance component and one set of offset parameter sets for the chrominance component.

[0216] The offset parameter sets for each of the luminance component and the chrominance component can be set according to, for example, coding parameters, image feature amounts detected from the luminance component and the chrominance component, a predetermined cost function, etc.

[0217] The filter parameters tc and beta and the two sets of offset parameter sets set by the setting unit 111 are provided to the filter unit 112 and the lossless coding unit 26.

[0218] In the lossless coding unit 26, the filter parameters tc and beta from the setting unit 111 and the two sets of offset parameter sets are set (included) in the syntax of the coded bitstream (for example, the picture parameter set and the slice header).

[0219] The filter parameters tc and beta and the two sets of offset parameter sets from the setting unit 111 and the decoded image from the calculation unit 30 are provided to the filter unit 112.

[0220] The filter unit 112 applies DF (filtered by DF) to the pixels near the block boundary of the decoded image using the filter parameters tc and beta from the setting unit 111.

[0221] In the application of DF, filter determination is performed. In the filter determination, the presence / absence of the application of DF and the type of DF to be applied when DF is applied are determined. Examples of DF types include long-tap filters, strong filters, and weak filters.

[0222] When applying DF to the pixels near the block boundary, the type of DF determined to be applied in the filter determination is applied.

[0223] In the application of DF including filter determination, a current offset parameter set is selected for each of the luminance component and the chrominance component of the decoded image from two sets of offset parameter sets from the setting unit 111.

[0224] That is, for the luminance component, the offset parameter set of the luminance component is selected as the current offset parameter set. For the chrominance component, the offset parameter set of the chrominance component is selected as the current offset parameter set.

[0225] For the luminance component, the filter parameters tc and beta are adjusted using the current offset parameter set selected for the luminance component (that is, the offset parameter set of the luminance component).

[0226] For the chrominance component, the filter parameters tc and beta are adjusted using the current offset parameter set selected for the chrominance component (that is, the offset parameter set of the chrominance component).

[0227] Using the adjusted filter parameters tc and beta, DF is applied to each of the luminance component and the chrominance component of the pixels near the block boundary of the decoded image.

[0228] The filter unit 112 generates a filtered image by applying DF and provides the filtered image to the adaptive offset filter 41.

[0229] <Configuration example of DF 31b>

[0230] Fig. 9 is a block diagram showing a configuration example of DF 31b of the decoder 51.

[0231] In Fig. 9 DF 31b has a filter unit 121.

[0232] The filter parameters tc and beta parsed from the lossless decoding unit 62 and two sets of offset parameter sets are provided to the filter unit 121. In addition, the decoded image is provided to the filter unit 121 from the calculation unit 65.

[0233] The filter unit 121 applies DF to pixels near the block boundary of the decoded image from the calculation unit 65 using the filter parameters tc and beta from the lossless decoding unit 62. The filter unit 121 generates a filtered image by applying DF and supplies the filtered image to the adaptive offset filter 81.

[0234] Since the application of DF in the filter unit 121 is based on Figure 8 The operation is performed in the same manner as in the filter unit 112 in FIG. 1 , and thus a description thereof will be omitted.

[0235] <Syntax>

[0236] Fig.10 is a diagram showing an example of the syntax of a picture parameter set when two sets of offset parameter sets are included in the picture parameter set.

[0237] exist Fig.10 , pps_beta_offset0_div2 and pps_tc_offset0_div2 are a set of offset parameter sets for adjusting filter parameters beta and tc, respectively, and pps_beta_offset1_div2 and pps_tc_offset1_div2 are another set of offset parameter sets for adjusting filter parameters beta and tc, respectively.

[0238] For example, pps_beta_offset0_div2 and pps_tc_offset0_div2 are offset parameter sets for luma components, and pps_beta_offset1_div2 and pps_tc_offset1_div2 are offset parameter sets for color difference components.

[0239] exist Fig.10 In the deblocking_offset_component_use_flag, the deblocking_offset_component_use_flag is a flag indicating whether two sets of offset parameter sets (pps_beta_offset0_div2 and pps_tc_offset0_div2 and pps_beta_offset1_div2 and pps_tc_offset1_div2) are offset parameter sets of the luma component and the color difference component, respectively.

[0240] For example, when deblocking_offset_component_use_flag is 1 (true), it indicates that two groups of offset parameter sets (pps_beta_offset0_div2 and pps_tc_offset0_div2 and pps_beta_offset1_div2 and pps_tc_offset1_div2) are offset parameter sets of a luma component and a color difference component, respectively.

[0241] If deblocking_offset_component_use_flag is not 1, that is, 0 (false), two groups of offset parameter sets (pps_beta_offset0_div2 and pps_tc_offset0_div2 and pps_beta_offset1_div2 and pps_tc_offset1_div2) can be used as encoding parameters, for example, multiple offset parameter sets as selection targets when selecting a current offset parameter set according to a block size.

[0242] In this case, a threshold value for encoding parameters may be included in the picture parameter set as a selection criterion for selecting pps_beta_offset0_div2 and pps_tc_offset0_div2 or pps_beta_offset1_div2 and pps_tc_offset1_div2.

[0243] exist Fig.10 In the pps_tc_beta_offset_switched_threshold, the threshold of the coding parameter is indicated.

[0244] For example, if pps_tc_beta_offset_switched_threshold is a threshold of block size, when the block size of a block constituting a block boundary to which DF is applied is equal to or greater than the threshold pps_tc_beta_offset_switched_threshold, one of two offset parameter sets pps_beta_offset0_div2 and pps_tc_offset0_div2 and pps_beta_offset1_div2 and pps_tc_offset1_div2 is selected.

[0245] On the other hand, if the block size of the block is not equal to or greater than the threshold pps_tc_beta_offset_switched_threshold, the other of the two sets of offset parameter sets pps_beta_offset0_div2 and pps_tc_offset0_div2 and pps_beta_offset1_div2 and pps_tc_offset1_div2 is selected.

[0246] The flag deblocking_offset_component_use_flag and the threshold pps_tc_beta_offset_switched_threshold are set in the setting unit 111 , and are included in the picture parameter set in the lossless encoding unit 26 .

[0247] Fig.11 is a diagram showing an example of the syntax of a slice header syntax when two sets of offset parameter sets are included in a slice header.

[0248] exist Fig.11 In the example, slice_beta_offset0_div2, slice_tc_offset0_div2, slice_beta_offset1_div2, slice_tc_offset1_div2, and slice_tc_beta_offset_switched_threshold correspond to Fig.10 pps_beta_offset0_div2, pps_tc_offset0_div2, pps_beta_offset1_div2, pps_tc_offset1_div2 and pps_tc_beta_offset_switched_threshold in .

[0249] Here, for example, for DF, it is assumed that a total of two sets of offset parameter sets are set, one set of offset parameter sets is used for the luminance component, and one set of offset parameter sets is used for the chrominance component, the offset parameter set of the luminance component is used as the current offset parameter set of the luminance component, and the offset parameter set of the chrominance component is used as the current offset parameter set of the chrominance component.

[0250] However, for DF, Figure 2 As described above, three groups of offset parameter sets may be set for each component of YUV or YCbCr, and the offset parameter set of each component may be set as the current offset parameter of each component.

[0251] in this case, Fig.10 The picture parameter set and Fig.11The slice header may include three sets of offset parameters for each component of YUV or YCbCr.

[0252] When a plurality of offset parameter sets included in a picture parameter set are fixed to an offset parameter set for each of a luma component and a color difference component, or an offset parameter set for each component, Fig.10 The picture parameter set in can be configured without deblocking_offset_component_use_flag and pps_tc_beta_offset_switched_threshold. Similarly, Fig.11 The slice header can be configured without deblocking_offset_component_use_flag and slice_tc_beta_offset_switched_threshold.

[0253] <Description of Computer to Which the Present Technology is Applied>

[0254] The above-described series of processing can be executed by hardware or software. In the case where the series of processing is executed by software, a program configuring the software is installed in a general-purpose computer or the like.

[0255] Fig.12 is a block diagram showing an example of a configuration of an embodiment of a computer installed with a program for executing the above-mentioned series of processes.

[0256] The program may be recorded in advance in the hard disk 905 or the ROM 903 which is a recording medium included in the computer.

[0257] Alternatively, the program may be stored (recorded) in a removable recording medium 911 driven by the drive 909. The removable recording medium 911 may be provided as so-called packaged software. Here, there are, for example, a floppy disk, a compact disk read-only memory (CD-ROM), a magneto-optical (MO) disk, a digital versatile disk (DVD), a magnetic disk, a semiconductor memory, and the like as the removable recording medium 911.

[0258] Note that, in addition to being installed to the computer from the above-mentioned removable recording medium 911, the program can also be downloaded to the computer via a communication network or a broadcasting network and installed in the hard disk 905 included in the computer. That is, the program can be sent to the computer from a download site in a wireless manner via an artificial satellite for digital satellite broadcasting, or can be sent to the computer in a wired manner via a network such as a local area network (LAN) or the Internet, for example.

[0259] The computer includes a central processing unit (CPU) 902 , and an input / output interface 910 is connected to the CPU 902 through a bus 901 .

[0260] When the user operates the input unit 907 or the like to input a command through the input / output interface 910, the CPU 902 executes a program stored in the read only memory (ROM) 903 according to the command. Alternatively, the CPU 902 loads a program stored in the hard disk 905 to the random access memory (RAM) 904 and executes the program.

[0261] Thus, the CPU 902 performs the processing according to the above-described flowchart or the processing performed by the components of the above-described block diagram. In addition, the CPU 902 outputs the processing result from the output unit 906, for example, through the input / output interface 910, or sends the processing result from the communication unit 908, and additionally records the processing result in the hard disk 905 or the like as needed.

[0262] Note that the input unit 907 is configured as a keyboard, a mouse, a microphone, etc. In addition, the output unit 906 is configured as a liquid crystal display (LCD), a speaker, or the like.

[0263] Here, the processing performed by the computer according to the program is not necessarily performed according to the order described as the flowchart in this specification. That is, the processing performed by the computer according to the program also includes processing performed in parallel or individually (for example, parallel processing or processing according to objects).

[0264] In addition, the program may be processed by a single computer (processor), or may be processed in a distributed manner by a plurality of computers. In addition, the program may be sent to a remote computer and executed.

[0265] In addition, in this specification, a system refers to a collection of multiple components (devices, modules (components), etc.), and it does not matter whether all the components are arranged in a single housing. Therefore, multiple devices housed in different housings and connected via a network and a device in which multiple modules are housed in one housing are both systems.

[0266] Note that the embodiments of the present technology are not limited to the above-mentioned embodiments and may be modified in various ways without departing from the gist of the present technology.

[0267] For example, the present technology may be configured as cloud computing in which one function is shared and jointly processed by a plurality of devices via a network.

[0268] In addition, each step described in the above flowchart may be performed by one device, or may be performed by a plurality of devices in a shared manner.

[0269] Furthermore, in the case where a plurality of kinds of processing are included in a single step, the plurality of kinds of processing included in the single step may be performed by one device or may be performed by a plurality of devices in a shared manner.

[0270] The effects described in this specification are merely illustrative and non-restrictive, and other effects may be obtained.

[0271] According to the above description, the embodiments of the present invention also disclose the following technical solutions, including but not limited to:

[0272] Solution 1. An image processing device, comprising:

[0273] a decoding unit that decodes the bit stream to generate a decoded image; and

[0274] A filter unit that applies a deblocking filter to a color difference component of pixels near a block boundary of a decoded image generated by the decoding unit using a color difference offset parameter set that is used when applying the deblocking filter to the color difference component of the decoded image generated by the decoding unit.

[0275] Solution 2. An image processing device according to Solution 1, wherein:

[0276] The filter unit applies a deblocking filter to a first color difference component and a second color difference component of pixels near a block boundary of a decoded image generated by the decoding unit using a first color difference offset parameter set and a second color difference offset parameter set, wherein the first color difference offset parameter set is used when the deblocking filter is applied to the first color difference component of the decoded image, and the second color difference offset parameter set is used when the deblocking filter is applied to the second color difference component of the decoded image.

[0277] Solution 3. An image processing device according to Solution 2, wherein:

[0278] When the color space for the decoded image is YUV,

[0279] The first color difference offset parameter set is an offset parameter set for a U component of the decoded image, and

[0280] The second color difference offset parameter set is an offset parameter set for a V component of the decoded image.

[0281] Solution 4. An image processing device according to Solution 2, wherein:

[0282] When the color space for the decoded image is YCbCr,

[0283] The first color difference offset parameter set is an offset parameter set for the Cb component of the decoded image, and

[0284] The second color difference offset parameter set is an offset parameter set for a Cr component of the decoded image.

[0285] Solution 5. An image processing device according to Solution 2, wherein:

[0286] The first color difference offset parameter set and the second color difference offset parameter set are set as syntax of the bitstream, and

[0287] The decoding unit parses the first color difference offset parameter set and the second color difference offset parameter set included in the bit stream.

[0288] Solution 6. An image processing device according to Solution 5, wherein:

[0289] The syntax is a picture parameter set or a slice header.

[0290] Solution 7. An image processing device according to Solution 5, wherein:

[0291] The syntax is picture parameter set and slice header.

[0292] Solution 8. The image processing device according to Solution 2, wherein:

[0293] The filter unit applies a deblocking filter to a luminance component of pixels near a block boundary of a decoded image generated by the decoding unit using a luminance offset parameter set used when applying the deblocking filter to the luminance component of the decoded image.

[0294] Solution 9. An image processing method, comprising:

[0295] A decoding step of decoding the bit stream to generate a decoded image; and

[0296] A filtering step, using a color difference offset parameter set, to apply a deblocking filter to the color difference component of pixels near the block boundary of the decoded image generated by the decoding step, wherein the color difference offset parameter set is used when applying the deblocking filter to the color difference component of the decoded image generated by the decoding step.

[0297] Solution 10. An image processing device, comprising:

[0298] a setting unit that sets a color difference offset parameter set used when a deblocking filter is applied to a color difference component of a decoded image obtained by decoding a bit stream; and

[0299] An encoding unit encodes the image to generate a bit stream including the color difference offset parameter set by the setting unit.

[0300] Solution 11. The image processing device according to Solution 10, wherein:

[0301] The setting unit sets a first color difference offset parameter set and a second color difference offset parameter set, wherein the first color difference offset parameter set is used when a deblocking filter is applied to a first color difference component of the decoded image, and the second color difference offset parameter set is used when a deblocking filter is applied to a second color difference component of the decoded image, and

[0302] The encoding unit generates a bitstream including the first color difference offset parameter set and the second color difference offset parameter set set by the setting unit.

[0303] Solution 12. The image processing device according to Solution 11, wherein:

[0304] When the color space for the decoded image is YUV,

[0305] The first color difference offset parameter set is a parameter set for the U component of the decoded image, and

[0306] The second color difference offset parameter set is a parameter set for a V component of the decoded image.

[0307] Solution 13. An image processing device according to Solution 11, wherein:

[0308] When the color space for the decoded image is YCbCr,

[0309] The first color difference offset parameter set is an offset parameter set for the Cb component of the decoded image, and

[0310] The second color difference offset parameter set is an offset parameter set for a Cr component of the decoded image.

[0311] Solution 14. The image processing device according to Solution 11, wherein:

[0312] The encoding unit generates a bitstream including the first color difference offset parameter set and the second color difference offset parameter set set by the setting unit as syntax.

[0313] Solution 15. An image processing device according to Solution 14, wherein:

[0314] The syntax is a picture parameter set or a slice header.

[0315] Solution 16. An image processing device according to Solution 14, wherein:

[0316] The syntax is picture parameter set and slice header.

[0317] Solution 17. The image processing device according to Solution 11, wherein:

[0318] The setting unit sets a luminance offset parameter set, the luminance offset parameter set being used when a deblocking filter is applied to a luminance component of the decoded image, and

[0319] The encoding unit generates a bitstream including the luma offset parameter set by the setting unit.

[0320] Solution 18. The image processing device according to Solution 11 further includes:

[0321] A filter unit that applies a deblocking filter to a color difference component of pixels near a block boundary of a locally decoded image that is locally decoded when encoding, using the color difference offset parameter set, to generate a filtered image, wherein

[0322] The encoding unit encodes the image using the filtered image generated by the filter unit.

[0323] Solution 19. An image processing method, comprising:

[0324] A setting step of setting a color difference offset parameter set, the color difference offset parameter set being used when a deblocking filter is applied to a color difference component of a decoded image obtained by decoding a bit stream; and

[0325] The encoding step encodes the image to generate a bit stream including the color difference offset parameter set by the setting step.

[0326] Reference numerals list

[0327] 10 Image Processing System

[0328] 11 Encoder

[0329] 21 A / D conversion unit

[0330] 22 Screen reordering buffer 22

[0331] 23 Computational Units

[0332] 24 Orthogonal Transformation Unit

[0333] 25 Quantization Units

[0334] 26 Lossless Coding Units

[0335] 27 Accumulation Buffer

[0336] 28 Inverse Quantization Unit

[0337] 29 Inverse Orthogonal Transformation Unit

[0338] 30 computing units

[0339] 31a, 31b DF

[0340] 32 frame memory

[0341] 33 Select Unit

[0342] 34 Intra prediction unit

[0343] 35 Motion prediction / compensation unit

[0344] 36 Prediction image selection unit

[0345] 37 Rate Control Unit

[0346] 41 Adaptive Offset Filter

[0347] 42 ALF

[0348] 51 Decoder

[0349] 61 Accumulation Buffer

[0350] 62 Lossless decoding unit

[0351] 63 Inverse Quantization Unit

[0352] 64 Inverse orthogonal transform unit

[0353] 65 computing units

[0354] 67 Screen Rearrangement Unit

[0355] 68 D / A conversion unit

[0356] 69 Frame Memory

[0357] 70 Select Units

[0358] 71 Intra prediction unit

[0359] 72 Motion prediction / compensation unit

[0360] 73 Select Unit

[0361] 81 Adaptive offset filter

[0362] 82 ALF

[0363] 111 Setting unit

[0364] 112, 121 filter unit

[0365] 901 Bus

[0366] 902 CPU

[0367] 903 ROM

[0368] 904 RAM

[0369] 905 Hard Drive

[0370] 906 Output Unit

[0371] 907 Input Unit

[0372] 908 Communication Unit

[0373] 909 Drive

[0374] 910 Input / Output Interface

[0375] 911 Removable recording media

Claims

1. An image processing device, comprising: a parsing unit, which parses the multiple sets of picture unit offset parameter sets according to a bitstream including a color difference offset parameter set set as a picture parameter set and a color difference offset parameter set set as a picture header as the multiple sets of picture unit offset parameter sets; A decoding unit, which decodes the bit stream to generate a decoded image; as well as A filter unit that applies a deblocking filter to color difference components of pixels in the vicinity of a block boundary of a decoded image generated by the decoding unit, using the plurality of picture unit offset parameter sets parsed by the parsing unit.

2. The image processing device according to claim 1, wherein: The parsing unit parses the slice unit offset parameter according to the bit stream including the color difference offset parameter set set as the slice unit offset parameter, The filter unit applies a deblocking filter to color difference components of pixels near a block boundary of a decoded image generated by the decoding unit using the slice unit offset parameter set parsed by the parsing unit when a slice is applied as a filtering processing unit.

3. The image processing device according to claim 2, wherein: The color difference offset parameter set includes a first color difference offset parameter set and a second color difference offset parameter set, the first color difference offset parameter set is used when a deblocking filter is applied to a first color difference component of the decoded image, and the second color difference offset parameter set is used when a deblocking filter is applied to a second color difference component of the decoded image, The filter unit applies a deblocking filter to first and second color difference components of pixels in the vicinity of a block boundary of a decoded image generated by the decoding unit, using the first and second color difference offset parameter sets.

4. The image processing device according to claim 1, wherein: The parsing unit parses the multiple sets of picture unit offset parameter sets according to a bitstream including a luma offset parameter set set as a picture parameter set and a luma offset parameter set set as a picture header as the multiple sets of picture unit offset parameter sets, The filter unit applies a deblocking filter to luminance components of pixels in the vicinity of a block boundary of a decoded image generated by the decoding unit, using the plurality of sets of picture unit offset parameter sets parsed by the parsing unit.

5. The image processing device according to claim 4, wherein: The parsing unit parses the slice unit offset parameter according to the bitstream including the luma offset parameter set set as a slice header as a slice unit offset parameter, The filter unit applies a deblocking filter to luminance components of pixels near a block boundary of a decoded image generated by the decoding unit using the slice unit offset parameter set parsed by the parsing unit when a slice is applied as a filtering processing unit.

6. An image processing method, comprising: A parsing step, parsing the multiple sets of picture unit offset parameter sets according to a bit stream including a color difference offset parameter set set as a picture parameter set and a color difference offset parameter set set as a picture header as the multiple sets of picture unit offset parameter sets; A decoding step of decoding the bit stream to generate a decoded image; as well as A filtering step, using the multiple sets of picture unit offset parameter sets parsed by the parsing step, to apply a deblocking filter to the color difference components of pixels near the block boundary of the decoded image generated by the decoding step.

7. An image processing device, comprising: A setting unit, which sets a plurality of color difference offset parameter sets as picture unit offset parameter sets; as well as An encoding unit encodes the image to generate a bit stream including the picture unit offset parameter set set by the setting unit as a picture parameter set and a picture header.

8. The image processing device according to claim 7, wherein: The setting unit sets the color difference offset parameter set as the slice unit offset parameter, The encoding unit generates a bitstream including the slice unit offset parameter set by the setting unit as a slice header.

9. The image processing apparatus according to claim 8, further comprising: A filter unit that applies a deblocking filter to a color difference component of pixels near a block boundary of a locally decoded image that is locally decoded when encoding, using the color difference offset parameter set, to generate a filtered image, wherein The encoding unit encodes the image using the filtered image generated by the filter unit.

10. The image processing device according to claim 8, wherein: The setting unit sets multiple groups of brightness offset parameter sets as picture unit offset parameter sets, The encoding unit generates a bitstream including the picture unit offset parameter set set by the setting unit as a picture parameter set and a picture header.

11. The image processing apparatus according to claim 9, wherein: The setting unit sets the brightness offset parameter set as the slice unit offset parameter set, The encoding unit generates a bitstream including the slice unit offset parameter set by the setting unit as a slice header.

12. An image processing method, comprising: A setting step, setting a plurality of color difference offset parameter sets as picture unit offset parameter sets; as well as The encoding step encodes the image to generate a bit stream including the picture unit offset parameter set set by the setting step as a picture parameter set and a picture header.