Image decoding apparatus, image decoding method, and storage medium

Abstract

This invention relates to an image decoding apparatus, an image decoding method, and a storage medium. An image decoding apparatus (200) includes a filter unit (250B) configured to take a pre-filtered decoding signal as input and output a post-filtered decoding signal. The filter unit (250B) is configured to perform amplitude limiting on the pre-filtered decoding signal such that the absolute value of the difference between a reference pixel value and a pixel value of the pre-filtered decoding signal is below a predefined threshold. The post-filtered decoding signal is generated by a linear weighted sum of the amplitude-limited value and the pixel value of the pre-filtered decoding signal. Furthermore, it is defined as being able to preserve the magnitude relationship between a threshold for the luminance signal and a threshold for the chrominance signal even if the bit depth changes.

Description

[0001] This application is a divisional application of application number 202080004683.1, filed on June 11, 2020, entitled "Image Decoding Apparatus, Image Decoding Method and Storage Medium". Technical Field

[0002] This invention relates to an image decoding apparatus, an image decoding method, and a storage medium. Background Technology

[0003] Previously, a technique was known to perform nonlinear filtering by limiting the input signal to an "ALF (Adaptive Loop Filter)" using a threshold (see, for example, Non-Patent Literature 1).

[0004] This threshold is defined mathematically and its final value is derived from settings such as the internal bit depth. This threshold is defined separately for the luminance signal and the chrominance signal.

[0005] Existing technical documents

[0006] Non-patent literature

[0007] Non-patent literature 1: Versatile Video Coding (Draft 5), JVET-N1001 Summary of the Invention

[0008] The problem that the invention aims to solve

[0009] However, in the aforementioned prior art, the relationship between the threshold values ​​of the luminance signal and the chrominance signal is reversed due to the internal bit depth. Therefore, even for the same input signal, the characteristics of the decoded image may change due to the setting of the internal bit depth, thus affecting subjective image quality.

[0010] Therefore, the present invention was made in view of the above-mentioned problems, and its object is to provide an image decoding apparatus, an image decoding method, and a storage medium that can prevent changes in the characteristics of the decoded image from affecting subjective image quality.

[0011] Solution for solving the problem

[0012] The main idea of ​​the first technical solution of the present invention is to provide an image decoding device, which includes a filter unit configured to take a pre-filtered decoding signal as input and output a post-filtered decoding signal. The filter unit is configured to perform amplitude limiting processing on the pre-filtered decoding signal so that the absolute value of the difference between a reference pixel value and a pixel value of the pre-filtered decoding signal is below a predefined threshold. The post-filtered decoding signal is generated by linearly weighting the value after amplitude limiting and the pixel value of the pre-filtered decoding signal. Furthermore, it is defined as being able to maintain the magnitude relationship between the threshold for the luminance signal and the threshold for the chrominance signal even if the internal bit depth changes.

[0013] The second technical solution of the present invention is an image decoding method, which includes the steps of taking a pre-filtered decoded signal as input and outputting a post-filtered decoded signal. In the steps, the pre-filtered decoded signal is subjected to amplitude limiting processing so that the absolute value of the difference between the reference pixel value and the pixel value of the pre-filtered decoded signal is below a predefined threshold. The post-filtered decoded signal is generated by linearly weighting the value after amplitude limiting and the pixel value of the pre-filtered decoded signal. Furthermore, it is defined as being able to preserve the magnitude relationship between the threshold used for the luminance signal and the threshold used for the chrominance signal even if the internal bit depth changes.

[0014] The main idea of ​​the third technical solution of the present invention is a storage medium storing a program thereon, characterized in that, when the program is executed by a computer, it implements the step of taking a pre-filtered decoded signal as input and outputting a post-filtered decoded signal, wherein in the step, the pre-filtered decoded signal is subjected to amplitude limiting processing so that the absolute value of the difference between the reference pixel value and the pixel value of the pre-filtered decoded signal is below a predefined threshold, and the post-filtered decoded signal is generated by a linear weighted sum of the value after amplitude limiting and the pixel value of the pre-filtered decoded signal, and is defined as being able to preserve the magnitude relationship between the threshold for the luminance signal and the threshold for the chrominance signal even if the internal bit depth changes.

[0015] The main idea of ​​the fourth technical solution of the present invention is to provide an image decoding device, which includes a filter unit configured to take a pre-filtered decoding signal as input and output a post-filtered decoding signal. The filter unit is configured to perform amplitude limiting processing on the difference between a reference pixel value and a pixel value of the pre-filtered decoding signal so that the absolute value of the difference between the reference pixel value and the pixel value of the pre-filtered decoding signal is below a predefined threshold. The post-filtered decoding signal is generated by linearly weighting the value after amplitude limiting, the filter coefficients, and the pixel value of the pre-filtered decoding signal. Furthermore, the threshold for the luminance signal is set to be above the threshold for the chrominance signal, regardless of the internal bit depth.

[0016] The fifth technical solution of the present invention is an image decoding method, which includes the steps of taking a pre-filtered decoded signal as input and outputting a post-filtered decoded signal. In the steps, the difference between a reference pixel value and the pixel value of the pre-filtered decoded signal is subjected to amplitude limiting processing so that the absolute value of the difference between the reference pixel value and the pixel value of the pre-filtered decoded signal is below a predefined threshold. The post-filtered decoded signal is generated by linearly weighting the value after amplitude limiting, the filter coefficients, and the pixel value of the pre-filtered decoded signal. Furthermore, the threshold for the luminance signal is set above the threshold for the chrominance signal and is independent of the internal bit depth.

[0017] The main idea of ​​the sixth technical solution of the present invention is to provide a storage medium storing a program that, when executed by a computer, implements the following steps: taking a pre-filtered decoded signal as input and outputting a post-filtered decoded signal. In this step, the difference between a reference pixel value and the pixel value of the pre-filtered decoded signal is subjected to amplitude limiting processing so that the absolute value of the difference between the reference pixel value and the pixel value of the pre-filtered decoded signal is below a predefined threshold. The post-filtered decoded signal is generated by linearly weighting the value after amplitude limiting, the filter coefficients, and the pixel value of the pre-filtered decoded signal. Furthermore, the threshold for the luminance signal is set above the threshold for the chrominance signal, regardless of the internal bit depth.

[0018] The effects of the invention

[0019] According to the present invention, an image decoding apparatus, an image decoding method, and a storage medium are provided that can prevent changes in the characteristics of the decoded image from affecting subjective image quality. Attached Figure Description

[0020] Figure 1This is a diagram illustrating an example of the structure of an image processing system 10 according to one embodiment.

[0021] Figure 2 This is a diagram illustrating an example of the functional blocks of an image encoding apparatus 100 according to one embodiment.

[0022] Figure 3 This is a diagram illustrating an example of a functional block of the in-loop filtering processing unit 150 of an image encoding apparatus 100 according to one embodiment.

[0023] Figure 4 This is a diagram illustrating an example of the function of the filter section 150B of the in-loop filtering processing section 150 of an image encoding apparatus 100 according to one embodiment.

[0024] Figure 5 This is a diagram illustrating an example of the function of the filter section 150B of the in-loop filtering processing section 150 of an image encoding apparatus 100 according to one embodiment.

[0025] Figure 6 This is a diagram illustrating an example of the function of the filter section 150B of the in-loop filtering processing section 150 of an image encoding apparatus 100 according to one embodiment.

[0026] Figure 7 This is a diagram illustrating an example of the function of the filter section 150B of the in-loop filtering processing section 150 of an image encoding apparatus 100 according to one embodiment.

[0027] Figure 8 This is a diagram illustrating an example of the functional blocks of an image decoding apparatus 200 according to one embodiment.

[0028] Figure 9 This is a diagram illustrating an example of a functional block of the in-loop filtering processing unit 250 of an image decoding apparatus 200 according to one embodiment.

[0029] Figure 10 This is a flowchart illustrating an example of the processing procedure of the in-loop filtering processing unit 250 of an image decoding apparatus 200 according to one embodiment. Detailed Implementation

[0030] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. Furthermore, the constituent elements in the following embodiments can be appropriately replaced with existing constituent elements, and various modifications, including combinations with other existing constituent elements, are possible. Therefore, the description of the following embodiments is not intended to limit the scope of the invention as described in the claims.

[0031] (First Implementation)

[0032] The following is for reference Figures 1-10 The image processing system 10 according to the first embodiment of the present invention will be described. Figure 1 This is a diagram illustrating the image processing system 10 of this embodiment.

[0033] like Figure 1 As shown, the image processing system 10 includes an image encoding device 100 and an image decoding device 200.

[0034] The image encoding device 100 is configured to generate encoded data by encoding an input image signal. The image decoding device 200 is configured to generate an output image signal by decoding the encoded data.

[0035] The encoded data can be transmitted from the image encoding device 100 to the image decoding device 200 via a transmission path. Alternatively, the encoded data can be stored in a storage medium and then provided from the image encoding device 100 to the image decoding device 200.

[0036] (Image encoding device 100)

[0037] The following is for reference Figure 2 The image encoding apparatus 100 of this embodiment will be described. Figure 2 This is a diagram illustrating an example of the functional blocks of the image encoding apparatus 100 of this embodiment.

[0038] like Figure 2 As shown, the image encoding apparatus 100 includes an inter-frame prediction unit 111, an intra-frame prediction unit 112, a subtractor 121, an adder 122, a transform and quantization unit 131, an inverse transform and inverse quantization unit 132, an encoding unit 140, an in-loop filtering processing unit 150, and a frame buffer 160.

[0039] The inter-frame prediction unit 111 is configured to generate a prediction signal through inter-frame prediction.

[0040] Specifically, the inter-frame prediction unit 111 is configured to determine the reference block contained in the reference frame by comparing the encoded target frame (hereinafter referred to as the target frame) with the reference frame stored in the frame buffer 160, and to determine the motion vector for the determined reference block.

[0041] Furthermore, the inter-frame prediction unit 111 is configured to generate a prediction signal contained in each prediction block based on a reference block and a motion vector. The inter-frame prediction unit 111 is configured to output the prediction signal to a subtractor 121 and an adder 122. The reference frame is a frame different from the target frame.

[0042] The intra-frame prediction unit 112 is configured to generate a prediction signal through intra-frame prediction.

[0043] Specifically, the intra-frame prediction unit 112 is configured to determine the reference blocks contained in the target frame and generate a prediction signal for each prediction block based on the determined reference blocks. Furthermore, the intra-frame prediction unit 112 is configured to output the prediction signal to the subtractor 121 and the adder 122.

[0044] The reference block is a block that is referenced in relation to the target block (hereinafter referred to as the target block). For example, the reference block is a block that is adjacent to the target block.

[0045] Subtractor 121 is configured to subtract the prediction signal from the input image signal and output the prediction residual signal to the transform and quantization unit 131. Subtractor 121 is configured to generate the prediction residual signal, which is the difference between the prediction signal generated by intra-frame prediction or inter-frame prediction and the input image signal.

[0046] Adder 122 is configured to add the prediction signal to the prediction residual signal output from the inverse transform and inverse quantization unit 132 to generate a pre-filtering decoded signal, and output such pre-filtering decoded signal to the intra-frame prediction unit 112 and the intra-loop filtering unit 150.

[0047] The decoded signal before filtering constitutes the reference block used in the intra-frame prediction unit 112.

[0048] The transform and quantization unit 131 is configured to perform transform processing on the prediction residual signal and obtain coefficient level values. Furthermore, the transform and quantization unit 131 can also be configured to perform quantization of the coefficient level values.

[0049] The transformation process involves converting the predicted residual signal into frequency component signals. This transformation can utilize either the basic pattern (transformation matrix) corresponding to the Discrete Cosine Transform (DCT) or the basic pattern (transformation matrix) corresponding to the Discrete Sine Transform (DST).

[0050] The inverse transform and inverse quantization unit 132 is configured to perform inverse transform processing on the coefficient level values ​​output from the transform and quantization unit 131. Alternatively, the inverse transform and inverse quantization unit 132 can be configured to perform inverse quantization of the coefficient level values ​​before the inverse transform processing.

[0051] The inverse transformation and inverse quantization are performed in the reverse order of the transformation and quantization performed by the transformation and quantization unit 131.

[0052] The encoding unit 140 is configured to encode the coefficient level values ​​output from the transform and quantization unit 131 and output the encoded data.

[0053] For example, the encoding is entropy encoding, which assigns codes of different lengths based on the probability of occurrence of coefficient level values.

[0054] In addition, the encoding unit 140 is configured to encode control data used in the decoding process, in addition to coefficient level values.

[0055] The control data may include size data such as coding unit (CU) size, prediction unit (PU) size, and transformation unit (TU) size.

[0056] The in-loop filtering unit 150 is configured to filter the pre-filtered decoded signal output from the adder 122 and output the filtered decoded signal to the frame buffer 160.

[0057] Alternatively, the in-loop filtering processing unit 150 may be configured to take the input image signal and the decoded signal before filtering as inputs to determine parameters related to filtering, and output such parameters to the encoding unit 140. The encoding unit 140 may also be configured to encode such parameters and transmit them as additional information to the image decoding device 200.

[0058] For example, the filtering process is an adaptive loop filtering process that reduces the encoding distortion of the decoded image.

[0059] The frame buffer 160 is configured as a reference frame used in the cumulative inter-frame prediction unit 111.

[0060] The filtered and decoded signal constitutes the reference frame used in the inter-frame prediction unit 111.

[0061] (Intra-loop filter processing unit 150)

[0062] The in-loop filtering processing unit 150 of this embodiment will be described below. Figure 3 This is a diagram showing the in-loop filtering processing unit 150 of this embodiment.

[0063] like Figure 3 As shown, the in-loop filtering processing unit 150 includes a category determination unit 150A, a filter unit 150B, and a parameter determination unit 150C.

[0064] The in-loop filtering processing unit 150 can select either the application or non-application of adaptive loop filtering processing on a per-coding tree unit (CTU) basis. Furthermore, when applying adaptive loop filtering processing, the in-loop filtering processing unit 150 can select which of multiple filter banks to use.

[0065] Each filter bank includes up to twenty-five types of filters for luminance signals and one type of filter for chrominance signals.

[0066] As described below, parameters, including information related to the application or non-application of such adaptive loop filtering and information related to the filter bank to be used in the adaptive loop filtering, are encoded as additional information for each CTU and transmitted to the image decoding device 200. Furthermore, as described below, such parameters are determined by the parameter determination unit 150C.

[0067] For example, when the input signal is in the "YCbCr4:2:0 format", CTU can also be defined as blocks obtained by dividing the luminance (Y) signal into 128×128 pixel size and the chrominance (Cb, Cr) signal into 64×64 pixel size.

[0068] Furthermore, for pixels whose luminance and chromaticity signals belong to CTUs that are determined not to be subject to adaptive loop filtering, the determination process performed by the category determination unit 150A and the filtering process performed by the filter unit 150B, which will be described later, can be omitted.

[0069] The category determination unit 150A is configured to take the decoded signal before filtering as input and output category determination information, which indicates which (class) filter among a variety of pre-determined filters (adaptive loop filters) is used.

[0070] The category determination unit 150A is configured to divide the decoded signal before filtering into small blocks and determine the category to be used for each such block. For example, the category determination unit 150A can determine which of twenty-five filter categories should be used for each 4×4 pixel block.

[0071] The method for classifying a category can be any method that can be used as input only, as long as it is a method that can be used to make a determination based on information that can be obtained from both the image encoding device 100 side and the image decoding device 200 side.

[0072] For example, in Non-Patent Document 1, the gradient of the pixel values ​​of the decoded signal before filtering is used to determine the aforementioned category. By using only information available from both the image encoding device 100 and the image decoding device 200, the same category determination can be performed on the image decoding device 200 side as on the image encoding device 100 side, thus eliminating the need to transmit category determination information from the image encoding device 100 side to the image decoding device 200 side.

[0073] The filter unit 150B is configured to take the unfiltered decoded signal as input and output the filtered decoded signal.

[0074] Specifically, the filter unit 150B is configured to take the decoded signal before filtering, the category determination information output from the category determination unit 150A, and the parameters (the parameter group of the loop filtering process) input from the parameter determination unit 150C as inputs to perform filtering and output the decoded signal after filtering.

[0075] The values ​​of each pixel in the decoded signal after filtering can be calculated using formulas such as the following.

[0076] [Mathematical Expression 1]

[0077] O(x,y)=I(x,y)+∑C(i,j).K(I(x+i,y+j)-(x,y),k(i,j))

[0078] (i, j) ≠ (0, 0)

[0079] Where I(x, y) is the pixel value of the decoded signal before filtering at coordinate (x, y), O(x, y) is the pixel value of the decoded signal after filtering at coordinate (x, y), (i, j) is the coordinate (reference pixel position) representing the relative position of the reference pixel with respect to the pixel at coordinate (x, y), C(i, j) is the filter coefficient corresponding to the reference pixel position (i, j), K() is the clipping process shown below, and k(i, j) is the threshold used in the clipping process.

[0080] K(I,k)=min(k,max(-k,I))

[0081] Here, `min()` is a function that returns the minimum value of the independent variables, while `max()` is a function that returns the maximum value of the independent variables. Therefore, `K()` returns -k if the input value I is less than -k, returns k if the input value I is greater than k, and returns the input value I directly in other cases.

[0082] That is, the filter unit 150B is configured to perform amplitude limiting processing on the decoded signal before filtering so that the absolute value of the difference between the reference pixel value I(x+i, y+j) and the pixel value I(x, y) of the decoded signal before filtering is less than or equal to a predefined threshold k(i, j).

[0083] In addition, the filter unit 150B is configured to generate the filtered decoded signal by a linear weighted sum of the value after the clipping process and the pixel value I(x,y) of the decoded signal before the filtering process.

[0084] The filter coefficients C are determined by the parameter determination unit 150C and transmitted to the image decoding device 200. To reduce the amount of code related to the filter coefficients C, the number of filter coefficients C can be reduced by applying the same filter coefficients C to multiple pixels. Figure 4 Specific examples are shown below.

[0085] Figure 4 (a) shows an example of the configuration of filter coefficients C in the filtering process of a luminance signal. Figure 4 (b) shows an example of the configuration of filter coefficients C in the filtering process of color difference signals.

[0086] like Figure 4 As shown in (a), for example, in the luminance signal, twelve filter coefficients C0 to C11 are used. Figure 4 In (a), the pixel denoted as X is the pixel position (x, y) whose pixel value has been corrected through such filtering. Pixels with the same filter coefficient C are arranged symmetrically around the pixel position (x, y).

[0087] like Figure 4 As shown in (b), for the color difference signal, the other filter coefficients are also arranged symmetrically with the pixel recorded as X as the center.

[0088] For example, when the reference pixel is a pixel outside the image, filtering can be achieved by copying (called padding) the pixel values ​​of the decoded signal located at the image boundary before filtering.

[0089] in, Figure 5 This shows an example of filtering at the bottom edge (image boundary) of an image.

[0090] exist Figure 5 In (a), the grayscale filter coefficients C0 to C3 reference pixels outside the image. At this time, as inputs to the grayscale filter coefficients C2 and C0, a method can be used... Figure 5 (a) The pixel value that is the same as the pixel value corresponding to the filter coefficient C6 located directly above the filter coefficient C2.

[0091] Similarly, as input to the grayscale filter coefficient C3, it can be used with... Figure 5 (a) The pixel value corresponding to the filter coefficient C7 positioned directly above filter coefficient C3 is used as the input to the grayed-out filter coefficient C1. This can be done using the same pixel value as in... Figure 5 (a) The pixel value that is the same as the pixel value corresponding to the filter coefficient C5 located directly above the filter coefficient C1.

[0092] Furthermore, considering the point symmetry of the aforementioned filter coefficients C, when the padding value is used for a portion of a reference pixel, the padding value can also be used as the input to the filter coefficients C at the position corresponding to such a reference pixel.

[0093] exist Figure 5 In example (b), except in Figure 5 In addition to the filtering at the bottom of the image illustrated in example (a), the fill values ​​can also be used as input for the grayed-out filter coefficients C0 to C3.

[0094] For example, as a response to Figure 5 The inputs to filter coefficients C0 and C2 in (b) can also be filled with and used with the pixel value corresponding to filter coefficient C6 directly below filter coefficient C2.

[0095] Similarly, for the inputs of filter coefficients C1 and C3, the pixel values ​​corresponding to filter coefficients C5 and C7 directly below filter coefficients C1 and C3 can also be filled and used respectively.

[0096] In the example above, the filtering process at the bottom of the image has been explained, but the same filtering process can also be applied to the top and left and right sides of the image.

[0097] Furthermore, if the pixel value is not only the image boundary, but also the boundary of the parallel processing unit called the slice or tile, or the boundary of the pipeline processing called the virtual boundary, the above filtering method can be applied to process it in the same way.

[0098] Furthermore, by pre-setting the filtering processing at the aforementioned image boundaries, tile boundaries, slice boundaries, virtual boundaries, etc., to the same processing, the installation can be simplified compared to situations where different processing is applied to each boundary.

[0099] In addition, the threshold used in the clipping process can be set separately for each filter category and each filter coefficient C.

[0100] For example, if there are twenty-five types of filters for the luminance signal, and each type of filter coefficient C has twelve different types as described above, it is necessary to set a threshold for a maximum of 25 × 12 = 300 filter coefficients. Such thresholds are determined by the parameter determination unit 150C (described later) and transmitted as additional information to the image decoding device 200.

[0101] In order to reduce the amount of code involved in information related to such thresholds, the threshold itself may not be transmitted; instead, only index information may be encoded, which indicates which of a number of predetermined thresholds to use for each filter coefficient.

[0102] Furthermore, if four thresholds for luminance signals and four thresholds for chrominance signals are prepared in advance, the thresholds for luminance signals and the thresholds for chrominance signals can be, for example, as follows: Figure 6 (a) Defined as such.

[0103] The variance of the luminance signal tends to be larger than that of the chrominance signal, therefore... Figure 6 As shown in (a), by setting the threshold for the luminance signal to be higher than the threshold for the chromatic difference signal, it is possible to perform amplitude limiting processing that takes into account the characteristics of each signal.

[0104] That is, such as Figure 6 As shown in (a), it is defined as even if the internal bit depth ( Figure 6 Even if the "bitdepth" in the signal changes, the relationship between the threshold used for the luminance signal and the threshold used for the chrominance signal can still be preserved (for example, the threshold used for the luminance signal is greater than or equal to the threshold used for the chrominance signal).

[0105] In addition, such as Figure 6 As shown in (a), the threshold for the luminance signal and the threshold for the chrominance signal can also be defined as values ​​calculated by multiplying the changes in internal bit depth by the same factor. Based on this structure, even if the internal bit depth changes, the relationship between the thresholds for the luminance signal and the thresholds for the chrominance signal can be preserved, and the same limiting processing can be performed regardless of the internal bit depth.

[0106] The internal bit depth refers to the bit precision used when calculating pixel values ​​of luminance and chrominance signals during encoding and decoding processes. Such an internal bit depth may be an integer value of 8 to 16 bits, and is transmitted as additional information from the image encoding device 100 to the image decoding device 200.

[0107] In addition, such as Figure 6 As shown in (a), by predefining the multiplier related to the internal bit depth using powers of 2, it is possible to achieve... Figure 6(b) Such limiting processing can be achieved by simply shifting bits, which can reduce the processing load in hardware and software.

[0108] That is, the thresholds for luminance signals and chrominance signals can also be defined as being calculated by shifting bits respectively according to the change in internal bit depth.

[0109] also, Figure 6 (a) and Figure 6 The threshold corresponding to index 0 in (b) is the same value as the maximum value that can be represented by the corresponding internal bit depth. Therefore, by choosing such a threshold, the filter coefficients C corresponding to such a threshold are essentially equivalent to not performing clipping.

[0110] To achieve the same effect as in the example above, it is not necessary to set the multiplier for all thresholds to the same value as described above. Within a predetermined internal bit depth (e.g., 8 to 16 bits), the magnitude relationship of the thresholds corresponding to each index is not reversed within the luminance signal and the chrominance signal, respectively. Furthermore, between the luminance signal and the chrominance signal, as long as the magnitude relationship of the thresholds corresponding to the same index is not reversed, the multiplier for each threshold can be set to different values.

[0111] Furthermore, when thresholds for luminance signals and chrominance signals with indices of 0 to N-1 (N being a natural number greater than 1) are set, thresholds for luminance signals and chrominance signals with the same index can also be defined as calculated by multiplying by the same factor corresponding to the internal bit depth.

[0112] In addition, the example above illustrates the case where the threshold is multiplied by a factor corresponding to the internal bit depth, but it is also possible to add an offset value instead of multiplying by a factor.

[0113] Furthermore, even when the internal bit depth of the luminance signal differs from the internal bit depth of the chrominance signal, such as Figure 6 In this way, by predefining the thresholds for the luminance signal and the chrominance signal respectively, it is possible to ensure that the relationship between the thresholds used in the limiting process between the luminance signal and the chrominance signal remains unchanged, even after conversion to make the dynamic range of the luminance signal and the chrominance signal the same.

[0114] Furthermore, the example above illustrates setting a threshold for each category and each filter coefficient C. However, it is also possible to set the same threshold for all filter coefficients C for each category, i.e., to set a separate threshold for each category. In this case, the number of thresholds to be transmitted is reduced, thus reducing the amount of code associated with threshold indexing.

[0115] Furthermore, as described later, a flag indicating whether to apply clipping to each category is transmitted before the threshold index. Therefore, if there is only one threshold for each category, it is not necessary to define a threshold corresponding to index 0 (equivalent to no clipping). Thus, in the example above, the number of possible threshold modes can be reduced from four to three, and a further reduction in the amount of code associated with the threshold index can be expected.

[0116] In the example above, the case where the threshold values ​​for the luminance signal and the chrominance signal were set to different values ​​was explained; however, they can also be set to the same value. Therefore, for example, the circuitry related to the limiting processing of the luminance and chrominance signals can be shared in the hardware, reducing the circuit size.

[0117] In addition, the thresholds for luminance signals and chrominance signals can also be defined by calculating the thresholds based on an internal bit depth of 10 bits and corresponding to the current internal bit depth.

[0118] For example, in Figure 6 (a) shows examples of the following cases: based on the threshold values ​​of 1024, 181, 32, and 6 for the 10-bit luminance signal and 1024, 161, 25, and 4 for the 10-bit chrominance signal, corresponding to the current internal bit depth ( Figure 6 The value of bitdepth is multiplied by a power of 2 to obtain the final threshold.

[0119] Among them, Figure 6 In this context, the threshold for 10 bits is represented by a specific numerical value (1024, 181, etc.), but it can also be defined in a mathematical form.

[0120] For example, it can also be defined as, such as Figure 7 As shown, the threshold for 10 bits of the luminance signal is set to “2^(10(10×(4-index) / 4)(where index=0,1,2,3))”, and multiplied by “2^(bitdepth-10)” to obtain the final threshold corresponding to the current internal bit depth.

[0121] Figure 7 The definition of the color difference signal shown can also be considered as: “2^(8×(3-index) / 4)×2^(2)” is the threshold when there are 10 bits, and multiply it by “2^(bitdepth-10)” to calculate the threshold corresponding to the current internal bit depth.

[0122] The parameter determination unit 150C is configured to take the input image signal and the decoded signal before filtering as input, determine the parameters related to the adaptive loop filter and output them as additional information to the encoding unit 140, and output the determined parameters to the filter unit 150B.

[0123] The parameters determined by the parameter determination unit 150C include, for example, the following parameters.

[0124] First, as such a parameter, a filter bank can be listed on a frame-by-frame basis. Each filter bank includes up to twenty-five classes of filters for luminance signals and one class of filters for chrominance signals. For each class of filters, the values ​​of the filter coefficients are determined, along with a flag indicating whether clipping is applied, and the index of the threshold used for clipping in each filter coefficient when clipping is applied with that class.

[0125] The parameter determination unit 150C can have multiple filter banks in advance for a frame, and select which filter bank to use for each region as described later.

[0126] Second, as such a parameter, a flag indicating whether an adaptive loop filter is applied to each CTU can be listed. Furthermore, the parameter determination unit 150C can set index information indicating which of the multiple filter banks to use when an adaptive loop filter is applied.

[0127] Furthermore, the method for determining such parameters can be a well-known one, so detailed explanation is omitted.

[0128] In addition, although Figure 3 Although not shown in the figure, the parameter determination unit 150C may also use the determination result of the category determination unit 150A and the result of the filter unit 150B implemented in the temporary parameter setting in the parameter determination.

[0129] (Image decoding device 200)

[0130] The following is for reference Figure 7 The image decoding apparatus 200 of this embodiment will be described. Figure 7 This is a diagram illustrating an example of the functional blocks of the image decoding apparatus 200 of this embodiment.

[0131] like Figure 7 As shown, the image decoding apparatus 200 includes a decoding unit 210, an inverse transform and inverse quantization unit 220, an adder 230, an inter-frame prediction unit 241, an intra-frame prediction unit 242, an intra-loop filtering processing unit 250, and a frame buffer 260.

[0132] The decoding unit 210 is configured to decode the encoded data generated by the image encoding device 100 and to decode the coefficient level values.

[0133] For example, the decoding is entropy decoding in the reverse order of entropy encoding performed by the encoding unit 140.

[0134] Alternatively, the decoding unit 210 can also be configured to acquire control data by decoding the encoded data.

[0135] In addition, as mentioned above, the control data may also include size data such as the coding block size, prediction block size, and transform block size.

[0136] The inverse transform and inverse quantization unit 220 is configured to perform inverse transform processing on the coefficient level values ​​output from the decoding unit 210. Alternatively, the inverse transform and inverse quantization unit 220 can be configured to perform inverse quantization of the coefficient level values ​​before the inverse transform processing.

[0137] The inverse transformation and inverse quantization are performed in the reverse order of the transformation and quantization performed by the transformation and quantization unit 131.

[0138] Adder 230 is configured to add the prediction signal to the prediction residual signal output from the inverse transform and inverse quantization unit 220 to generate a pre-filtering decoded signal, and output the pre-filtering decoded signal to the intra-frame prediction unit 242 and the intra-loop filtering unit 250.

[0139] The decoded signal before filtering constitutes the reference block used in the intra-frame prediction unit 242.

[0140] Like the inter-frame prediction unit 111, the inter-frame prediction unit 241 is configured to generate a prediction signal through inter-frame prediction.

[0141] Specifically, the inter-frame prediction unit 241 is configured to generate a prediction signal for each prediction block based on the motion vector decoded from the encoded data and the reference signal contained in the reference frame. The inter-frame prediction unit 241 is configured to output the prediction signal to the adder 230.

[0142] Like the intra-frame prediction unit 112, the intra-frame prediction unit 242 is configured to generate a prediction signal through intra-frame prediction.

[0143] Specifically, the intra-prediction unit 242 is configured to determine the reference blocks contained in the target frame and generate a prediction signal for each prediction block based on the determined reference blocks. The intra-prediction unit 242 is configured to output the prediction signal to the adder 230.

[0144] The in-loop filtering processing unit 250, like the in-loop filtering processing unit 150, is configured to filter the decoded signal before filtering output from the adder 230 and output the decoded signal after filtering to the frame buffer 260.

[0145] For example, the filtering process is an adaptive loop filtering process that reduces the encoding distortion of the decoded image.

[0146] Frame buffer 260, like frame buffer 160, is configured as a reference frame used in the cumulative inter-frame prediction unit 241.

[0147] The filtered and decoded signal constitutes the reference frame used in the inter-frame prediction unit 241.

[0148] (In-loop filtering processing unit 250)

[0149] The in-loop filtering processing unit 250 of this embodiment will be described below. Figure 9 This is a diagram showing the in-loop filtering processing unit 250 of this embodiment.

[0150] like Figure 9 As shown, the in-loop filtering processing unit 250 includes a category determination unit 250A and a filter unit 250B.

[0151] Like the category determination unit 150A, the category determination unit 250A is configured to take the decoded signal before filtering as input and output category determination information, which indicates which category of filter among a variety of pre-determined in-loop filters is used.

[0152] Like filter unit 150B, filter unit 250B is configured to perform filtering based on the pre-filtering decoded signal, the category determination information determined by category determination unit 250A, and parameters transmitted from the image encoding device 100 side as additional information, and output the filtered decoded signal.

[0153] Figure 10 This is a flowchart illustrating an example of the processing procedure of the in-loop filtering processing unit 250 of the image decoding apparatus 200 of this embodiment.

[0154] like Figure 10 As shown, in step S101, the in-loop filtering processing unit 250 takes the decoded signal before filtering processing as input and outputs category determination information, which indicates which category of filter among a variety of predetermined in-loop filters is used.

[0155] In step S102, the in-loop filtering processing unit 250 performs filtering processing based on the decoded signal before filtering processing, the category determination information determined by the category determination unit 250A, and the parameters transmitted from the image encoding device 100 side as additional information, and outputs the decoded signal after filtering processing.

[0156] According to the image encoding apparatus 100 and image decoding apparatus 200 of this embodiment, the threshold processing of the input signal of the adaptive interpolation filter is independent of the setting of the internal bit depth. When the dynamic range of the luminance signal and the chrominance signal is the same after conversion, the magnitude relationship between the threshold for the luminance signal and the threshold for the chrominance signal remains unchanged. Therefore, it is possible to prevent the characteristics of subjective image quality from changing unintentionally.

[0157] Furthermore, the image encoding device 100 and the image decoding device 200 described above can be implemented by a program that causes a computer to execute each function (each step).

[0158] Furthermore, in the above embodiments, the present invention has been described using the image encoding device 100 and the image decoding device 200 as examples. However, the present invention is not limited to such examples and can also be applied to image encoding / decoding systems that have the functions of the image encoding device 100 and the image decoding device 200.

[0159] Symbol Explanation

[0160] 10…Image processing system; 100…Image encoding device; 111, 241…Inter-frame prediction unit; 112, 242…Intra-frame prediction unit; 121…Subtractor; 122, 230…Adder; 131…Transform and quantization unit; 132, 220…Inverse transform and inverse quantization unit; 140…Encoding unit; 150, 250…Intra-loop filtering unit; 150A, 250A…Class determination unit; 150B, 250B…Filter unit; 150C…Parameter determination unit; 160, 260…Frame buffer; 200…Image decoding device; 210…Decoding unit.

Claims

1. An image decoding device, characterized in that, It includes a filter section configured to take the unfiltered decoded signal as input and output the filtered decoded signal. The filter unit is configured to perform amplitude limiting on the difference between the reference pixel value and the pixel value of the decoded signal before filtering, such that the absolute value of the difference between the reference pixel value and the pixel value of the decoded signal before filtering is below a predefined threshold, and to generate the decoded signal after filtering by a linear weighted sum of the value after amplitude limiting, the filter coefficients, and the pixel value of the decoded signal before filtering. Furthermore, the threshold used for the luminance signal is set to be higher than the threshold used for the chrominance signal, regardless of the internal bit depth.

2. An image decoding method, comprising the steps of taking a pre-filtered decoded signal as input and outputting a post-filtered decoded signal, characterized in that, In the above step, the difference between the reference pixel value and the pixel value of the decoded signal before filtering is subjected to amplitude limiting processing so that the absolute value of the difference between the reference pixel value and the pixel value of the decoded signal before filtering is below a predefined threshold. The decoded signal after filtering is generated by linearly weighting the value after amplitude limiting processing, the filter coefficients, and the pixel value of the decoded signal before filtering. Furthermore, the threshold used for the luminance signal is set to be higher than the threshold used for the chrominance signal, regardless of the internal bit depth.

3. A storage medium having a program stored thereon, characterized in that, when said program is executed by a computer, the following is implemented: The steps involve using the unfiltered decoded signal as input and the filtered decoded signal as output. In the above step, the difference between the reference pixel value and the pixel value of the decoded signal before filtering is subjected to amplitude limiting processing so that the absolute value of the difference between the reference pixel value and the pixel value of the decoded signal before filtering is below a predefined threshold. The decoded signal after filtering is generated by linearly weighting the value after amplitude limiting processing, the filter coefficients, and the pixel value of the decoded signal before filtering. Furthermore, the threshold used for the luminance signal is set to be higher than the threshold used for the chrominance signal, regardless of the internal bit depth.

Images