Encoding device, decoding device, and program
By correcting the frequency components of undecoded reference pixels using DCT and IDCT, the devices improve prediction efficiency in intra-picture prediction, particularly for high-frequency components.
Patent Information
- Application Number
- JP2020012909
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2020-01-29
- Publication Date
- 2024-12-04
- Estimated Expiration
- 2040-01-29
AI Technical Summary
In intra-picture prediction of HEVC, using undecoded reference pixels with low-frequency components for high-frequency components leads to decreased prediction efficiency, as the same pixels are often used consecutively, resulting in inaccurate predictions.
The encoding and decoding devices correct the frequency components of undecoded reference pixels by performing DCT and IDCT on decoded neighboring pixels, adjusting the frequency power values, and applying weights based on distance to improve prediction accuracy.
This method allows for accurate prediction of TUs containing high-frequency components, enhancing prediction efficiency even when reference pixels are undecoded.
Smart Images

Figure 0007594364000002 
Figure 0007594364000003 
Figure 0007594364000004
Abstract
Description
[Technical field]
[0001] The present invention relates to an encoding device, a decoding device, and a program, and more particularly to an encoding device, a decoding device, and a program that perform intra-frame prediction of image information. [Background technology]
[0002] In recent years, with the increasing definition of images (including video), there is a demand for coding methods with improved coding efficiency. For example, in predictive coding methods for video such as MPEG (Moving Picture Experts Group)-2, H.264 / AVC (Advanced Video Coding), and H.265 / HEVC (High Efficiency Video Coding), each frame of a video is divided into blocks, and a predictive image is generated for each block as a basic processing unit, and the difference between the original image and the predictive image (residual signal) is coded and output.
[0003] There are two types of generation methods for predictive images: intra-frame prediction (intra-prediction) and motion-compensated prediction (inter-prediction). Intra-frame prediction predicts a block to be predicted by interpolation or extrapolation using surrounding pixels as reference pixels (Non-Patent Document 1). Motion-compensated prediction predicts a frame by interpolation from previous and next frames. Of these, a predictive image (I-frame (Intra-coded Frame)) generated by intra-frame prediction is an image (frame) that is the basis for inter-frame prediction, so a predictive coding method with higher accuracy and better prediction efficiency is required for intra-frame prediction.
[0004] For example, in intra-screen prediction, when directional prediction is performed on a large-sized predicted block, the prediction becomes less likely to be accurate as the distance from the reference pixel increases. To address this issue, it has been proposed to perform processing such as changing the number of reference pixels used depending on the distance between the reference pixel and the predicted pixel (Patent Document 1). [Prior art documents] [Patent documents]
[0005] [Patent Document 1] Patent No. 5044415 [Non-patent literature]
[0006] [Non-Patent Document 1] Eiichiro Okubo (editor), et al., "Impress Standard Textbook Series H.265 / HEVC Textbook," (2013), pp.115-124 Summary of the Invention [Problem to be solved by the invention]
[0007] In intra-frame prediction of HEVC, a representative predictive coding method for video, adjacent pixels to the left and above the transform unit (TU) to be predicted are used as reference pixels, and predicted pixels are generated by interpolation or extrapolation processing using the reference pixels.
[0008] 7 is a diagram showing the positional relationship between a predicted TU (predicted transform unit) 1 with a side size of N (N=4 in the diagram) and reference pixels used for its prediction. If the top left pixel position of predicted TU1 is set as the origin (0,0) and the x and y axes are set as shown, a total of 4N+1 pixels are prepared as reference pixels: a pixel on the left side of predicted TU1 at x=-1 and y=-1 to 2N-1 coordinates and a pixel on the top side of predicted TU1 at y=-1 and x=-1 to 2N-1 coordinates.
[0009] Here, if the pixel at the reference pixel position is undecoded (belongs to an undecoded TU), the pixel value of a nearby decoded reference pixel is substituted for the undecoded reference pixel position. However, because the same pixel is often used consecutively, the reference pixels at these undecoded positions tend to be low-frequency components, and there is a problem that prediction efficiency is likely to decrease if high-frequency components are included in the TU to be predicted.
[0010] Therefore, in consideration of the above-mentioned problems, an object of the present invention is to provide an encoding device, a decoding device, and a program that can accurately predict TUs containing high-frequency components and improve prediction efficiency in intra-screen prediction, even when the pixel at the reference pixel position has not been decoded. [Means for solving the problem]
[0011] In order to solve the above problems, the coding device according to the present invention is a coding device that performs intra-screen prediction of an image, and the intra-screen prediction unit includes a reference pixel preparation unit that prepares reference pixels for a predicted transform unit (TU) and prepares reference pixels equal to decoded neighboring reference pixels at undecoded reference pixel positions, a reference pixel correction unit that corrects frequency components of the reference pixels prepared at the undecoded reference pixel positions, and a prediction processing unit that performs prediction processing of the predicted transform unit based on the corrected reference pixels, and the reference pixel correction unit corrects the frequency power value of the reference pixels prepared at the undecoded reference pixel positions using a frequency power value of a reference pixel in a transform unit to which a decoded neighboring reference pixel used at the undecoded reference pixel positions belongs. death, the reference pixel correction unit obtains a source frequency power value included in each frequency by performing a discrete cosine transform (DCT) on a reference pixel in a transform unit to which a decoded neighboring reference pixel used at an undecoded reference pixel position belongs, and obtains a destination frequency power value included in each frequency by performing a discrete cosine transform (DCT) on the reference pixel prepared at the undecoded reference pixel position, corrects the destination frequency power value with the source frequency power value, and sets the reference pixel at the undecoded reference pixel position by performing an inverse discrete cosine transform (IDCT) on the corrected destination frequency power value; The reference pixel correction unit further corrects the correction destination frequency power value of the reference pixel prepared at an undecoded reference pixel position with the correction source frequency power value, and performs IDCT to multiply the correction amount added to the pixel value of the reference pixel prepared at the undecoded reference pixel position by a weight according to a distance between the undecoded reference pixel position and a decoded neighboring reference pixel used at that position, thereby adjusting the correction amount. The present invention is characterized by the above.
[0013] In addition, it is desirable that the encoding device corrects the destination frequency power value by any one of replacing the destination frequency power value with the source frequency power value, taking an average of the destination frequency power value and the source frequency power value, and merging the destination frequency power value and the source frequency power value at a predetermined ratio for each frequency.
[0015] In order to solve the above problems, a decoding device according to the present invention is a decoding device that performs intra-screen prediction of an image, the intra-screen prediction unit includes a reference pixel preparation unit that prepares reference pixels for a predicted transform unit (TU) and prepares reference pixels equal to decoded neighboring reference pixels at undecoded reference pixel positions, a reference pixel correction unit that corrects frequency components of the reference pixels prepared at the undecoded reference pixel positions, and a prediction processing unit that performs prediction processing of the predicted transform unit based on the corrected reference pixels, and the reference pixel correction unit corrects the frequency power value of the reference pixels prepared at the undecoded reference pixel positions using a frequency power value of a reference pixel in a transform unit to which a decoded neighboring reference pixel used at the undecoded reference pixel positions belongs. death, the reference pixel correction unit obtains a source frequency power value included in each frequency by performing a discrete cosine transform (DCT) on a reference pixel in a transform unit to which a decoded neighboring reference pixel used at an undecoded reference pixel position belongs, and obtains a destination frequency power value included in each frequency by performing a discrete cosine transform (DCT) on the reference pixel prepared at the undecoded reference pixel position, corrects the destination frequency power value with the source frequency power value, and sets the reference pixel at the undecoded reference pixel position by performing an inverse discrete cosine transform (IDCT) on the corrected destination frequency power value; The reference pixel correction unit further corrects the correction destination frequency power value of the reference pixel prepared at an undecoded reference pixel position with the correction source frequency power value, and performs IDCT to multiply the correction amount added to the pixel value of the reference pixel prepared at the undecoded reference pixel position by a weight according to a distance between the undecoded reference pixel position and a decoded neighboring reference pixel used at that position, thereby adjusting the correction amount. The present invention is characterized by the above.
[0016] In order to solve the above problems, the present invention provides a program for causing a computer to function as the encoding device.
[0017] In order to solve the above problems, the present invention provides a program for causing a computer to function as the decoding device. Effect of the Invention
[0018] According to the encoding device, decoding device, and program of the present invention, in intra-screen prediction, even if the pixel at the reference pixel position has not been decoded, it is possible to accurately predict a TU containing high-frequency components, thereby improving prediction efficiency. [Brief description of the drawings]
[0019] [Figure 1] FIG. 2 is a block diagram of an example of an intra-screen prediction unit used in the present invention. [Diagram 2] FIG. 2 is a block diagram of an example of an encoding device according to the present invention. [Diagram 3] FIG. 13 is a diagram showing an example of preparing reference pixels for a predicted TU. [Figure 4]11 is a diagram showing an example of correcting pixel values at undecoded reference pixel positions; FIG. [Diagram 5] 13 is a flowchart illustrating an example of an operation of an intra-screen prediction unit. [Figure 6] FIG. 2 is a block diagram of an example of a decoding device of the present invention. [Figure 7] FIG. 13 is a diagram showing the positional relationship between a predicted TU and reference pixels used for the prediction. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0020] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.
[0021] (Embodiment 1) 1 is a block diagram of an example of an intra-screen prediction unit constituting a part of the encoding device (and decoding device) of the present invention. The intra-screen prediction unit 10 receives a predicted image, performs a prediction process using a decoded TU for each predicted TU, and outputs a predicted image.
[0022] The intra-screen prediction unit 10 includes a reference pixel preparation unit 11 that prepares reference pixels for each predicted TU, a reference pixel correction unit 12 that corrects the reference pixels, and a prediction processing unit 13 that performs prediction processing of the predicted TU based on the corrected reference pixels. Of these, the reference pixel preparation unit 11 and the prediction processing unit 13 perform processing that has been performed in conventional intra-screen prediction, and the reference pixel correction unit 12 performs new processing that has not been performed in conventional intra-screen prediction. In the present invention, the reference pixel correction unit 12 corrects decoded neighboring reference pixel values used at undecoded reference pixel positions, thereby improving prediction accuracy. Note that the intra-screen prediction unit 10 does not need to have such a block configuration, and it is sufficient if it has a function of performing processing of each of the reference pixel preparation unit 11, the reference pixel correction unit 12, and the prediction processing unit 13, which will be described later.
[0023] First, a coding device according to a first embodiment of the present invention will be described. Fig. 2 is a block diagram of an example of a coding device according to the present invention, which is a coding device 100 that performs coding according to H.265 / HEVC, a representative coding method. Note that the coding device 100 according to the present invention is not limited to a device that performs HEVC coding, and may be any coding device that includes an internal intra-frame prediction unit, such as a device that performs H.264 / AVC coding.
[0024] The encoding device 100 includes a division unit 101, a subtraction processing unit 102, a transformation unit 103, a quantization unit 104, an entropy encoding unit 105, an inverse quantization unit 106, an inverse transformation unit 107, an addition processing unit 108, a block memory 109, an intra prediction unit (intra prediction unit) 110, a loop filter 111, a frame memory 112, a motion compensation prediction unit (inter prediction unit) 113, and a switching control unit 114. Of these, the intra prediction unit 10 in Fig. 1 is used as the intra prediction unit 110. The encoding device 100 can be realized by a computer and a program.
[0025] The division unit 101 divides an input image into blocks of a predetermined size. Each divided block (block image) becomes a basic processing unit for prediction and encoding. The size of a block is a maximum of 64×64 pixels, and is further divided into 4×4, 8×8, 16×16, or 32×32 pixels, etc., as a unit of prediction. The blocked image is output to a subtraction processing unit 102, and subsequent processing is performed for each image block.
[0026] The subtraction processing unit 102 performs subtraction processing of a predicted image from the intra-screen prediction unit 110 or the motion compensation prediction unit 113 from the blocked image input from the division unit 101, calculates the difference between the two images, and outputs a residual signal (difference image) to the conversion unit 103.
[0027] The transform unit 103 performs transform coding processing (such as Discrete Cosine Transform (DCT) or Discrete Sine Transform (DST)) defined in H.265 / HEVC or the like on the residual signal input from the subtraction processing unit 102, and outputs the obtained transform coefficients to the quantization unit 104. In the following, a description will be given assuming that DCT is used for the processing.
[0028] The quantization unit 104 performs a quantization process on the transform coefficients input from the transform unit 103 based on a predetermined quantization parameter, and outputs the result (quantized transform coefficients) to the entropy coding unit 105 and the inverse quantization unit .
[0029] The entropy coding unit 105 entropy codes the quantized transform coefficients (encoded image data) input from the quantization unit 104 as well as data necessary for decoding the image to generate a bit stream. This bit stream becomes the output signal of the encoding device 100.
[0030] The inverse quantization unit 106 inverse quantizes the quantized transform coefficients input from the quantization unit 104 , restores them to transform coefficients, and outputs them to the inverse transform unit 107 .
[0031] The inverse transform unit 107 performs inverse transform (such as Inverse Discrete Cosine Transform (IDCT)) on the inversely quantized data (transformation coefficients). That is, the inverse quantization unit 106 and the inverse transform unit 107 perform a decoding process that is the opposite of the encoding process performed by the transform unit 103 and the quantization unit 104, and output the result to the addition processing unit 108.
[0032] The addition processing unit 108 adds the data inversely quantized and inversely transformed by the inverse quantization unit 106 and the inverse transformation unit 107, i.e., the decoded data of the residual signal, to predicted image data processed by the intra-screen prediction unit 110 or the motion compensation prediction unit 113 described below, and outputs the synthesized image data (reconstructed block image) to the block memory 109 and the loop filter 111.
[0033] The block memory 109 is a storage unit that accumulates the reconstructed block image input from the addition processing unit 108. The block memory 109 stores a predicted image for intra prediction and outputs it to the intra prediction unit 110.
[0034] The intra prediction unit 110 generates a predicted image by intra prediction (intra prediction) of the predicted TU based on the data of the predicted image stored in the block memory 109. In the present invention, the intra prediction unit 110 has the configuration of the intra prediction unit 10 shown in FIG. 1, and generates a predicted image that corrects reference pixels. The process will be described later. The intra prediction unit 110 outputs the intra prediction image to the switching control unit 114.
[0035] The loop filter 111 is configured, for example, by a deblocking filter that reduces block distortion and a sample adaptive offset that reduces ringing distortion. By performing filtering on the image data combined by the addition unit 108, the coding distortion generated by the quantization processing in the coding loop is reduced. The loop filter 111 outputs the filtered image data to the frame memory 112.
[0036] The frame memory 112 accumulates the image data processed by the loop filter 111. The frame memory 112 functions as a storage unit that stores reference pictures used in motion compensation prediction (inter prediction).
[0037] The motion compensation prediction unit 113 performs motion compensation prediction processing (inter-frame prediction based on motion vectors) based on the images stored in the frame memory 112 and information on the image motion obtained by a motion detection unit (not shown), and outputs the motion prediction image to the switching control unit 114.
[0038] The switching control unit 114 selects and outputs to the subtraction processing unit 102 either the intra prediction image from the intra prediction unit 110 or the motion prediction image from the motion compensation prediction unit 113 .
[0039] The encoding device 100 of the present invention can improve the prediction efficiency of intra prediction by correcting reference pixels in the intra prediction unit 110.
[0040] Next, the operation of the intra prediction unit 110 (10) will be described in detail. The reference pixel preparation unit 11 in Fig. 1 receives a predicted image, prepares and outputs reference pixels used in the prediction process of the predicted TU. Fig. 3 shows an example of preparing reference pixels for predicted TU1.
[0041] The size of one side of a TU is set to N. Then, as shown in FIG. 3, 4N+1 pixels adjacent to TU1 to be predicted are prepared as reference pixels p[x][y]. The arrangement relationship of the 4N+1 reference pixels with respect to the predicted TU1 is the same as that in FIG. 7. In FIG. 3, the top left pixel (pixel) position of the predicted TU1 (N=4) is set to coordinates (12, 12), and a total of 17 pixels are prepared as reference pixels, including the pixel at coordinates x=11, y=11 to 19 on the left side of the predicted TU1 and the pixel at coordinates x=11 to 19 on the top side of the predicted TU1.
[0042] The procedure for preparing the reference pixels is as follows. (1) If all reference pixels have been decoded, they are used as the values of p[x][y] as is.
[0043] (2) If there are no decoded pixels, set p[x][y]=1<<(BitDepth-1) (BitDepth is the bit depth).
[0044] (3) In other cases (when only part of the data has been decrypted), (3-1) Scan the reference pixels in order from the bottom left corner (11,19) in Figure 3 to the top right corner (19,11), and use the first decoded pixel value (here, p
[11]
[15] ) as the value of the bottom left corner (p
[11]
[19] ). (3-2) In Figure 3, scan from (11,18) to (11,11) in order, and if the pixel is not decoded, set the pixel value just below it as the pixel value. In other words, set p
[11] [y+1] as the value of p
[11] [y]. (3-3) Similarly, scan from (12,11) to the top right corner (19,11) in Figure 3 in order, and if the pixel is not decoded, set the pixel value to the immediate left of it as the pixel value of the pixel. In other words, set p[x-1]
[11] as the value of p[x]
[11] .
[0045] By the above steps (3-1) and (3-2), the values of the undecoded reference pixels p
[11]
[19] to p
[11]
[16] become equal to the adjacent decoded reference pixel p
[11]
[15] . By the above steps, values are set for all 4N+1 reference pixels and prepared as p[x][y].
[0046] Next, the process of the reference pixel correction unit 12 will be described. The reference pixel correction unit 12 corrects pixel values at undecoded reference pixel positions among the reference pixels prepared by the reference pixel preparation unit 11. Fig. 4 shows an example of correcting pixel values at undecoded reference pixel positions. For example, decoded neighboring reference pixels used at undecoded reference pixel positions, a predicted image, and TU information (TU partition information) are input to the reference pixel correction unit 12, and the corrected reference pixels are output.
[0047] The reference pixel correction unit 12 first analyzes the frequency power of the reference pixels of a decoded TU (adjacent decoded TU5) having decoded neighboring reference pixels used for undecoded reference pixel positions. Now, as a result of the processing of the reference pixel preparation unit 11, the reference pixel values p
[11]
[19] to p
[11]
[16] are the values of the decoded neighboring reference pixel p
[11]
[15] . The reference pixel correction unit 12 performs frequency component analysis (e.g., DCT analysis) on the frequency power of the reference pixels in the TU (decoded TU5) to which the decoded neighboring reference pixel p
[11]
[15] originally belongs, and outputs the result as frequency power analysis information.
[0048] That is, the frequency power contained in the reference pixels p
[11]
[15] to p
[11]
[12] of the decoded TU5 is calculated by DCT analysis. The DCT calculation formula is shown below.
[0049]
number
[0050] In FIG. 4, since N=4 in the DCT of p
[11]
[15] to p
[11]
[12] of the decoded TU5 including the reference pixel p
[11]
[15] of the correction source, the power coefficient of the kth frequency: X k (correction source), (k = 0, 1, 2, 3) is output. This coefficient is sometimes called the "frequency power value".
[0051] Next, the pixel value used at the undecoded reference pixel position is corrected. First, the frequency power of the pixel at the undecoded reference pixel position (which is now equal to the decoded neighboring reference pixel) is subjected to frequency component analysis (for example, DCT analysis). That is, p
[11]
[19] to p
[11]
[16] , which are the correction destinations, are subjected to DCT analysis to obtain the power coefficient of the kth frequency: X k (correction destination), (k = 0, 1, 2, 3) is obtained. Note that if the pixel values used at the undecoded reference pixel positions are all equal, the power coefficient will be only the direct current (DC) component, and the other frequency components will be zero.
[0052] And the resulting frequency power value: X k (Correction destination) is the frequency power value mentioned above: X k (correction source) is used for correction. For example, X k (Correction destination) to X k Replace with (correction source), X k (Correction destination) and X k Take the average of each element of (the correction source), or X k (Correction destination) and X k (the correction source) can be merged for each element (each frequency) at a predetermined ratio. The predetermined ratio may be changed for each frequency. The correction method may be selected based on the characteristics of the image, etc.
[0053] If the number of reference pixels in the TU to which the undecoded reference pixel position belongs differs from the number of reference pixels in the TU to which the decoded neighboring reference pixels belong, kFor example, X k (Base correction) is N=4, X k When (correction destination) is N=2, the average of X0 (correction source) and X1 (correction source) can be used as X0 (correction destination), and the average of X2 (correction source) and X3 (correction source) can be used as X1 (correction destination).
[0054] As described above, by correcting the frequency power value of the undecoded reference pixel position and then performing IDCT, a corrected pixel value to be used at the undecoded reference pixel position is obtained. Therefore, it is possible to correct the frequency component (frequency power) of the pixel value at the undecoded reference pixel position. In other words, it is possible to perform a correction by adding the frequency component of a nearby decoded transform unit to the reference pixel at the undecoded reference pixel position.
[0055] Furthermore, after the above correction, a weight α can be multiplied according to the distance between the undecoded reference pixel position and the decoded neighboring reference pixel used at that position. For example, p
[11]
[16] is close to p
[11]
[15] , so a large weight (α=1.0) is applied, and p
[11]
[19] is far from p
[11]
[15] , so a small weight (α=0.7) is applied. In this manner, the correction amount at the undecoded reference pixel position can be adjusted by multiplying the correction amount of the pixel value by a weight according to the distance. Note that, before performing IDCT, the DCT components may be multiplied by a weight according to the distance to adjust the correction amount.
[0056] In this embodiment, the reference pixel preparation unit 11 is based on HEVC, and performs correction on the reference pixels of the undecoded TU prepared by processing to make all pixel values of the undecoded reference pixel positions p
[11]
[19] to p
[11]
[16] equal to the value of the decoded neighboring reference pixel p
[11]
[15] , but the correction by the reference pixel correction unit 12 is to correct the pixels of the undecoded reference pixel positions by adding frequency components of the neighboring decoded TUs, so the reference pixel preparation processing is not limited to this. By performing the same correction on the pixels of the undecoded reference pixel positions prepared by other processing, it is possible to obtain the effect of improving prediction efficiency.
[0057] Next, the process of the prediction processing unit 13 will be described. The process of the prediction processing unit 13 can use intra-plane prediction process performed in conventional encoding methods. For example, if the encoding method is HEVC, any one of (1) planar prediction, (2) direct current (DC) prediction, and (3) directional prediction is performed. The prediction process is performed using the corrected reference pixel value output from the reference pixel correction unit 12.
[0058] Here, (1) planar prediction is a method for smoothly generating each predicted pixel value based on the four reference pixel values surrounding the predicted TU, (2) direct current (DC) prediction is a process for filling the inside of the predicted TU with the average value of 2N reference pixels immediately to the left and above the predicted TU, and (3) directional prediction is a process for predicting a reference pixel value (or an internal division value) in a reference direction based on one of 33 reference directions, and a method with good prediction efficiency can be selected from these prediction methods to make a prediction.
[0059] Next, a more generalized explanation will be given of the processing of the intra-screen prediction unit 10. Fig. 5 is a flowchart showing an example of the operation of the intra-screen prediction unit 10. Each step will be explained below.
[0060] Step S11: For the predicted TU (size of one side is N), prepare 4N+1 adjacent pixels (see FIG. 7) as reference pixels p[x][y]. If all reference pixels have been decoded, they are used as the value of p[x][y] as is, and if there is no decoded pixel, p[x][y]=1<<(BitDepth-1). Otherwise, scan the reference pixels in order from the lower left end (-1, 2N-1) to the upper right end (2N-1, -1), and use the first decoded pixel value as the value of p[-1][2N-1]. Next, scan from (-1, 2N-2) one above the lower left end to (-1, -1) in order, and if (-1, y) has not been decoded, use p[-1][y+1] as the value of p[-1][y]. Furthermore, it scans sequentially from (0, -1) to the upper right end (2N-1, -1), and if (x, -1) has not been decoded, p[x-1][-1] is set to the value of p[x][-1].
[0061] Step S12: The frequency power of the reference pixels in the TU (decoded TU5 adjacent to undecoded TU6) to which the decoded neighboring reference pixel used at the undecoded reference pixel position originally belongs is subjected to DCT analysis, and the result is output as frequency power analysis information (power coefficients for each frequency of the correction source) (see FIG. 4).
[0062] Step S13: DCT analysis is performed on the pixel value of the undecoded reference pixel position to be corrected, and the obtained frequency power analysis information (power coefficient for each frequency of the correction destination) is corrected based on the frequency power analysis information of the reference pixel of the decoded TU obtained in step S12 (see FIG. 4). In the correction, for example, the coefficient sequence (frequency power value) of the correction destination may be replaced with the coefficient sequence (frequency power value) of the correction source, the average of the correction destination frequency power value and the correction source frequency power value may be taken, and the power values of each frequency of the correction destination and the correction source may be merged at a predetermined ratio. Then, the corrected correction destination frequency power value is subjected to IDCT to obtain the corrected reference pixel value to be used at the undecoded reference pixel position. Note that the correction amount may be weighted based on the distance between the correction destination and correction source pixels.
[0063] Step S14: Filter processing such as a linear filter or a smoothing filter is performed on the reference pixels (including the corrected reference pixels) as necessary. Note that this filter processing is a process for reducing distortion and noise, and is a normal process performed in HEVC and the like.
[0064] Step S15: Prediction processing of the predicted TU is performed based on the reference pixels obtained in steps S11 to S14. For example, in HEVC, planar prediction or direct current (DC) prediction is provided as directional prediction having 33 reference directions and non-directional prediction, and intra-frame prediction is performed by these prediction processes.
[0065] In this embodiment, the DCT analysis is performed as the frequency component analysis, but the present invention is not limited to this, and other methods for obtaining frequency power coefficients, such as DST analysis, may be used.
[0066] (Embodiment 2) A decoding device will be described as embodiment 2. The intra prediction unit 10 is used not only in the local decoding process in the encoding device, but also in the decoding process in the decoding device.
[0067] 6 is a block diagram of an example of a decoding device of the present invention, which is a decoding device 200 that performs decoding by H.265 / HEVC, a representative encoding method. Note that the decoding device 200 of the present invention is not limited to a device that performs decoding by HEVC, and may be any decoding device that includes an internal intra-frame prediction unit, such as a device that performs decoding by H.264 / AVC. The decoding device 200 inputs a bit stream that is encoded data, performs a decoding process, and outputs a restored image.
[0068] The decoding device 200 includes an entropy decoding unit 201, an inverse quantization unit 202, an inverse transformation unit 203, an addition processing unit 204, a loop filter 205, a block memory 206, an intra prediction unit 207, a frame memory 208, a motion compensation prediction unit 209, and a switching control unit 210. Of these, the intra prediction unit 10 in Fig. 1 is used as the intra prediction unit 207. The decoding device 200 can be realized by a computer and a program.
[0069] The entropy decoding unit 201 entropy decodes the input bit stream and decodes the transmitted coded data of the image as well as data necessary for decoding the image. The coded data of the image is output to the inverse quantization unit 202 in units of blocks.
[0070] The inverse quantization unit 202 inverse quantizes the coded data (quantized coefficients) of the image input from the entropy decoding unit 201. This inverse quantization produces transform coefficients generated by the transform coding process (discrete cosine transform, etc.) on the coding side. The inverse quantization unit 202 then outputs the inverse quantized data to the inverse transform unit 203.
[0071] The inverse transform unit 203 performs inverse transform (such as inverse discrete cosine transform) on the inversely quantized data (transformation coefficients). That is, the inverse quantization unit 202 and the inverse transform unit 203 perform decoding processing that is the opposite of the processing performed by the transform unit 103 and the quantization unit 104 in the encoding device 100, and output the result to the addition processing unit 204.
[0072] The addition processing unit 204 adds the data inversely quantized and inversely transformed by the inverse quantization unit 202 and the inverse transformation unit 203 (this corresponds to the differential image data) to predicted image data processed by the intra-screen prediction unit 207 (or motion compensation prediction unit 209) described below, and outputs the synthesized image data (reconstructed block image) to the loop filter 205 and the block memory 206.
[0073] The loop filter 205 performs sample adaptive offset processing, deblocking filter processing, or the like, on the image data input from the addition processing unit 204. The loop filter 205 outputs the filtered image data to the frame memory 208, and also outputs the image data as an output image (reconstructed image) of the decoding device 200.
[0074] The block memory 206 is a storage unit that sequentially accumulates the composite image data (reconstructed block image) generated by the addition processing unit 204. A predicted image for intra prediction is stored in the block memory 206, and is output to the intra prediction unit 207.
[0075] The intra prediction unit 207 generates a predicted image by intra prediction (intra prediction) of the predicted TU based on the data of the predicted image stored in the block memory 206. In the present invention, the intra prediction unit 207 has the configuration of the intra prediction unit 10 shown in FIG. 1, and generates a predicted image by correcting reference pixels. The process is as described above. Note that, on the decoding side, information on what kind of prediction process was performed on the encoding side is sent, and prediction is performed based on that information. The intra prediction unit 207 outputs the intra prediction image to the switching control unit 210.
[0076] The frame memory 208 accumulates the image data processed by the loop filter 205. The frame memory 208 functions as a storage unit that stores reference pictures used in motion compensation prediction (inter prediction).
[0077] The motion compensation prediction unit 209 performs motion compensation prediction processing based on the image stored in the frame memory 208 and image motion information obtained by a motion detection unit (not shown), and outputs a motion prediction image to a switching control unit 210.
[0078] The switching control unit 210 selects and outputs to the addition processing unit 204 either the intra prediction image from the intra prediction unit 207 or the motion prediction image from the motion compensation prediction unit 209 .
[0079] The decoding device 200 of the present invention can improve the prediction efficiency of intra prediction by correcting reference pixels in the intra prediction unit 207.
[0080] In the above embodiment, the configurations and operations of the encoding device 100 and the decoding device 200 have been described, but the present invention is not limited thereto, and may be configured as an encoding method or a decoding method for correcting reference pixels in intra-frame prediction. Alternatively, the present invention may be configured as an intra-frame prediction method for images based on the flowchart of FIG.
[0081] A computer can be suitably used to function as the above-mentioned encoding device 100 or decoding device 200, and such a computer can be realized by storing a program describing the processing contents for realizing each function of the encoding device 100 or decoding device 200 in a storage unit of the computer, and reading and executing this program by the CPU of the computer. Note that this program can be recorded on a computer-readable recording medium.
[0082] Although the above-mentioned embodiment has been described as a representative example, it is obvious to those skilled in the art that many modifications and substitutions can be made within the spirit and scope of the present invention. Therefore, the present invention should not be interpreted as being limited by the above-mentioned embodiment, and various modifications and changes are possible without departing from the scope of the claims. For example, it is possible to combine multiple building blocks described in the embodiment into one, or to divide one building block. [Explanation of symbols]
[0083] 1 Predicted TU 2~5 Decrypted TU 6 Undecoded TU 10 In-screen prediction section 11 Reference pixel preparation unit 12 Reference pixel correction section 13 Prediction processing section 100 Encoding device 101 Split section 102 Subtraction processing unit 103 Conversion unit 104 Quantization section 105 Entropy coding unit 106 Inverse quantization section 107 Reverse conversion unit 108 Addition processing unit 109 Block Memory 110 Intra-frame prediction unit (intra-prediction unit) 111 Loop Filter 112 Frame Memory 113 Motion compensation prediction unit (inter prediction unit) 114 Switching control section 200 Decryption device 201 Entropy Decoding Unit 202 Inverse quantization section 203 Inverse conversion unit 204 Addition processing unit 205 Loop Filter 206 Block Memory 207 In-screen prediction section 208 Frame Memory 209 Motion compensation prediction unit 210 Switching control section
Claims
1. In a coding device that performs intra-frame prediction of an image, The in-screen prediction unit A reference pixel preparation unit that prepares reference pixels for a predicted transform unit (TU) and prepares reference pixels equal to decoded neighboring reference pixels at undecoded reference pixel positions; a reference pixel correction unit that corrects a frequency component of the reference pixel prepared at an undecoded reference pixel position; a prediction processing unit that performs prediction processing on the predicted conversion unit based on the corrected reference pixels; Equipped with The reference pixel correction unit corrects a frequency power value of a reference pixel prepared at an undecoded reference pixel position by using a frequency power value of a reference pixel in a transform unit to which a decoded neighboring reference pixel used at an undecoded reference pixel position belongs, the reference pixel correction unit obtains a source frequency power value included in each frequency by performing a discrete cosine transform (DCT) on a reference pixel in a transform unit to which a decoded neighboring reference pixel used at an undecoded reference pixel position belongs, and obtains a destination frequency power value included in each frequency by performing a DCT on the reference pixel prepared at an undecoded reference pixel position; correcting the correction destination frequency power value with the correction source frequency power value; The corrected frequency power value is subjected to IDCT (inverse discrete cosine transform) to set the reference pixel at the undecoded reference pixel position; the reference pixel correction unit further corrects the destination frequency power value of the reference pixel prepared at an undecoded reference pixel position with the source frequency power value, and performs IDCT to multiply the correction amount added to the pixel value of the reference pixel prepared at the undecoded reference pixel position by a weight according to a distance between the undecoded reference pixel position and a decoded neighboring reference pixel used at that position, thereby adjusting the correction amount.
2. 2. The encoding device according to claim 1, A coding device, characterized in that the correction destination frequency power value is corrected by any one of replacing the correction destination frequency power value with the correction source frequency power value, taking an average of the correction destination frequency power value and the correction source frequency power value, and merging the correction destination frequency power value and the correction source frequency power value at a predetermined ratio for each frequency.
3. In a decoding device that performs intra-frame prediction of an image, The in-screen prediction unit A reference pixel preparation unit that prepares reference pixels for a predicted transform unit (TU) and prepares reference pixels equal to decoded neighboring reference pixels at undecoded reference pixel positions; a reference pixel correction unit that corrects a frequency component of the reference pixel prepared at an undecoded reference pixel position; a prediction processing unit that performs prediction processing on the predicted conversion unit based on the corrected reference pixels; Equipped with The reference pixel correction unit corrects a frequency power value of a reference pixel prepared at an undecoded reference pixel position by using a frequency power value of a reference pixel in a transform unit to which a decoded neighboring reference pixel used at an undecoded reference pixel position belongs, the reference pixel correction unit obtains a source frequency power value included in each frequency by performing a discrete cosine transform (DCT) on a reference pixel in a transform unit to which a decoded neighboring reference pixel used at an undecoded reference pixel position belongs, and obtains a destination frequency power value included in each frequency by performing a DCT on the reference pixel prepared at an undecoded reference pixel position; correcting the correction destination frequency power value with the correction source frequency power value; The corrected frequency power value is subjected to IDCT (inverse discrete cosine transform) to set the reference pixel at the undecoded reference pixel position; the reference pixel correction unit further corrects the destination frequency power value of the reference pixel prepared at an undecoded reference pixel position with the source frequency power value, and performs IDCT to multiply the correction amount added to the pixel value of the reference pixel prepared at the undecoded reference pixel position by a weight according to a distance between the undecoded reference pixel position and a nearby decoded reference pixel used at that position, thereby adjusting the correction amount.
4. A program causing a computer to function as the encoding device according to claim 1 or 2.
5. A program causing a computer to function as the decoding device according to claim 3.
Citation Information
Patent Citations
JP1975044415A
Intra prediction system for moving image encoding apparatus and moving image decoding apparatus
JP2008245088A
