Intra prediction method and encoder and decoder using the same
The intra prediction method improves video compression efficiency by deriving boundary sample values from reference and neighboring samples, addressing the high cost of transmitting and storing high-resolution images.
Patent Information
- Application Number
- JP2025002775
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2011-04-25
- Filing Date
- 2025-01-08
- Publication Date
- 2025-10-06
- Estimated Expiration
- 2032-04-20
AI Technical Summary
The increasing demand for high-resolution and high-quality images leads to increased transmission and storage costs due to the large amount of information, necessitating a more efficient video compression technique.
An intra prediction method that derives values of specific boundary samples of a predicted block based on reference samples located in the prediction direction and neighboring samples, improving prediction efficiency and accuracy.
Enhances the efficiency and accuracy of video compression by effectively predicting pixel values in high-resolution and high-quality images, reducing transmission and storage costs.
Smart Images

Figure 0007749871000016 
Figure 0007749871000017 
Figure 0007749871000018
Abstract
Description
[Technical Field]
[0001] The present invention relates to an intra prediction method in a video encoder and decoder, and more particularly to a method for deriving values of specific boundary samples of a predicted block for a current block and an apparatus using the same. [Background technology]
[0002] Recently, the demand for high-resolution and high-quality images has been increasing in various application fields, but as images become higher in resolution and quality, the amount of information pertaining to the images also increases.
[0003] When transmitting a large amount of high-resolution, high-quality video using existing media such as wired or wireless broadband lines or storing it using existing storage media, the transmission and storage costs increase. Therefore, a highly efficient video compression technology can be used to effectively transmit, store, or play back high-resolution, high-quality video.
[0004] To improve the efficiency of video compression, inter prediction and intra prediction can be used.
[0005] Inter prediction predicts pixel values in a current picture from previous and / or subsequent pictures, while intra prediction predicts pixel values in the current picture using pixel information in the current picture.
[0006] In addition to inter-prediction and intra-prediction, weighted prediction can be used to prevent degradation of image quality due to changes in lighting, and entropy coding can be used to assign short codes to symbols that appear frequently and long codes to symbols that appear less frequently. Summary of the Invention [Problem to be solved by the invention]
[0007] SUMMARY OF THE INVENTION It is an object of the present invention to provide an effective video compression technique and an apparatus for utilizing the same.
[0008] Another object of the present invention is to provide an intra prediction method and an apparatus using the same that can increase prediction efficiency.
[0009] Another object of the present invention is to provide a method for deriving values of specific boundary samples of a predicted block for a current block and an apparatus using the same. [Means for solving the problem]
[0010] According to an embodiment of the present invention, there is provided an intra prediction method, the intra prediction method including the steps of deriving a prediction mode of a current block and generating a predicted block for the current block based on the prediction mode of the current block. When the prediction mode of the current block is an intra_Angular prediction mode, values of boundary samples not located in a prediction direction of the intra_Angular prediction mode, among left boundary samples and upper boundary samples of the predicted block, are derived based on reference samples located in the prediction direction and neighboring reference samples.
[0011] When the intra-directional prediction mode is a vertical prediction mode, the value of the left boundary sample is derived based on the upper reference sample of the left boundary sample and the adjacent reference sample, and the value of the prediction sample that is not the left boundary sample is derived as the value of the upper reference sample of the prediction sample.
[0012] When the intra-directional prediction mode is a vertical prediction mode, the value of the left boundary sample is derived based on the reference sample above the left boundary sample, the reference sample adjacent to the left boundary sample, and the reference sample adjacent to the upper left of the current block.
[0013] When the intra-directional prediction mode is a horizontal prediction mode, the value of the top boundary sample is derived based on the reference sample to the left of the top boundary sample and the adjacent reference sample, and the value of a prediction sample that is not a top boundary sample is derived as the value of the reference sample to the left of the prediction sample.
[0014] When the intra directional prediction mode is a horizontal prediction mode, the value of the top boundary sample is derived based on a reference sample to the left of the top boundary sample, a reference sample adjacent to the top boundary sample, and a reference sample adjacent to the upper left of the current block.
[0015] When the prediction direction of the intra directional prediction mode is right-upward, the value of the left boundary sample is derived based on the reference sample located in the prediction direction and the neighboring reference samples.
[0016] When the prediction direction of the intra directional prediction mode is bottom-left, the value of the upper boundary sample is derived based on the reference sample located in the prediction direction and the neighboring reference samples.
[0017] According to another embodiment of the present invention, there is provided an image encoding apparatus, the image encoding apparatus including: a prediction unit that generates a prediction block for a current block based on a prediction mode of the current block; and an entropy encoding unit that encodes information on the prediction block. When the prediction mode of the current block is an intra_Angular prediction mode, the prediction unit derives values of boundary samples that are not located in a prediction direction of the intra_Angular prediction mode, among left and top boundary samples of the prediction block, based on reference samples located in the prediction direction and adjacent reference samples.
[0018] According to another embodiment of the present invention, there is provided a video decoding apparatus, the video decoding apparatus including: an entropy decoder for entropy-decoding information received from an encoder; and a predictor for generating a predicted block for a current block based on the entropy-decoded information. When a prediction mode of the current block is an intra_Angular prediction mode, the predictor derives values of boundary samples not located in a prediction direction of the intra_Angular prediction mode, among left and top boundary samples of the predicted block, based on reference samples located in the prediction direction and adjacent reference samples.
[0019] When the intra-directional prediction mode is a vertical prediction mode, the predictor derives the value of the left boundary sample based on the reference sample above the left boundary sample and the adjacent reference samples.
[0020] When the intra directional prediction mode is a vertical prediction mode, the prediction unit derives the value of the left boundary sample based on the reference sample above the left boundary sample, the reference sample adjacent to the left boundary sample, and the reference sample adjacent to the upper left of the current block.
[0021] When the intra-directional prediction mode is a horizontal prediction mode, the prediction unit derives the value of the top boundary sample based on the reference sample to the left of the top boundary sample and the adjacent reference samples.
[0022] When the intra directional prediction mode is a horizontal prediction mode, the prediction unit derives the value of the upper boundary sample based on a reference sample to the left of the upper boundary sample, a reference sample adjacent to the upper boundary sample, and a reference sample adjacent to the upper left of the current block. [Effects of the Invention]
[0023] According to the present invention, it is possible to increase the efficiency of intra prediction and improve the performance of video compression.
[0024] According to the present invention, the accuracy of the values of predicted samples located adjacent to the reference sample can be improved. [Brief explanation of the drawings]
[0025] [Figure 1] 1 is a block diagram illustrating a video encoder according to an embodiment of the present invention; [Figure 2] 1 is a block diagram illustrating a video decoder according to an embodiment of the present invention; [Figure 3] 1 is a flow diagram that outlines a method for intra prediction in a video decoder. [Figure 4] Indicates the prediction direction in intra prediction mode. [Figure 5] This indicates that the current block is coded in the Intra_DC prediction mode. [Figure 6] 10 illustrates a case where the prediction direction is vertical in an intra prediction mode according to an embodiment of the present invention. [Figure 7] 10 illustrates a case where the prediction direction is horizontal in an intra prediction mode according to an embodiment of the present invention. [Figure 8] This is an example in which intra prediction modes are classified according to prediction directions. [Figure 9] 10 illustrates a case where the prediction direction is upper right in an intra prediction mode according to an embodiment of the present invention. [Figure 10] 1 illustrates a case where the prediction direction is bottom-left in an intra prediction mode according to an embodiment of the present invention. [Figure 11] 10 illustrates a case where the prediction direction is vertical in an intra prediction mode according to another embodiment of the present invention. [Figure 12] 10 illustrates a case where the prediction direction is horizontal in an intra prediction mode according to another embodiment of the present invention. [Figure 13] 1 is a diagram for explaining the operation of an encoder in a system to which the present invention is applied; [Figure 14] 1 is a diagram for explaining the operation of a decoder in a system to which the present invention is applied; DETAILED DESCRIPTION OF THE INVENTION
[0026] The present invention may have various embodiments, and specific embodiments will be described with reference to the drawings. However, the present invention is not limited to the specific embodiments described, and various modifications are possible within the scope of the technical concept of the present invention.
[0027] The terms used in this specification are merely used to describe specific embodiments and are not used to limit the technical idea of the present invention. The singular expressions include the plural expressions unless the context clearly indicates otherwise.
[0028] Meanwhile, each component in the drawings described in the present invention is illustrated independently for the convenience of explaining the different characteristic functions of the video encoder / decoder, and does not mean that each component is implemented by separate hardware or software. For example, two or more components may be integrated into one component, or one component may be divided into multiple components. Embodiments in which each component is integrated / separated are also within the scope of the present invention as long as they do not deviate from the essence of the present invention.
[0029] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the accompanying drawings. In the following drawings, the same reference numerals are used to refer to the same components, and redundant description of the same components will be omitted.
[0030] 1 is a block diagram illustrating a video encoder according to an embodiment of the present invention. Referring to FIG. 1, the video encoder 100 includes a picture division unit 105, a prediction unit 110, a transform unit 115, a quantization unit 120, a reordering unit 125, an entropy encoding unit 130, a dequantization unit 135, an inverse transform unit 140, a filter unit 145, and a memory 150.
[0031] The picture division unit 105 can divide an input picture into at least one processing unit, where the processing unit may be a prediction unit (Prediction Unit), a transform unit (TU), or a coding unit (CU).
[0032] The prediction unit 110 may include an inter prediction unit that performs inter prediction and an intra prediction unit that performs intra prediction. The prediction unit 110 performs prediction on a processing unit of a picture divided by the picture division unit 105 to generate a prediction block. At this time, the processing unit of a picture may be a coding unit, a transform unit, or a prediction unit. The prediction unit 110 also determines whether to perform inter prediction or intra prediction on the corresponding processing unit, and performs prediction according to the determined prediction method. At this time, the unit for which the prediction method is determined may be different from the unit for which prediction is performed. For example, the unit for which the prediction method is determined is a prediction unit, and the unit for which prediction is performed is a transform unit.
[0033] In inter prediction, prediction is performed based on information about at least one picture before and after the current picture to generate a predicted block, while in intra prediction, prediction is performed based on pixel information within the current picture to generate a predicted block.
[0034] In inter prediction, a reference picture is selected for the current block, and a reference block of the same size as the current block can be selected in integer pixel units. A predicted block is then generated that minimizes the residual value and motion vector magnitude from the current block. Inter prediction methods include skip mode, merge mode, and MVP (Motion Vector Prediction). The predicted block can also be generated in sub-integer units such as 1 / 2 or 1 / 4. In this case, the motion vector can also be expressed in sub-integer pixel units. For example, the luma component can be expressed in 1 / 4 pixel units, and the chroma component can be expressed in 1 / 8 pixel units. Information such as the index of the reference picture selected through inter prediction, the motion vector, and the residual signal are entropy coded and transmitted to the decoder.
[0035] In intra prediction, a prediction mode is determined by a prediction unit, and prediction is performed by a prediction unit or a transform unit. In addition, intra prediction can support 33 directional prediction modes and at least two non-directional prediction modes. In this case, the non-directional prediction modes can include a DC prediction mode and a planar mode.
[0036] Meanwhile, in this specification, using a sample means using information of the corresponding sample, for example, pixel values, etc. However, for convenience of explanation, it should be noted that the expressions "using sample information" or "using pixel values" can also be simply expressed as "using a sample."
[0037] A prediction unit may have various sizes / shapes. For example, in the case of inter prediction, the prediction unit may have a size such as 2N×2N, 2N×N, N×2N, or N×N, and in the case of intra prediction, the prediction unit may have a size such as 2N×2N or N×N. In this case, the prediction unit of N×N size may be set to be applied only in specific cases. For example, the prediction unit of N×N size may be set to be available only for the minimum size coding unit, or may be set to be available only for intra prediction. In addition to the prediction units of the above sizes, prediction units of sizes such as N×mN, mN×N, 2N×mN, or mN×2N (m<1) may be defined and used.
[0038] A residual block between the generated prediction block and the original block is input to the transform unit 115. In addition, information on the prediction mode, prediction unit, motion vector, etc. used for prediction is coded by the entropy coding unit 130 and transmitted to the decoder.
[0039] The transform unit 115 performs a transform on the residual block to generate transform coefficients. The processing unit in the transform unit 115 is a transform unit, which may have a quad tree structure. The size of the transform unit may be determined within a predetermined maximum and minimum size range. The transform unit 115 transforms the residual block using a discrete cosine transform (DCT), a discrete sine transform (DST), etc.
[0040] The quantization unit 120 generates quantized coefficients by quantizing the transform coefficients generated by the transform unit 115. The quantized coefficients generated by the quantization unit 120 are transmitted to the reordering unit 125 and the inverse quantization unit 135.
[0041] The rearrangement unit 125 rearranges the quantized coefficients transmitted from the quantization unit 120. By rearranging the quantized coefficients, the efficiency of encoding in the entropy encoding unit 130 can be improved. The rearrangement unit 125 rearranges the quantized coefficients in a two-dimensional block format into a one-dimensional vector format through a coefficient scanning method. The rearrangement unit 125 can also improve the entropy encoding efficiency in the entropy encoding unit 130 by changing the order of coefficient scanning based on the probabilistic statistics of the quantized coefficients transmitted from the quantization unit 120.
[0042] The entropy coding unit 130 performs entropy coding on the quantized coefficients reordered by the reordering unit 125. In this case, coding methods such as Exponential Golomb and CABAC (Context-Adaptive Binary Arithmetic Coding) can be used. The entropy coding unit 130 also encodes various information transmitted from the prediction unit 110, such as block type information, prediction mode information, partition unit information, prediction unit information, transmission unit information, motion vector information, reference picture information, block interpolation information, and filtering information.
[0043] In addition, the entropy coding unit 130 can make certain changes to the parameter set or syntax to be transmitted, if necessary.
[0044] The inverse quantization unit 135 inversely quantizes the values quantized by the quantization unit 120, and the inverse transform unit 140 inversely transforms the values inversely quantized by the inverse quantization unit 135. The residual block reconstructed by the inverse quantization unit 135 and the inverse transform unit 140 is integrated with the predicted block generated by the prediction unit 110 to generate a reconstructed block.
[0045] The filter unit 145 can apply a deblocking filter, an adaptive loop filter (ALF), a sample adaptive offset (SAO), etc. to the reconstructed picture.
[0046] A deblocking filter can remove block distortion that occurs at the boundaries between blocks in a restored picture. ALF can perform filtering based on the value obtained by comparing the restored image filtered by the deblocking filter with the original image. ALF can also be operated only when high efficiency is applied. SAO restores the offset difference between the original image and the pixel unit for the residual block to which the deblocking filter is applied, and is applied in the form of band offset, edge offset, etc.
[0047] On the other hand, filtering may not be performed on reconstruction blocks used in inter prediction.
[0048] The memory 150 may store reconstructed blocks or pictures, which may be transferred to the prediction unit 110 that performs inter prediction.
[0049] 2 is a block diagram illustrating a video decoder according to an embodiment of the present invention. Referring to FIG. 2, the video decoder 200 includes an entropy decoding unit 210, a reordering unit 215, an inverse quantization unit 220, an inverse transform unit 225, a prediction unit 230, a filter unit 235, and a memory 240.
[0050] When a video bitstream generated by a video encoder is input, the video decoder can decode the bitstream according to the procedure in which the video information was processed by the video encoder.
[0051] For example, if the video encoder performs entropy encoding using CABAC, the entropy decoder 210 performs entropy decoding using CABAC accordingly.
[0052] The residual signal entropy decoded by the entropy decoding unit 210 is transmitted to the realignment unit 215, and information for generating a prediction block from the entropy decoded information by the entropy decoding unit 210 is transmitted to the prediction unit 230.
[0053] The reordering unit 215 reorders the bitstream entropy decoded by the entropy decoding unit 210 based on the reordering method used in the video encoder. The reordering unit 215 receives information related to coefficient scanning performed in the encoder, and restores and reorders coefficients expressed in one-dimensional vector form into coefficients in two-dimensional block form through a method of scanning in reverse based on the scanning order performed in the encoder.
[0054] The inverse quantization unit 220 performs inverse quantization based on the quantization parameter provided by the encoder and the coefficient values of the reordered blocks.
[0055] The inverse transform unit 225 performs an inverse transform of the transform performed by the transform unit of the encoder. The inverse transform may be performed based on a transmission unit or a division unit determined by the encoder. The transform unit of the encoder may selectively perform a DCT or a DST according to a plurality of information such as a prediction method, a size of a current block, a prediction direction, etc., and the inverse transform unit 225 of the decoder may perform an inverse transform based on the transform information performed by the transform unit of the encoder.
[0056] The prediction unit 230 generates a prediction block based on information for generating a prediction block provided from the entropy decoding unit 210 and information on already decoded blocks and / or pictures provided from the memory 240. A reconstructed block is generated based on the prediction block generated by the prediction unit 230 and a residual block provided from the inverse transform unit 225. For example, if the current block is encoded in inter prediction mode, inter prediction is performed on the current prediction unit based on information included in at least one picture preceding and following the current picture. In this case, motion information required for inter prediction, such as a motion vector and a reference picture index, may be derived from a skip flag, a merge flag, etc. provided from the encoder.
[0057] The reconstructed blocks and / or pictures may be provided to a filter unit 235. The filter unit 235 applies deblocking filtering, SAO and / or adaptive loop filtering, etc. to the reconstructed blocks and / or pictures.
[0058] The reconstructed pictures and blocks can be stored in the memory 240 so that they can be used as reference pictures or reference blocks, and can be transmitted to an output unit (not shown).
[0059] Meanwhile, the encoder encodes the current block using the most efficient encoding method based on image information of the current block, and the decoder determines a decoding method based on the encoding method used by the encoder. The encoding method used by the encoder can be derived via a bitstream transmitted from the encoder or based on information of the current block. If the current block is encoded in intra prediction mode, intra prediction is performed to generate a predicted block based on pixel information in the current picture.
[0060] FIG. 3 is a flow diagram that schematically illustrates a method for intra prediction in a video decoder.
[0061] The decoder derives the prediction mode of the current block (S310).
[0062] Intra prediction can have a prediction direction depending on the position of a reference sample used for prediction. An intra prediction mode having a prediction direction is called an intra_Angular prediction mode. On the other hand, intra prediction modes without a prediction direction include Intra_Planar prediction mode, Intra_DC prediction mode, and Intra_Fromluma prediction mode.
[0063] FIG. 4 shows prediction directions in intra prediction modes, and Table 1 shows mode values of the intra prediction modes shown in FIG.
[0064] [Table 1]
[0065] Intra prediction performs prediction on a current block based on a derived prediction mode. Since the reference samples used for prediction and the specific prediction method vary depending on the prediction mode, if the current block is coded in intra prediction mode, the decoder derives the prediction mode of the current block to perform prediction.
[0066] The decoder can determine whether neighboring samples of the current block can be used for prediction and construct reference samples to be used for prediction (S320). In intra prediction, neighboring samples of the current block refer to 2*nS length samples adjacent to the left boundary and bottom left of the current block, which has a size of nS×nS, and 2*nS length samples adjacent to the top boundary and top right of the current block. However, some of the neighboring samples of the current block may not yet be decoded or may not be available. In such cases, the decoder can construct reference samples to be used for prediction by substituting unavailable samples for available samples.
[0067] The decoder may perform filtering of reference samples based on a prediction mode (S330). The decoder may perform filtering of reference samples before performing prediction. Whether to perform filtering of reference samples is determined based on the prediction mode of the current block. Adaptive filtering of reference samples based on the prediction mode is called Mode Dependent Intra Smoothing (MDIS) or simply smoothing filtering.
[0068] Table 2 shows an example of determining whether to apply a filter to a reference sample based on a prediction mode.
[0069] [Table 2]
[0070] In Table 2, when intraFilterType is 1, smoothing filtering is performed. For example, in the Intra_Planar mode, when nS=8, smoothing filtering can be performed. In this case, smoothing filters with various filtering coefficients can be applied. For example, a smoothing filter with a coefficient of
[0121] can be applied.
[0071] The decoder generates a prediction block for the current block based on the prediction mode and the reference sample (S340). The decoder generates a prediction block for the current block based on the prediction mode derived in the prediction mode derivation step (S310) and the reference sample obtained through the reference sample construction step (S320) and the reference sample filtering step (S330).
[0072] In the prediction block generation step (S340), if the current block is coded in Intra_DC prediction mode, the left and top boundary samples of the prediction block may be 2-tap filtered to minimize discontinuity at the block boundary. Here, the boundary samples refer to samples located within the prediction block and adjacent to the boundary of the prediction block.
[0073] FIG. 5 shows a case where the current block is coded in the Intra_DC prediction mode.
[0074] 5, when a current block 500 is coded in Intra_DC prediction mode, a left boundary sample 522 and an upper boundary sample 521 of the current block 500 are likely to be similar to a left reference sample 530 and an upper reference sample 510, respectively, and therefore, a smoothing filter can be applied as shown in FIG. 5. The shaded portion 505 in the drawing indicates the region to be filtered.
[0075] For some intra directional prediction modes, a 2-tap filter can be applied to the left boundary sample and the top boundary sample, similar to the Intra_DC mode. However, it is not applied to both the left boundary sample and the top boundary sample, but is adaptively applied to the left boundary sample or the top boundary sample depending on the prediction direction. In other words, it is applied only to boundary samples adjacent to reference samples not actually used in directional prediction.
[0076] Specifically, in the prediction block generation step (S340), if the current block is encoded in an intra-directional prediction mode, the value of the prediction sample may be derived based on a reference sample located in the prediction direction. In some intra-directional prediction modes, a boundary sample not located in the prediction direction, among the left boundary sample and the top boundary sample of the prediction block, may be adjacent to a reference sample not used for prediction. That is, the distance to the reference sample not used for prediction is significantly closer than the distance to the reference sample used for prediction. Since the value of a prediction sample is more likely to be similar to a reference sample that is close, in the present invention, in order to improve prediction performance and coding efficiency, filtering is performed with adjacent reference samples for the boundary sample not located in the prediction direction, among the left boundary sample and the top boundary sample.
[0077] For ease of explanation, the process of deriving predicted sample values in intra-directional prediction mode will be described below as a step of deriving predicted sample values as values of reference samples located in the prediction direction, and a step of filtering and modifying boundary samples not located in the prediction direction among the left boundary sample and the top boundary sample of the predicted block. Furthermore, [x, y] coordinates are set with increasing coordinate values in a lower right direction based on the top left sample of the current block and the predicted block. Furthermore, the size of the current block and the predicted block is defined as nS. For example, the top left boundary sample of the predicted block has a position [0,0], the left boundary sample has a position [0,0...nS-1], and the top boundary sample has a position [0...nS-1,0].
[0078] First, the value of a predicted sample is derived based on a reference sample located in the prediction direction.
[0079] For example, if the current block is coded in vertical prediction mode, the value of the predicted sample is derived as the value of the sample having the same x-coordinate among the reference samples adjacent to the upper boundary of the current block. That is, the value of the predicted sample predSamples[x,y] is derived as shown in Equation 1.
[0080]
number
[0081] Here, p[a,b] denotes the value of the sample having the position [a,b].
[0082] For example, if the current block is coded in horizontal prediction mode, the value of the predicted sample is derived as the value of the sample having the same y-coordinate among the reference samples adjacent to the left boundary of the current block. That is, the value of the predicted sample predSamples[x,y] is derived as shown in Equation 2.
[0083]
number
[0084] For example, if the current block is coded in an intra-directional prediction mode in which the prediction direction is from top to bottom, the value of the predicted sample is derived as the value of the reference sample located in the prediction direction among the reference sample adjacent to the top boundary of the current block and the reference sample located in the top right corner.
[0085] For example, if the current block is coded in an intra-directional prediction mode in which the prediction direction is left-downward, the value of the predicted sample is derived as the value of the reference sample located in the prediction direction among the reference sample adjacent to the left boundary of the current block and the reference sample located in the bottom left.
[0086] After deriving the value of a prediction sample based on the reference sample located in the prediction direction, the value of the corresponding boundary sample can be corrected by filtering the boundary sample not located in the prediction direction among the left boundary sample and the top boundary sample of the prediction block based on the adjacent reference sample. Hereinafter, a method for filtering the boundary sample not located in the prediction direction among the left boundary sample and the top boundary sample of the prediction block using the reference sample not located in the prediction direction will be described in detail with reference to Figures 5 to 13.
[0087] FIG. 6 illustrates a case where the prediction direction is vertical in an intra prediction mode according to an embodiment of the present invention.
[0088] Referring to FIG. 6, in the case of an Intra_Vertical prediction mode, a smoothing filter can be applied to the left boundary samples 620 .
[0089] As described above, when the current block 600 is encoded in vertical prediction mode, the value of the predicted sample is derived as the value of the upper reference sample. In this case, the reference sample adjacent to the left boundary of the current block 600 is not used for directional prediction but is adjacent to the left boundary sample of the current block 600. That is, the left boundary sample 620 is closer to the left reference sample 630, which is a reference sample not used for prediction, than to the upper reference sample 610, which is a reference sample used for prediction. Here, the upper reference sample 610 refers to a sample adjacent to the upper boundary of the current block and having the same x-coordinate ([x, -1]), and the left reference sample 630 refers to a sample adjacent to the left boundary of the current block and having the same y-coordinate ([-1, y]). Therefore, since the value of the left boundary sample 620 is likely to be similar to the value of the adjacent left reference sample 630, a smoothing filter can be applied to the left boundary sample 620 as shown in FIG. 6. The shaded area 605 in the drawing indicates a region to be filtered.
[0090] For example, when a smoothing filter with a coefficient of ≡2 is applied, the value predSamples[x,y] of the modified left boundary sample 620 can be derived as shown in Equation 3.
[0091]
number
[0092] The coefficient of the filter is not limited to
[0011] / 2, but a filter having coefficients such as
[0013] / 4,
[0017] / 8, etc. Also, the coefficient of the filter can be adaptively determined according to the size of the current block.
[0093] Meanwhile, when filtering with the left reference sample, information on neighboring blocks may be further taken into consideration. For example, the value of the modified left boundary sample 620 may be derived as shown in Equation 4 by considering a change in sample value depending on the y-coordinate value of the left boundary sample 620 with respect to the upper left reference sample 640.
[0094]
number
[0095] When deriving the value of the left boundary sample 620 using the above method, the value of the predicted sample may exceed a specified bit depth. Therefore, the value of the predicted sample may be limited to a specified bit depth, or the difference may be weighted. For example, in the case of a predicted sample of a luma component, the value of the modified left boundary sample 620 may be derived as shown in Equation 5.
[0096]
number
[0097] FIG. 7 illustrates a case where the prediction direction is horizontal in an intra prediction mode according to an embodiment of the present invention.
[0098] Referring to FIG. 7, in the case of an Intra_Horizontal prediction mode, a smoothing filter can be applied to the upper boundary samples 720 .
[0099] As described above, when the current block 700 is encoded in horizontal prediction mode, the value of the prediction sample is derived as the value of the left reference sample. In this case, the reference sample adjacent to the top boundary of the current block 700 is not used for directional prediction but is adjacent to the top boundary sample of the current block 700. That is, the top boundary sample 720 is closer to the top reference sample 710, which is a reference sample not used for prediction, than to the left reference sample 730, which is a reference sample used for prediction. Here, the top reference sample 710 refers to a sample adjacent to the top boundary of the current block and having the same x-coordinate ([x, -1]), and the left reference sample 730 refers to a sample adjacent to the left boundary of the current block and having the same y-coordinate ([-1, y]). Therefore, since the value of the top boundary sample 720 is likely to be similar to the value of the adjacent top reference sample 710, a smoothing filter can be applied to the top boundary sample 720 as shown in FIG. 7. The shaded area 705 in the drawing indicates a region to be filtered.
[0100] For example, when a smoothing filter with a coefficient of
[0011] / 2 is applied, the value predSamples[x,y] of the modified upper boundary sample 620 can be derived as shown in Equation 6.
[0101]
number
[0102] The coefficient of the filter is not limited to
[0011] / 2, but a filter having coefficients such as
[0013] / 4,
[0017] / 8, etc. Also, the coefficient of the filter can be adaptively determined according to the size of the current block.
[0103] Meanwhile, when filtering with the upper reference sample, information of neighboring blocks may be further taken into consideration. For example, the value of the modified upper boundary sample 720 may be derived as shown in Equation 7 by taking into consideration a change in sample value depending on the x-coordinate value of the upper boundary sample 720 with respect to the upper-left reference sample 740.
[0104]
number
[0105] When deriving the value of the upper boundary sample 720 using the above method, the value of the predicted sample may exceed a specified bit depth. Therefore, the value of the predicted sample may be limited to a specified bit depth, or the difference may be weighted. For example, in the case of a predicted sample of a luma component, the value of the modified upper boundary sample 720 may be derived as shown in Equation 8.
[0106]
number
[0107] Meanwhile, the method of applying a smoothing filter to the left boundary sample or the top boundary sample based on the prediction mode of the current block can be applied not only to the vertical prediction mode and / or the horizontal prediction mode but also to other intra-directional modes.
[0108] For example, intra-directional prediction modes may be classified based on the prediction direction, and adaptive filtering may be performed depending on the group to which the corresponding mode belongs.
[0109] FIG. 8 shows an example in which intra prediction modes are classified according to prediction direction.
[0110] When the prediction direction of the intra prediction mode is upper-right 810, a smoothing filter can be applied to the left boundary sample, as in the vertical prediction mode. When the prediction direction of the intra prediction mode is lower-left 820, a smoothing filter can be applied to the top boundary sample, as in the horizontal prediction mode.
[0111] FIG. 9 illustrates a case where the prediction direction is upper right in an intra prediction mode according to an embodiment of the present invention.
[0112] As described above, when the current block 900 is encoded in an intra-directional prediction mode in which the prediction direction is right-up, the value of the prediction sample is derived as the value of the reference sample 910 located in the prediction direction from the reference sample adjacent to the right boundary of the current block and the reference sample located in the upper right corner. In this case, the reference sample adjacent to the left boundary of the current block 900 is not used, but is adjacent to the left boundary sample. That is, the left boundary sample 920 is closer to the left reference sample 930 than to the reference sample 910 located in the prediction direction. Here, the left reference sample 630 refers to a sample adjacent to the left boundary of the current block and having the same y-coordinate ([-1, y]). Therefore, since the value of the left boundary sample 920 is likely to be similar to the value of the adjacent left reference sample 930, a smoothing filter can be applied to the left boundary sample 920 as shown in FIG. 9. The shaded area 905 in the drawing indicates a region to be filtered.
[0113] FIG. 10 illustrates a case where the prediction direction is bottom-left in an intra prediction mode according to an embodiment of the present invention.
[0114] As described above, when the current block 1000 is encoded in an intra-directional prediction mode with a prediction direction of left-bottom, the value of the prediction sample is derived as the value of the reference sample 1030 located in the prediction direction from the reference sample adjacent to the left boundary of the current block and the reference sample located in the bottom-left. In this case, the reference sample adjacent to the top boundary of the current block 1000 is not used, but is adjacent to the top boundary sample. That is, the top boundary sample 1020 is closer to the top reference sample 1010 than to the reference sample 1030 located in the prediction direction. Here, the top reference sample 1010 refers to a sample adjacent to the top boundary of the current block and having the same x-coordinate ([x, -1]). Therefore, since the value of the top boundary sample 1020 is likely to be similar to the value of the adjacent top reference sample 1030, a smoothing filter can be applied to the top boundary sample 1020 as shown in FIG. 10. The shaded area 1005 in the drawing indicates a region to be filtered.
[0115] Meanwhile, as described above, for convenience of explanation, the process of deriving predicted sample values has been described as being divided into a step of deriving predicted sample values as values of reference samples located in a prediction direction and a step of filtering and modifying boundary samples that are not located in a prediction direction among left boundary samples and upper boundary samples of a prediction block, but the process of deriving predicted sample values may be performed in a single step without being divided into multiple steps. For example, in the process of deriving values of boundary samples that are not located in a prediction direction among left boundary samples and upper boundary samples of a prediction block, the step of filtering the corresponding boundary sample may be performed in a single step together with the step of deriving predicted sample values as values of reference samples located in a prediction direction, without being performed as a separate step.
[0116] For example, in the embodiment of FIG. 6, the value of the left boundary sample 620 can be derived based on the upper reference sample 610 and the reference sample 630 adjacent to the left boundary sample, as shown in Equations 3 to 5.
[0117] For example, in the embodiment of FIG. 7, the value of the upper boundary sample 720 can be derived based on the left reference sample 730 and the reference sample 710 adjacent to the upper boundary sample as shown in Equations 6 to 8.
[0118] For example, in the embodiment of FIG. 9, the value of the left boundary sample 920 can be derived based on a reference sample 910 located in the prediction direction and a reference sample 930 adjacent to the left boundary sample.
[0119] For example, in the embodiment of FIG. 10, the value of the upper boundary sample 1020 can be derived based on the reference sample 1030 located in the prediction direction and the reference sample 1010 adjacent to the upper boundary sample.
[0120] On the other hand, smoothing filtering is not performed on the left and top boundary samples of the prediction block that are not boundary samples located in the prediction direction, so the value of the prediction sample is derived as the value of the reference sample located in the prediction direction.
[0121] For example, if the current block is coded in a vertical prediction mode, the value of the prediction sample that is not the left boundary sample is derived as shown in Equation 9.
[0122]
number
[0123] For example, if the current block is coded in a horizontal prediction mode, the value of the prediction sample that is not the upper boundary sample is derived as shown in Equation 10.
[0124]
number
[0125] Meanwhile, the method of applying a smoothing filter to the left boundary sample or the top boundary sample according to the prediction mode of the current block may be applied to only some of the predicted samples of the corresponding boundary sample, rather than to all of the predicted samples of the corresponding boundary sample.
[0126] When the distance to the reference sample used for directional prediction is close, the error of the predicted sample is likely to be small. In this case, it is correct not to apply a smoothing filter, i.e., not to consider other sample information. Therefore, it is possible to determine whether to perform filtering with adjacent reference samples depending on the position of the boundary sample within the block.
[0127] For example, in vertical prediction mode, a smoothing filter can be applied to only some of the left boundary samples, and in horizontal prediction mode, a smoothing filter can be applied to only some of the upper boundary samples.
[0128] 11 illustrates a case where the prediction direction is vertical in an intra prediction mode according to another embodiment of the present invention. Referring to FIG. 11, a smoothing filter may be applied to only some of the left boundary samples. That is, since the accuracy of prediction decreases as the distance from the reference sample used for prediction increases, the smoothing filter may be applied only to samples in an area where the accuracy decreases.
[0129] For example, a smoothing filter can be applied only to the left boundary sample 1120, which is far from the upper reference sample 1110, based on half the height of the current block 1100. The shaded area 1105 in the drawing indicates the region to be filtered.
[0130] Even when the prediction mode of the current block is the horizontal prediction mode, it is possible to determine whether to perform filtering with the adjacent reference samples depending on the position of the upper boundary sample within the block.
[0131] 12 illustrates a case where the prediction direction is horizontal in an intra prediction mode according to another embodiment of the present invention, in which a smoothing filter may be applied to only some of the upper boundary samples.
[0132] For example, a smoothing filter may be applied only to the top boundary samples 1220 that are far from the top reference sample 1230, based on half the width of the current block 1200. The shaded area 1205 in the drawing indicates the region to be filtered.
[0133] Meanwhile, the area to which the smoothing filter is applied is not limited to half the height or width of the current block. That is, it may be set to 1 / 4 or 3 / 4 of the size, and may be adaptively determined based on the distance from the sample used for prediction according to the intra prediction mode. In this case, the area to which the smoothing filter is applied may be defined in a look-up table, thereby reducing the computational burden on the encoder or decoder.
[0134] Meanwhile, the technical idea of the present invention can be applied to both the luma component and the chroma component, but may be applied only to the luma component and not to the chroma component. When applied only to the luma component, the value of the predicted sample of the chroma component is derived in the same manner as in a general intra prediction mode.
[0135] FIG. 13 is a diagram for explaining the operation of an encoder in a system to which the present invention is applied.
[0136] The encoder performs prediction on the current block (S1310). The encoder generates a prediction block for the current block based on the prediction mode of the current block. In this case, neighboring samples of the current block can be used as reference samples to derive the values of the prediction samples.
[0137] When the prediction mode of the current block is an intra-directional prediction mode, the encoder may derive values of boundary samples that are not located in the prediction direction of the intra-directional prediction mode, among the left boundary sample and the top boundary sample of the prediction block, based on reference samples located in the prediction direction and reference samples adjacent to the corresponding boundary samples. Here, the boundary samples refer to samples located within the prediction block and adjacent to the boundary of the prediction block.
[0138] For example, when the intra-directional prediction mode is a vertical prediction mode, the encoder may derive the value of the left boundary sample based on an upper reference sample of the left boundary sample and a reference sample adjacent to the left boundary sample, where the upper reference sample refers to a sample adjacent to the upper boundary of the current block and having the same x-coordinate.
[0139] For example, if the intra-directional prediction mode is a vertical prediction mode, the encoder can derive the value of the left boundary sample based on the reference sample above the left boundary sample, the reference sample adjacent to the left boundary sample, and the reference sample adjacent to the upper left of the current block.
[0140] For example, when the intra-directional prediction mode is a horizontal prediction mode, the encoder may derive the value of the top boundary sample based on a left reference sample of the top boundary sample and a reference sample adjacent to the top boundary sample, where the left reference sample refers to a sample adjacent to the left boundary of the current block and having the same y-coordinate.
[0141] For example, if the intra-directional prediction mode is a horizontal prediction mode, the encoder can derive the value of the upper boundary sample based on a reference sample to the left of the upper boundary sample, a reference sample adjacent to the upper boundary sample, and a reference sample adjacent to the upper left of the current block.
[0142] For example, if the prediction direction of the prediction mode is right-up, the encoder may derive the value of the left boundary sample based on a reference sample located in the prediction direction and a reference sample adjacent to the left boundary sample.
[0143] For example, when the prediction direction of the prediction mode is bottom-left, the encoder may derive the value of the top boundary sample based on a reference sample located in the prediction direction and a reference sample adjacent to the top boundary sample.
[0144] Meanwhile, the encoder can derive the value of a non-boundary prediction sample, which is not located in the prediction direction of the intra-directional prediction mode, among the left boundary sample and the top boundary sample of the prediction block as the value of a reference sample located in the prediction direction.
[0145] For example, when the intra-directional prediction mode is a vertical prediction mode, the encoder may derive the value of the predicted sample as the value of the upper reference sample of the predicted sample.
[0146] For example, if the intra-directional prediction mode is a horizontal prediction mode, the encoder may derive the value of the predicted sample as the value of the reference sample to the left of the predicted sample.
[0147] The encoder entropy-encodes information about the prediction block generated through the prediction step S1310 (S1320). As described above, the entropy encoding may use encoding methods such as Exponential Golomb and CABAC, and may assign codewords in consideration of the frequency of occurrence of each prediction mode or prediction type.
[0148] The encoder signals information encoded through the entropy encoding step (S1320) (S1330). For example, the encoder may signal prediction mode information, a residual signal between the predicted block and the original block, and, if a smoothing filter is applied during intra prediction, information on the coefficients of the smoothing filter may be signaled.
[0149] FIG. 14 is a diagram for explaining the operation of a decoder in a system to which the present invention is applied.
[0150] The decoder receives information from the encoder (S1410). The information received from the encoder can be transmitted in a bitstream.
[0151] The decoder entropy decodes the information received through the information receiving step (S1410) (S1420). The decoder can obtain information for predicting the current block, such as the prediction method (inter prediction / intra prediction, etc.) of the current block, the motion vector (inter prediction), the prediction mode (intra prediction), and the residual signal through the entropy decoding step (S1520).
[0152] The decoder performs prediction for the current block based on the information obtained through the entropy decoding step (S1420) (S1430). The decoder generates a prediction block for the current block based on the prediction mode of the current block. In this case, neighboring samples of the current block can be used as reference samples to derive the values of the prediction samples.
[0153] The prediction method performed in the decoder is the same as or similar to the prediction method performed in the encoder.
[0154] That is, when the prediction mode of the current block is an intra-directional prediction mode, the decoder can derive the value of the boundary sample, among the left boundary sample and the top boundary sample of the predicted block, that is not located in the prediction direction of the intra-directional prediction mode, based on the reference sample located in the prediction direction and the reference sample adjacent to the corresponding boundary sample.
[0155] For example, if the intra-directional prediction mode is a vertical prediction mode, the decoder may derive the value of the left boundary sample based on the reference sample above the left boundary sample and the reference sample adjacent to the left boundary sample.
[0156] For example, if the intra-directional prediction mode is a vertical prediction mode, the decoder can derive the value of the left boundary sample based on the reference sample above the left boundary sample, the reference sample adjacent to the left boundary sample, and the reference sample adjacent to the upper left of the current block.
[0157] For example, if the intra-directional prediction mode is a horizontal prediction mode, the decoder may derive the value of the top boundary sample based on a reference sample to the left of the top boundary sample and a reference sample adjacent to the top boundary sample.
[0158] For example, if the intra-directional prediction mode is a horizontal prediction mode, the decoder can derive the value of the top boundary sample based on a reference sample to the left of the top boundary sample, a reference sample adjacent to the top boundary sample, and a reference sample adjacent to the upper left of the current block.
[0159] For example, if the prediction direction of the prediction mode is right-up, the decoder may derive the value of the left boundary sample based on a reference sample located in the prediction direction and a reference sample adjacent to the left boundary sample.
[0160] For example, if the prediction direction of the prediction mode is bottom-left, the decoder may derive the value of the top boundary sample based on a reference sample located in the prediction direction and a reference sample adjacent to the top boundary sample.
[0161] In addition, the decoder can derive the value of a prediction sample that is not a boundary sample located in the prediction direction of the intra-directional prediction mode, among the left boundary sample and the top boundary sample of the prediction block, as the value of a reference sample located in the prediction direction.
[0162] For example, when the intra-directional prediction mode is a vertical prediction mode, the decoder may derive the value of the predicted sample as the value of the upper reference sample of the predicted sample.
[0163] For example, if the intra-directional prediction mode is a horizontal prediction mode, the decoder may derive the value of the predicted sample as the value of the reference sample to the left of the predicted sample.
[0164] The decoder reconstructs the image based on the predicted block generated through the prediction step (S1430) (S1440).
[0165] In the above exemplary system, the method is described based on a flow chart expressed as a series of steps or blocks, but the present invention is not limited to the order of steps, and some steps may occur in a different order or simultaneously with other steps. Furthermore, the above-described embodiments include examples of various aspects. Therefore, the present invention includes all alterations, modifications, and variations that fall within the scope of the claims.
[0166] In the above description of the present invention, when a component is said to be "coupled" or "connected" to another component, it should be understood that the component is directly coupled or connected to the other component, but that there may be other components between the two components. Conversely, when a component is said to be "directly coupled" or "directly connected" to another component, it should be understood that there is no other component between the two components.
Claims
1. 1. A method for decoding video performed by a decoder, comprising: obtaining prediction mode information; determining whether a current block is coded in an intra prediction mode based on the prediction mode information; deriving the intra prediction mode for the current block as a horizontal prediction mode associated with a horizontal prediction direction based on the determining step; deriving neighboring reference samples of the current block; generating a predicted sample of the current block based on the intra prediction mode and the surrounding reference samples; generating reconstructed samples based on the predicted samples; and The predicted sample is generated based on a first reference sample located along the horizontal prediction direction with respect to a position of the predicted sample, the first reference sample being located outside the current block and adjacent to a left boundary of the current block; Among the predicted samples, a predicted sample adjacent to an upper boundary of the current block is filtered based on a second reference sample located along a vertical direction with respect to a position of the predicted sample, the second reference sample being located outside the current block and adjacent to an upper side of the predicted sample; a filtering coefficient applied to the values of the prediction sample is greater than a filtering coefficient applied to the values of the second reference sample; The predicted samples are filtered based on the following formula: predSamples[x, y] represents a filtered value for the predicted sample located at coordinates (x, y), p[-1, y] represents a value of a sample located at coordinates (-1, y), p[x, -1] represents a value of a sample located at coordinates (x, -1), and p[-1, -1] represents a value of a sample located at coordinates (-1, -1).
2. 1. A method for encoding video performed by an encoder, comprising: determining whether the current block is coded in an intra-prediction mode; deriving the intra prediction mode for the current block as a horizontal prediction mode associated with a horizontal prediction direction based on the determining step; deriving neighboring reference samples of the current block; generating prediction mode information based on the intra prediction mode; generating a predicted sample of the current block based on the intra prediction mode and the surrounding reference samples; generating residual information based on the prediction samples; encoding video information including the prediction mode information and the residual information; and The predicted sample is generated based on a first reference sample located along the horizontal prediction direction with respect to a position of the predicted sample, the first reference sample being located outside the current block and adjacent to a left boundary of the current block; Among the predicted samples, a predicted sample adjacent to an upper boundary of the current block is filtered based on a second reference sample located along a vertical direction with respect to a position of the predicted sample, the second reference sample being located outside the current block and adjacent to an upper side of the predicted sample; a filtering coefficient applied to the values of the prediction sample is greater than a filtering coefficient applied to the values of the second reference sample; The predicted samples are filtered based on the following formula: predSamples[x, y] represents filtered values for the predicted samples located at coordinates (x, y), p[-1, y] represents the value of the sample located at coordinates (-1, y), p[x, -1] represents the value of the sample located at coordinates (x, -1), and p[-1, -1] represents the value of the sample located at coordinates (-1, -1).
3. A method for transmitting data for video information, comprising: obtaining a bitstream of the video information, wherein the bitstream is generated based on: determining whether a current block is coded in an intra prediction mode; deriving the intra prediction mode for the current block as a horizontal prediction mode associated with a horizontal prediction direction based on the determining; deriving neighboring reference samples for the current block; generating prediction mode information based on the intra prediction mode; generating prediction samples for the current block based on the intra prediction mode and the neighboring reference samples; generating residual information based on the prediction samples; and encoding the video information including the prediction mode information and the residual information; transmitting the data including the bitstream of the video information; The predicted sample is generated based on a first reference sample located along the horizontal prediction direction with respect to a position of the predicted sample, the first reference sample being located outside the current block and adjacent to a left boundary of the current block; Among the prediction samples, a prediction sample adjacent to an upper boundary of the current block is filtered based on a second reference sample located along a vertical direction with respect to a position of the prediction sample, the second reference sample being adjacent to an upper side of the prediction sample; a filtering coefficient applied to the values of the prediction sample is greater than a filtering coefficient applied to the values of the second reference sample; The predicted samples are filtered based on the following formula: predSamples[x, y] represents filtered values for the prediction samples located at coordinates (x, y), p[-1, y] represents the value of the sample located at coordinates (-1, y), p[x, -1] represents the value of the sample located at coordinates (x, -1), and p[-1, -1] represents the value of the sample located at coordinates (-1, -1).
Citation Information
Patent Citations
Device and method for encoding image, and device and method for decoding image
JP2009094828A
Image encoding device and decoding device, image encoding method and decoding method, program for the devices and the methods, and recording medium recording program
WO2009051091A1