Image encoding / decoding method and device using improved intra prediction method, and recording medium

Abstract

The present disclosure relates to an image encoding / decoding method and device using an improved intra prediction method, and a recording medium, the method comprising the steps of: determining reference pixels of a current coding block on the basis of reference pixel information of the current coding block; and generating a prediction block of the current coding block on the basis of the reference pixels and a prediction mode of the current coding block, wherein the reference pixels may be determined on the basis of all or some pixels included in a reference pixel line of the current coding block.

Description

Image encoding / decoding method, device, and recording medium using an improved on-screen prediction method

[0001] The present disclosure relates to an improved intra-screen prediction method that improves encoding and decoding efficiency of existing intra-screen prediction methods, and can be utilized in the field of video technology utilizing intra-screen prediction methods.

[0002] Recently, the demand for multimedia data, such as video, has been rapidly increasing on the Internet. However, the pace of growth in channel bandwidth is struggling to keep pace with the rapidly increasing volume of multimedia data.

[0003] Although intra-screen prediction methods are used in terms of image encoding / decoding, existing prediction methods cannot keep up with the demand for data, and thus the need for improved intra-screen prediction methods such as the present disclosure has increased.

[0004] The purpose of the present disclosure is to improve the encoding / decoding efficiency of images by using an improved intra-screen prediction method.

[0005] A method, device, and recording medium for encoding / decoding an image using an improved on-screen prediction method of the present disclosure include: a step of determining a reference pixel of a current coding block based on reference pixel information of the current coding block; and a step of generating a prediction block of the current coding block based on the reference pixel and a prediction mode of the current coding block, wherein the determination of the reference pixel may be determined based on all or part of pixels included in a reference pixel line of the current coding block.

[0006] In a method, device and recording medium for encoding / decoding an image using the improved on-screen prediction method of the present disclosure, the number of reference pixel lines of the current coding block may be plural.

[0007] In a method, device and recording medium for encoding / decoding an image using the improved on-screen prediction method of the present disclosure, the reference pixel line of the current coding block may include at least one of a reference pixel line adjacent to the current coding block or a reference pixel line not adjacent to the current coding block.

[0008] In a method, device, and recording medium for encoding / decoding an image using the improved on-screen prediction method of the present disclosure, the step of determining the reference pixel may include the step of determining a reference area of ​​the current coding block; the step of determining a reference pixel line within the reference area; and the step of determining all or part of the pixels of the reference pixel line as reference pixels of the current coding block.

[0009] In the method, device and recording medium for encoding / decoding an image using the improved on-screen prediction method of the present disclosure, the generation of the prediction block can be performed using a corrected reference pixel obtained by correcting a pixel value by performing reference pixel filtering on the reference pixel.

[0010] In the method, device and recording medium for encoding / decoding an image using the improved on-screen prediction method of the present disclosure, the reference pixel filtering can be performed using surrounding reference pixels of the target reference pixel.

[0011] In a method, device and recording medium for encoding / decoding an image using the improved on-screen prediction method of the present disclosure, the number of surrounding reference pixels may be 8.

[0012] In a method, device and recording medium for encoding / decoding an image using the improved on-screen prediction method of the present disclosure, the reference pixel line may include at least one of a vertical reference pixel line, a horizontal reference pixel line and a diagonal reference pixel line.

[0013] In the method, device and recording medium for encoding / decoding an image using the improved on-screen prediction method of the present disclosure, the horizontal reference pixel line may be composed of pixels between the pixel pointed to by the upper left pixel of the current coding block and the pixel pointed to by the upper right pixel.

[0014] In a method, device and recording medium for encoding / decoding an image using the improved on-screen prediction method of the present disclosure, the vertical reference pixel line may be composed of pixels between the pixel pointed to by the upper left pixel of the current coding block and the pixel pointed to by the lower left pixel.

[0015] In a method, device and recording medium for encoding / decoding an image using the improved on-screen prediction method of the present disclosure, when both the horizontal reference pixel line and the vertical reference pixel line are used, the lengths of the horizontal reference pixel line and the vertical reference pixel line may be different.

[0016] In the method, device and recording medium for encoding / decoding an image using the improved on-screen prediction method of the present disclosure, the reference pixels used in the horizontal reference pixel line may be some pixels included in the horizontal reference pixel line, and the reference pixels used in the vertical reference pixel line may be all pixels included in the vertical reference pixel line.

[0017] In a method, device and recording medium for encoding / decoding an image using the improved on-screen prediction method of the present disclosure, the reference pixel of the upper left pixel of the current coding block may include a first pixel of a first reference pixel line and a second pixel of a second reference pixel line.

[0018] The encoding / decoding efficiency of an image is improved when encoding / decoding an image using the improved on-screen prediction method according to the present disclosure.

[0019] FIG. 1 is a block diagram showing the configuration of an image encoding device (100) according to one embodiment of the present disclosure.

[0020] FIG. 2 may illustrate embodiments of a method for generating a prediction block using an in-screen prediction mode.

[0021] Figure 3 is a flowchart illustrating a method for encoding the optimal prediction mode of the current prediction block.

[0022] FIG. 4 is a diagram illustrating one embodiment of a method for setting an MPM candidate.

[0023] FIG. 5 is a block diagram illustrating an image decoding device (500) according to one embodiment of the present disclosure.

[0024] Figure 6 is a flowchart illustrating a method for decoding an optimal on-screen prediction mode of a current prediction block.

[0025] Figure 7 is a diagram showing a process of dividing an input image into coding block units in three steps.

[0026] Figures 8a to 8c are exemplary drawings showing the division process for each step in Figure 7.

[0027] Figure 9 is a flowchart showing the process of generating a prediction block of a current coding block in an encoding device and a decoding device.

[0028] Figure 10 illustrates the restored pixels around the current coding block that can be used as reference pixels.

[0029] Figure 11 is an example drawing for explaining reference pixel filtering.

[0030] Figure 12 illustrates examples of vertical mode direction (1201), upper right diagonal mode direction (1202), and non-directional mode (1203) when the current coding block is 4x4 and the reference pixel line is 2.

[0031] Figures 13 and 14 are flowcharts showing the process of encoding / decoding prediction information for generating prediction blocks.

[0032] Figure 15 illustrates the area restored before the current coding block and the area to be restored after the current coding block.

[0033] Figure 16 shows an example of the current coding block and some restored pixels in Figure 15.

[0034] Figures 17 and 18 are flowcharts showing the process of encoding / decoding prediction information for generating prediction blocks.

[0035] Figure 19 is a flowchart showing the process of generating a prediction block of a current coding block in an encoding device and a decoding device.

[0036] FIG. 20 and FIG. 21 are diagrams for explaining another method regarding a reference pixel candidate determination process and a method for determining a reference pixel used for prediction among the reference pixel candidates.

[0037] FIG. 22a and FIG. 22b are drawings for explaining a method of using diagonal reference pixel lines in multiple directions, including vertical and horizontal directions, rather than selecting reference pixel lines only in the vertical or horizontal direction.

[0038] Figure 23 is a diagram for explaining the first prediction block filtering method.

[0039] Figures 24 and 25 are flowcharts showing the process of encoding / decoding prediction mode information and reference pixel line information in on-screen prediction.

[0040] Figures 26 and 27 are flowcharts showing the process of encoding / decoding prediction mode information and reference pixel line information in on-screen prediction.

[0041] Figures 28 and 29 are flowcharts showing the process of encoding / decoding prediction mode information and reference pixel line information in on-screen prediction.

[0042] Figures 30a and 30b are diagrams showing various reference pixel lines and prediction modes.

[0043] FIGS. 31a to 31c are diagrams for explaining a method of generating a prediction block in sub-block units using non-adjacent reference pixel lines.

[0044] Figures 32a to 32e are diagrams showing reference pixel lines used in PLANAR prediction mode and reference pixels used on the right and bottom.

[0045] The present invention is susceptible to various modifications and embodiments. Specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present invention to specific embodiments, but rather to encompass all modifications, equivalents, and alternatives falling within the spirit and technical scope of the present invention. Throughout the description of each drawing, similar reference numerals have been used to designate similar components.

[0046] While terms such as "first" and "second" may be used to describe various components, these components should not be limited by these terms. These terms are used solely to distinguish one component from another. For example, without departing from the scope of the present invention, a first component may be referred to as a "second component," and similarly, a second component may also be referred to as a "first component." The term "and / or" includes a combination of multiple related items described herein or any of multiple related items described herein.

[0047] When a component is referred to as being "connected" or "connected" to another component, it should be understood that it may be directly connected or connected to that other component, but that there may be other components intervening. Conversely, when a component is referred to as being "directly connected" or "connected" to another component, it should be understood that there are no other components intervening.

[0048] The terminology used in this application is only used to describe specific embodiments and is not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly indicates otherwise. In this application, it should be understood that the terms "comprise" or "have" indicate the presence of a feature, number, step, operation, component, part, or combination thereof described in the specification, but do not exclude in advance the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0049] Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings. Hereinafter, identical components in the drawings will be designated by the same reference numerals, and redundant descriptions of identical components will be omitted.

[0050]

[0051] FIG. 1 is a block diagram showing the configuration of an image encoding device (100) according to one embodiment of the present disclosure.

[0052] The image encoding device (100) is a device for encoding an image and may include at least one of a block division unit (101), a prediction unit (102), a transformation unit (103), a quantization unit (104), an entropy encoding unit (105), an inverse quantization unit (106), an inverse transformation unit (107), an addition unit (108), an in-loop filter unit (109), a memory unit (110), or a subtraction unit (111). That is, the image encoding device (100) may include some or all of the units listed above.

[0053] The block splitting unit (101) can perform splitting from a block to be encoded with a maximum size (hereinafter referred to as a maximum encoding block) to a block to be encoded with a minimum size (hereinafter referred to as a minimum encoding block). The splitting can be performed based on at least one of quad-tree splitting (hereinafter referred to as QT (Quad-Tree) splitting), dual-tree splitting (hereinafter referred to as DT (Dual-Tree) splitting), ternary-tree (hereinafter referred to as TT (Ternary-tree) splitting), or geometric splitting. QT splitting can be a method of quadrupling an upper block into lower blocks whose width and height are half of those of the upper block. DT splitting can be a method of bisecting an upper block into lower blocks whose width or height is half of that of the upper block. Here, DT splitting can be called binary tree (BT) splitting. TT partitioning may be a method of dividing a parent block into three parts (at a 1:1:1 ratio or a 1:2:1 ratio) by either the width or height of the parent block to obtain a lower block. Geometric partitioning may be a method of dividing a parent block into geometric shapes (including diagonal shapes) by using the partitioning angle and the distance from the center point.

[0054] The block segmentation unit (101) can segment an input image into at least one block. At this time, the input image can have various shapes and sizes, such as a picture, slice, tile, or segment. A block can mean a coding unit (CU), a prediction unit (PU), or a transformation unit (TU).

[0055] Hereinafter, in the embodiments of the present disclosure, the term "encoding unit" may be used to mean a unit that performs encoding or a unit that performs decoding. Furthermore, the term "encoding unit" may refer to an encoding block. In other words, the term "decoding unit" may also refer to a decoding block.

[0056] The prediction unit (102) may include an inter-prediction unit that performs inter-prediction and an intra-prediction unit that performs intra-prediction. The prediction unit (102) may generate a prediction block using surrounding pixels of a block to be predicted (hereinafter, referred to as a prediction block) in a current original block or a reference picture that has already been decoded previously. Among the restoration information used to generate the prediction block, pixels, blocks, and pictures may be referred to as reference pixels, reference blocks, and reference pictures. Here, the prediction block may be generated from at least one prediction block within an encoding block. If there is one prediction block within an encoding block, the prediction block may have the same form as the encoding block. Meanwhile, generating a prediction block in the prediction unit (102) may mean the same as generating a prediction pixel of the prediction block. Both an intra-prediction unit and an inter-prediction unit may be used to generate the prediction block. In this case, generation of a prediction block can be performed through a weighted sum of a first prediction block obtained through intra-screen prediction and a second prediction block obtained through inter-screen prediction.

[0057] Prediction techniques for video signals can be broadly divided into intra-screen prediction and inter-screen prediction. Intra-screen prediction is a method of generating a prediction block using pixels surrounding the current block, and inter-screen prediction is a method of generating a prediction block by finding the block most similar to the current block in a reference picture that has already been encoded and decoded. Here, inter-screen prediction can be called inter-prediction, and intra-screen prediction can be called intra-prediction.

[0058] The prediction unit (102) can determine the optimal prediction mode of the prediction block by using various techniques such as rate-distortion optimization (RDO) for the residual block obtained by subtracting the prediction block from the current original block. The RDO cost calculation formula can be as shown in Mathematical Expression 1 below.

[0059] [Mathematical Formula 1]

[0060]

[0061] Here, D is the degradation due to quantization, R is the rate of the compressed stream, J is the RD cost, Φ is the encoding mode, and λ is a Lagrangian multiplier, which can be a coefficient for scale correction to match the units between the amount of errors and the amount of bits. In order to be selected as the optimal encoding mode during the encoding process, the J when the mode is applied, that is, the RD-cost value, must be smaller than when other modes are applied. The formula for calculating the RD-cost value can be calculated by simultaneously considering the bit rate and error.

[0062] The residual value (residual block) between the generated prediction block and the original block can be input to the transformation unit (103). In addition, the prediction mode information, motion vector information, etc. used for prediction can be encoded together with the residual value in the entropy encoding unit (105) and transmitted to the decoder. When a specific encoding mode is used, it may be possible to encode the original block as is and transmit it to the decoding unit without generating the prediction block through the prediction unit (102).

[0063] The intra-screen prediction unit can generate a prediction block based on reference pixel information surrounding the current block, which is pixel information within the current picture. If the coding mode of a block surrounding the current block on which intra-screen prediction is to be performed is inter-screen prediction, the reference pixels included in the surrounding block to which inter-screen prediction is applied can be replaced with reference pixels within another block surrounding which intra-screen prediction is applied. That is, if the reference pixel is unavailable, the unavailable reference pixel information can be used by replacing it with at least one reference pixel among the available reference pixels.

[0064] In intra-screen prediction, the prediction mode can have a directional prediction mode that uses reference pixel information according to the prediction direction, and a non-directional mode that does not use directional information when performing prediction. The mode for predicting luminance information and the mode for predicting chrominance information can be different. Furthermore, when predicting chrominance information, the intra-screen prediction mode information used to predict luminance information or the predicted luminance signal information can be utilized.

[0065] Meanwhile, there can be a total of (N+2) prediction modes in the on-screen prediction, including Planar mode, DC mode, and N angular prediction modes.

[0066]

[0067] FIG. 2 may illustrate embodiments of a method for generating a prediction block using an in-screen prediction mode.

[0068] 201 of FIG. 2 may represent a method for generating a prediction block in DC mode.

[0069] Referring to 201 in Fig. 2, a prediction block can be generated by applying the average value of the reference pixels a0 to q0 to all prediction pixels in regions R1 to R4. In addition, a final prediction block can be generated through FIR (Finite Impulse Response) filtering with two adjacent reference pixels (h0, j0) for the pixels in region R1, and with one adjacent reference pixel (one pixel corresponding to the position of each pixel among a0 to g0, k0 to q0) for regions R2 and R3.

[0070] 202 of FIG. 2 may represent a method for generating a prediction block in Planar mode.

[0071] Referring to 202 of FIG. 2, the Planar mode can generate a final prediction block by using a linear interpolation method between the reference pixel located at the top / left and the reference pixel copied at the bottom / right for each prediction pixel location.

[0072] 203 in Fig. 2 may represent the prediction directions of N Angular prediction modes.

[0073] Referring to 203 of FIG. 2, the Angular prediction mode can generate a final prediction block by applying the values ​​of adjacent reference pixels to the prediction block according to the prediction direction.

[0074]

[0075] Figure 3 is a flowchart illustrating a method for encoding the optimal prediction mode of the current prediction block.

[0076] Referring to FIG. 3, in step S301, an MPM (Most Probable Mode) candidate can be set. Here, the method for setting the MPM candidate will be described later with reference to FIG. 4.

[0077] And, in step S302, information indicating whether to encode the optimal screen prediction mode using MPM can be encoded.

[0078] In step S303, it is possible to determine whether MPM operation information is available. If the information is true, then in step S304, index information indicating which MPM candidate the optimal intra-screen prediction mode is identical to can be encoded. Conversely, if the information is false, then in step S305, information indicating which prediction mode among the remaining intra-screen prediction modes, excluding the intra-screen prediction mode that is an MPM candidate, is optimal can be encoded.

[0079]

[0080] FIG. 4 is a diagram illustrating one embodiment of a method for setting an MPM candidate.

[0081] In Fig. 4, L may represent the on-screen prediction mode information of the surrounding block located to the left of the current prediction block, and A may represent the on-screen prediction mode information of the surrounding block located above.

[0082] Referring to Figure 4, three MPM candidates can be finally determined based on each specified condition.

[0083] Meanwhile, the number of MPM candidates can be determined as P (P>0, P is an integer), and the method of determining the candidates can also be diverse.

[0084] The transformation unit (103) can generate a transformation block by transforming a residual block, which is the difference between the original block and the predicted block. Here, the transformation block may be the smallest unit used for the transformation and quantization process.

[0085] The transformation unit (103) can transform the residual signal into a frequency domain to generate a transformation block having a transformation coefficient. Here, various transformation techniques such as a Discrete Cosine Transform (DCT)-based transformation, a Discrete Sine Transform (DST), and a Karhunen Loeve Transform (KLT) can be used as a method of transforming the residual signal into a frequency domain, and the residual signal can be transformed into a frequency domain to generate a transformation coefficient using these techniques. In order to use the transformation technique, a matrix operation is performed using a basis vector, and various combinations of transformation techniques can be used during the matrix operation depending on which prediction mode the prediction block is encoded in. For example, during prediction within a screen, a Discrete Cosine Transform can be used in the horizontal direction and a Discrete Sine Transform can be used in the vertical direction depending on the prediction mode.

[0086] Meanwhile, the transformation method can be determined based on the intra prediction mode of the prediction block used to generate the residual block. For example, depending on the intra prediction mode, DCT may be used in the horizontal direction, and DST may be used in the vertical direction.

[0087] The transform block may be divided into QT (Quad Tree) division method, BT (Binary Tree) division method, TT (Ternary Tree) division method, or geometric division method in units of encoding blocks, and an optimal transform block division form may be determined, and the transform block division information may be transmitted to the image decoding device (500). This transform block division method may be referred to as a CU unit RT (Residual Tree) structure. This may mean that the transformation may or may not be performed in units of prediction blocks, and that the transform block may be determined by ignoring the boundaries of the prediction blocks.

[0088] In addition, the transform block is divided into a QT division method, a BT division method, a TT division method, or a Geometric division method in units of prediction blocks, and the transform block division form is determined, and the transform block division information can be transmitted to the image decoding device (500) in units of transform blocks. This transform block division method can be referred to as a PU unit RT structure. This can mean that the transform block cannot be generated in a division form that exceeds the boundary of the prediction block, and the transform block cannot be determined to exceed the boundary of the prediction block.

[0089] In the CU unit RT structure described above, transformation can be performed by determining the entire encoding block as a transformation block without transmitting transformation block division information. A similar method can be applied to the PU unit RT structure.

[0090] The quantization unit (104) can generate a quantized transform block by quantizing the transform block. That is, the quantization unit (104) can generate a quantized transform block (Quantized Transform Coefficient) having quantized transform coefficients by quantizing the transform coefficients of the transform block generated from the transform unit (103). As a quantization method, dead zone uniform threshold quantization (DZUTQ) or a quantization weighted matrix may be used, but various quantization methods such as improved quantization methods may be used.

[0091] Meanwhile, although the image encoding device (100) is illustrated and described above as including both a transform unit (103) and a quantization unit (104), the transform unit (103) and the quantization unit (104) may be optionally included. That is, the image encoding device (100) may generate a transform block by transforming a residual block and may not perform a quantization process, may not only perform a quantization process without transforming the residual block into frequency coefficients, but may even not perform both a transform and a quantization process.

[0092] Even if some or all of the processes of the transformation unit (103) and the quantization unit (104) in the video encoding device (100) are not performed, a block that is input to the entropy encoding unit (105) can be commonly referred to as a 'quantized transformation block'.

[0093] The entropy encoding unit (105) can encode the quantized transform block and output a bitstream. That is, the entropy encoding unit (105) can encode the coefficients of the quantized transform block output from the quantization unit (104) using various entropy encoding techniques, and can generate and output a bitstream including additional information (e.g., information on the prediction mode, quantization coefficients, etc.) necessary for decoding the corresponding block in an image decoding device described below.

[0094] Entropy encoding can be performed using various encoding methods, such as, for example, Exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC).

[0095] The inverse quantization unit (106) can restore the inverse quantization transform block by performing the quantization technique used in quantization on the quantized transform block in reverse.

[0096] The inverse transform unit (107) restores the residual block by inversely transforming the inverse quantization transform block using the same method used in the transformation, and can perform the inverse transformation by performing the transformation technique used in the transformation unit (104) in reverse.

[0097] Meanwhile, in the above, the inverse quantization unit (106) and the inverse transformation unit (107) can perform inverse quantization and inverse transformation by using the quantization method and transformation method used in the quantization unit (104) and the transformation unit (103) in reverse. In addition, when the transformation unit (103) and the quantization unit (104) only perform quantization and not transformation, only inverse quantization may be performed and inverse transformation may not be performed. If neither transformation nor quantization is performed, the inverse quantization unit (106) and the inverse transformation unit (107) may not perform inverse transformation and inverse quantization, or may be omitted and not included in the image encoding device (100).

[0098] The addition unit (108) can restore the current block by adding the residual signal generated in the inverse transformation unit (107) and the prediction block generated through prediction.

[0099] The in-loop filter unit (109) is a process that additionally filters the entire picture after all blocks in the current picture have been restored, and may include at least one of deblocking filtering, SAO (Sample Adaptive Offset), or ALF (Adaptive Loop Filter).

[0100] A deblocking filter can remove block distortion caused by boundaries between blocks in a reconstructed picture. To determine whether to perform deblocking, a deblocking filter can be applied to the current block based on the pixels contained in several columns or rows within the block. When applying a deblocking filter to a block, a strong filter or a weak filter can be applied depending on the required deblocking filtering strength. Furthermore, when applying a deblocking filter, horizontal and vertical filtering can be processed in parallel when performing vertical and horizontal filtering.

[0101] SAO (Sample Adaptive Offset) refers to the process of minimizing the difference between the restored image and the original image by subtracting or adding a specific value to the restored pixels.

[0102] Adaptive Loop Filtering (ALF) can be performed based on the comparison of the filtered restored image with the original image. After dividing the pixels included in the image into predetermined groups, a filter to be applied to each group can be determined, and filtering can be performed differentially for each group. Information regarding whether to apply ALF can be transmitted by luminance signal for each coding unit (CU), and the shape and filter coefficients of the ALF filter to be applied can vary depending on each block. Furthermore, an ALF filter of the same form (fixed form) can be applied regardless of the characteristics of the target block.

[0103] The memory (110) stores the restored current block that has undergone additional filtering in the in-loop filter unit (109) after adding the residual signal generated in the inverse transformation unit (107) and the prediction block generated through prediction, and can be used to predict the next block or the next picture, etc.

[0104] The subtraction unit (111) can generate a residual block by subtracting the predicted block from the current original block.

[0105]

[0106] FIG. 5 is a block diagram illustrating an image decoding device (500) according to one embodiment of the present disclosure.

[0107] Referring to FIG. 5, the image decoding device (500) may include at least one of a block entropy decoding unit (501), an inverse quantization unit (502), an inverse transformation unit (503), a prediction unit (504), an addition unit (505), an in-loop filter unit (506), or a memory unit (507). That is, the image decoding device (500) may include some or all of the units listed above.

[0108] When an image bitstream generated by an image encoding device (100) is input to an image decoding device (500), the input bitstream can be decoded according to a process opposite to the process performed in the image encoding device (100). In addition, an 'encoding block' in the image encoding device (100) can be referred to as a 'decoding block' in the image decoding device (500).

[0109] The entropy decoding unit (501) can interpret the bitstream transmitted from the image encoding device (100) to obtain various pieces of information and quantized transform coefficients necessary for decoding the corresponding block.

[0110] And, the entropy decoding unit (501) can perform entropy decoding in a procedure opposite to that of the entropy encoding unit (105) of the video encoding device (100) performing entropy encoding. For example, various methods such as Exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC) can be applied corresponding to the method performed in the video encoder. In the entropy decoding unit (501), the coefficients of the transform block can be decoded based on various types of flags indicating, for each partial block within the transform block, a coefficient that is not 0, a coefficient whose absolute value is greater than 1 or 2, and the sign of the coefficient. A coefficient that is not expressed by the flag alone can be decoded through the sum of the coefficient expressed by the flag and the signaled coefficient.

[0111] Additionally, the entropy decryption unit (501) can decrypt block division information and perform a restoration process on a block-by-block basis.

[0112] In addition, the entropy decoding unit (501) can decode information related to intra prediction and inter prediction performed in the image encoding device. The inverse quantization unit (502) can obtain an inverse quantized block having inverse quantized coefficients by performing the inverse quantization technique used in quantization on the quantized coefficients decoded by the entropy decoding unit (501). The inverse quantization unit (502) can operate substantially the same as the inverse quantization unit (106) of FIG. 1.

[0113] The inverse transform unit (503) can obtain a residual block having a differential signal by inversely transforming the inverse quantization transform block using the same method used during the transformation. The inverse transform unit (503) can operate substantially the same as the inverse transform unit (107) of FIG. 1.

[0114] The prediction unit (504) generates a prediction block using the coding mode information decoded by the entropy decoding unit (501), and this can use the same prediction method as that performed by the prediction unit (102) of the video encoding device (100).

[0115]

[0116] Figure 6 is a flowchart illustrating a method for decoding an optimal on-screen prediction mode of a current prediction block.

[0117] Referring to FIG. 6, an MPM candidate can be set in step S601. The method for setting the MPM candidate may be the same as the MPM candidate setting method of FIG. 3 described above in the prediction unit (102) of the video encoding device (100).

[0118] And, in step S602, information indicating whether to encode the prediction mode within the screen using MPM can be decoded.

[0119] In step S603, it is possible to determine whether MPM operation information is present based on the information decoded in step S602. If the information is true, the optimal intra-screen prediction mode of the current prediction block can be determined by decoding the index information indicating which MPM candidate the intra-screen prediction mode is identical to in step S604. Conversely, if the information is false, the intra-screen prediction mode of the current prediction block can be determined by decoding the information indicating which prediction mode is optimal among the remaining intra-screen prediction modes excluding the intra-screen prediction mode that is the MPM candidate in step S605.

[0120] The addition unit (505) can create a restoration block by adding the prediction block generated by the prediction unit (504) and the residual block generated through the inverse transformation unit (503).

[0121] The in-loop filter unit (506) performs additional filtering across the entire picture after restoring all blocks in the current picture, including deblocking filtering and SAO (Sample Adaptive Offset). Information on whether a deblocking filter has been applied to the corresponding block or picture can be received from the video encoding device (100), and if a deblocking filter has been applied, information on whether a strong filter or a weak filter has been applied. The in-loop filter unit (506) can operate substantially the same as the in-loop filter unit (109) of FIG. 1.

[0122] The memory (507) stores the restored current block that has undergone additional filtering in the in-loop filter unit (506) after adding the residual signal generated in the inverse transformation unit (503) and the prediction block generated through prediction, and can be used to predict the next block or the next picture, etc.

[0123] As described above, in the following embodiments of the present disclosure, for the sake of convenience of explanation, the term "coding unit" is used as a coding unit, but it may also be a unit that performs not only encoding but also decoding. In addition, a unit or a unit may be an area created by dividing one picture. In addition, in the embodiments of the present disclosure, a unit may mean a block, and a current block may mean a current encoding target block or a current decoding target block. In addition, a current block, an encoding block, and a decoding block may be referred to as a coding block.

[0124]

[0125] [Example 1]

[0126] This embodiment describes a method for segmenting a current image (picture) and a process for generating a prediction block by applying intra-screen prediction to a block currently being coded (encoded or decoded) (= coding block). At this time, adjacent restored pixels and non-adjacent restored pixels surrounding the current coding block can be used as reference pixels.

[0127] Figure 7 is a diagram showing a process of dividing an input image into coding block units in three steps.

[0128] Figures 8a to 8c are exemplary drawings showing the division process for each step in Figure 7.

[0129] Below, the image segmentation process is explained using FIG. 7 and FIG. 8a to FIG. 8c.

[0130] The input image of Fig. 1 can be a general rectangular YUV (YCbCr) or RGB format image. Alternatively, if the input image is not rectangular, such as a 360-degree video image or game content image, or if the image format is not YUV or RGB format, the input image can be converted into a format that can be compressed through a separate process.

[0131] The restored image of Fig. 5 can be output as a typical square-shaped YUV (YCbCr) or RGB format image. Additionally, depending on the output image, an additional process may be performed to change the image format or convert it into a 360-degree video image, game content image, etc. The process of Fig. 7 can be performed in the block division unit of Fig. 1 or the entropy decoding unit of Fig. 5.

[0132] Block segmentation using the first segmentation information (S701) may be the first step of segmenting an image. Block segmentation using the first segmentation information may be generating a first block by dividing the input image into one or more blocks. The image may be equally segmented into MxN blocks, or may be segmented into blocks of different sizes. Alternatively, the segmentation shape may not be rectangular. In addition, blocks already segmented from an image may be further segmented. That is, segmentation may be performed recursively. Alternatively, the image may not be segmented. Blocks segmented using the first segmentation information may be referred to as first blocks. The first block may be a slice, a tile, or the like. The first segmentation information may be transmitted from an encoding device to a decoding device or may be derived without information transmission using a preset method.

[0133] 801 to 804 of FIG. 8A may be examples of dividing an input image into a first block using at least one of the first information or the second information included in the first segmentation information.

[0134] 801 is an example of creating a first block by dividing the first block into equal sizes using the first information, 802 is an example of creating a first block by dividing the first block into unequal sizes using the first information, and 803 is an example of dividing an image into three first blocks in a non-rectangular shape using the second information.

[0135] 804 may be an example of creating a first block by splitting a block using second information and then further splitting the block using first information. Conversely, it may also be possible to create a first block by splitting a block using first information and then further splitting the block using second information.

[0136] Block division using second division information (S702) may be a step of division of the first block (the block obtained in block division using the first division information (S701)). The second division information may divide the input first block into one or more sub-blocks to generate a second block. The block may be divided evenly into MxN blocks, or may be divided into blocks of different sizes. In addition, the division shape may not be rectangular. In addition, blocks divided from the first block may be further divided. That is, division may be performed recursively. Alternatively, the first block may not be divided. Sub-blocks divided using the second division information may be referred to as second blocks. The second block may be a CTU, a super block, etc. The second division information may be transmitted from an encoding device to a decoding device or may be derived without information transmission by a preset method.

[0137] 805 to 808 of FIG. 8b may be examples of dividing an input first block into a second block using at least one of the first information or the second information included in the second division information.

[0138] 805 is an example of creating a second block by dividing a second block into equal-sized blocks using first information, 806 is an example of creating a second block by dividing a second block into unequal-sized blocks using first information, and 807 is an example of dividing a first block into three second blocks in a non-rectangular shape using second information.

[0139] 808 may be an example of creating a second block by splitting a block using second information and then further splitting the block using first information. Conversely, it may also be possible to create a second block by splitting a block using first information and then further splitting the block using second information.

[0140] Block segmentation using the third segmentation information (S703) may be a step of segmenting the second block (the block obtained in block segmentation using the second segmentation information (S702)). The third segmentation information may generate a third block by segmenting the input second block into one or more sub-blocks. The block may be segmented evenly into MxN blocks, or into blocks of different sizes. In addition, the segmentation shape may not be rectangular. In addition, a block segmented from the second block may be further segmented. That is, segmentation may be performed recursively. Alternatively, the second block may not be segmented. Sub-blocks segmented using the third segmentation information may be referred to as a third block. The third block may be a CU, PU, ​​TU, macroblock, etc. Alternatively, the third segmentation information may include CU segmentation information, PU segmentation information, TU segmentation information, etc. The third segmentation information may be transmitted from an encoding device to a decoding device or may be derived without transmitting information by a preset method.

[0141] 809 to 812 of FIG. 8c may be examples of dividing the input second block into third blocks using third division information.

[0142] 809 can be an example of an even division. Specifically, after splitting once with QT to create four sub-blocks, each sub-block can be further split with QT to create four sub-blocks for each sub-block. In other words, even division can be performed by recursive division. Alternatively, a division can be performed in which the second block is divided into MxN-sized sub-blocks (e.g., 16 sub-blocks of 809) all at once.

[0143] 810 may be obtained by dividing the second block into four sub-blocks (A, B, C, D) by QT division, and then performing QT division once more on the first sub-block A to divide it into four sub-blocks. The second sub-block B may be obtained by performing vertical TT division. In this case, the TT division may be performed by dividing the horizontal length of the block in a ratio of 1:2:1. The third sub-block C may be obtained by dividing it into three sub-blocks (C-1, C-2, C-3) by TT division, and then the first sub-block C-1 of the TT division may not be divided any further, the second sub-block C-2 may be divided BT in the horizontal direction, and the third sub-block C-3 may not be divided any further. The fourth sub-block D may be divided into two sub-blocks (D-1, D-2) by performing horizontal BT division, and then, for the first sub-block D-1, after horizontal BT division (D-1-1, D-1-2), sub-block D-1-1 may not be divided any further, and sub-block D-1-2 may be subject to vertical BT division. Sub-block D-2 may be subject to vertical TT division.

[0144] 811 may be a second block divided into four sub-blocks (A, B, C, D) by QT division, and then the first sub-block A may be divided into three sub-blocks (A-1, A-2, A-3) by performing vertical TT division. Sub-block A-1 may not be divided any further, and sub-block A-2 may be divided into horizontal BT division. Sub-block A-3 may not be divided any further. The second sub-block B may be divided into vertical TT division. The third sub-block C may be divided into three sub-blocks (C-1, C-2, C-3) by performing horizontal TT division, and then each sub-block may not be divided any further. The fourth sub-block D may be divided into three sub-blocks (D-1, D-2, D-3) by performing horizontal TT division, and then the first sub-block D-1 may not be divided any further, sub-block D-2 may perform vertical BT, and sub-block D-3 may not be divided any further.

[0145] 812 may be a QT split of the second block into four sub-blocks (A, B, C, D), and then the first sub-block A may be split into three sub-blocks (A-1, A-2, A-3) by performing horizontal TT splitting. Sub-block A-1 may not be split any further, and sub-block A-2 may be split into two sub-blocks (A-1-1, A-1-2) by performing horizontal BT splitting. Sub-block A-1-1 may be split into three sub-blocks (A-1-1-1, A-1-1-2, A-1-1-3) by horizontal TT splitting. Sub-block A-1-1-1 may not be split any further, sub-block A-1-1-2 may be split vertically BT splitting, and sub-block A-1-1-3 may not be split any further. Sub-block A-1-2 may be divided into three sub-blocks (A-1-2-1, A-1-2-2, A-1-2-3) by vertical TT division. Sub-block A-1-2-1 may not be divided any further, sub-block A-1-2-2 may be divided horizontally BT, and sub-block A-1-2-3 may not be divided any further. Sub-block A-3 may not be divided. The second sub-block B may be divided into two sub-blocks (B-1, B-2) by horizontal BT division. Sub-block B-1 may not be divided any further, and sub-block B-2 may be divided vertically BT to generate two sub-blocks (B-2-1, B-2-2). Sub-block B-2-1 may be divided into four sub-blocks by QT division, and sub-block B-2-2 may not be divided any further.The third sub-block C may be divided into three sub-blocks (C-1, C-2, C-3) by performing vertical TT division, C-1 may not be divided any further, sub-block C-2 may be divided into two sub-blocks (C-2-1, C-2-2) by performing horizontal BT division, and sub-block C-2-1 may be divided into two sub-blocks by performing horizontal BT division, and C-2-2 may be divided into two sub-blocks by performing vertical BT division. Sub-block C-3 may not be divided any further. The fourth sub-block D may be finally divided into four sub-blocks by performing QT division.

[0146] In the block division explanation using FIGS. 8A to 8C, it was explained that BT or TT division or QT division is possible after QT division, or after BT, TT division, but QT division can be prevented from being possible after BT, TT division by a method preset in the encoding device and the decoding device. In addition, after BT, TT division, additional division is made possible, but it may also be possible to prevent additional division from being possible by a method preset in the encoding device and the decoding device.

[0147] Coding (encoding and decoding) can be performed block by block in the third block. That is, the third block can be a coding block.

[0148] The following describes a method for generating a prediction block in units of current coding blocks input to the prediction unit (102 in FIG. 1, 504 in FIG. 5).

[0149] In-screen prediction can be used as a prediction mode to generate prediction blocks for the current coding block.

[0150]

[0151] Figure 9 is a flowchart showing the process of generating a prediction block of a current coding block in an encoding device and a decoding device.

[0152] The reference pixel determination step (S901) of FIG. 9 may determine whether to use only adjacent reference pixel lines as reference pixels of the current coding block, or to also use pixels of reference pixel lines that are not adjacent to the adjacent reference pixel lines. Information regarding whether to use only adjacent reference pixel lines as reference pixels of the current coding block, or to also use pixels of reference pixel lines that are not adjacent to the adjacent reference pixel lines, may be included in the reference pixel line information. That is, the reference pixel line of the current coding block may include at least one of a reference pixel line adjacent to the current coding block or a reference pixel line that is not adjacent to the current coding block. For example, the reference pixel line of the current coding block may include only reference pixel lines adjacent to the current coding block. For example, the reference pixel line of the current coding block may include both a reference pixel line adjacent to the current coding block and a reference pixel line that is not adjacent to the current coding block.

[0153] This may be a step of determining a reference pixel used for generating a prediction block of the current coding block among pixels restored before the current coding block.

[0154]

[0155] Figure 10 illustrates the restored pixels around the current coding block that can be used as reference pixels.

[0156] Reference pixels adjacent to the current coding block can be used as a single reference pixel line. Reference pixels located one pixel apart from the current coding block can be used as a single reference pixel line. Similarly, reference pixels located two pixels apart from the current coding block can be used as a single reference pixel line.

[0157] In order to generate a prediction block in units of coding blocks, N reference pixel lines (N=0, 1, 2, 3, 4, 5…) can be used, and the number of available reference pixel lines or whether to use a reference pixel line that is not adjacent to the current coding block is determined using a method preset in the encoding device and the decoding device, or whether to use a non-adjacent reference pixel line is transmitted from the encoding device to the decoding device at a stage higher than the current coding block, and if it is determined here to be used, whether to use a non-adjacent reference pixel line can be determined once more at the coding block stage.

[0158] The reference pixel filtering (S902) step may be a step of correcting the pixel value of the reference pixel determined in the reference pixel determination (S901) step using surrounding restoration information, block size information, etc. At this time, whether to correct the pixel value can be determined equally in the encoding / decoding device by a pre-set method. Alternatively, information regarding correction can be transmitted from the encoding device to the decoding device so that pixel correction can be performed equally in the encoding device and the decoding device.

[0159] The following may be a specific example of how to perform reference pixel filtering. Correction may be performed on pixels in a reference pixel line directly used for generating prediction blocks by the prediction mode.

[0160]

[0161] Figure 11 is an example drawing for explaining reference pixel filtering.

[0162] Referring to Fig. 11, in reference pixel filtering, an example of correcting the pixel value of a reference pixel adjacent to the current block using reference pixels that are not adjacent to the current block can be confirmed.

[0163] Fig. 11 is an example showing reference pixels surrounding the current block. Reference pixels adjacent to the current block are reference pixel line 0, pixels one pixel apart from the current block are reference pixel line 1, and N reference pixel lines can be used in this manner. In this embodiment, for the convenience of explanation, up to three reference pixel lines can be used.

[0164] The restored pixels in Fig. 11 are pixels existing at the top of the current block, and for the sake of convenience of explanation, only the top is described, but a filtering process (distortion search process and reference pixel correction) can be performed on all pixels existing at the top, left, upper left, upper right, and lower left. That is, reference pixel correction can be performed on all reference pixels used for prediction of the current block. In 1101 of Fig. 11, C may be the current reference pixel for which correction is to be performed. In addition, the current reference pixel C may be referred to as a checking pixel. r may be a restored pixel used for correction. In addition, r may be referred to as a comparison pixel. The position and number of r pixels used as comparison pixels may be used in the same manner as preset in the encoding device and the decoding device. In the present embodiment, it may be the case that 8 surrounding restored pixels are used as comparison pixels. For example, the average of the pixel values ​​of r pixels may be obtained and compared with the pixel value of pixel C.

[0165] [Equation 2]

[0166] C pixel value < weight × average value of r pixels

[0167] As compared in Equation 2, if the C pixel value is smaller, the C pixel value is used as is, and if the C pixel value is larger, the pixel value of the corresponding pixel can be replaced with the average value of r pixels. At this time, the actual restoration value of the current checking pixel does not change, and when the current checking pixel is used to generate a prediction block, only the pixel value of the reference pixel can be replaced with the average value. 1101 to 1105 of Fig. 11 can perform pixel value correction for each checking pixel using Equation 2. For the convenience of explanation, the comparison pixel existing on the left of the checking pixel in 1101 is omitted, but in reality, it can be used for reference pixel correction. In addition, the weight value can use a pre-set value in the encoding device and the decoding device without a separate transmission process. In addition, if there are less than 8 reference pixels available around the checking pixel, the values ​​of nearby comparison pixels can be copied to create 8 comparison pixels. Alternatively, even if there are less than 8, whether to correct the reference pixel can be determined using only the available comparison pixels.

[0168] Taking 1106 as an example, the original pixel value of the current checking pixel C is 58, and the average value of the surrounding comparison pixels can be calculated as (12+12+12+12+14+12+12+13) / 8 or (12+12+12+12+14+12+12+13) >> 3. Assuming that the pixel value of the checking pixel and the average value of the r pixel are 58 and 12.375, respectively, and the weight is 1.2, since the pixel value of the C pixel is greater than the average value of the r pixel multiplied by the weight, the reference pixel value of the C pixel can be replaced or corrected with 12.375.

[0169] In addition, if the checking pixel is corrected with the average value of the comparison pixel rather than the restored pixel value as the reference pixel, the checking pixel can become the comparison pixel of the next checking pixel. In this case, the original restored pixel value or the corrected value can be used as the pixel value of the comparison pixel. Which value to use can be determined using a method preset in the encoding device and the decoding device.

[0170] The following describes another method of performing reference pixel filtering at step S902.

[0171] Rather than performing reference pixel filtering on all reference pixels used in predictive block generation, filtering can be performed only on specified reference pixels. The reference pixel filtering method is explained using Fig. 11.

[0172] The restored pixels in Fig. 11 are pixels existing at the top of the current block, and for the sake of convenience of explanation, only the top is described, but a filtering process (distortion search process and reference pixel correction) can be performed on all pixels existing at the top, left, upper left, upper right, and lower left. That is, reference pixel correction can be performed on all reference pixels used for prediction of the current block. In 1101 of Fig. 11, C may be the current reference pixel for which correction is to be performed. In addition, the current reference pixel C may be referred to as a checking pixel. r may be a restored pixel used for correction. In addition, r may be referred to as a comparison pixel. The position and number of r pixels used as comparison pixels can be used in the same manner as preset in the encoding device and the decoding device. In the present embodiment, it may be the case that 8 surrounding restored pixels are used as comparison pixels. The average of the pixel values ​​of the r pixels can be obtained and compared with the pixel value of the C pixel.

[0173]

[0174] First, the encoding device can use mathematical expression 3 to check mathematical expression 3 for reference pixels within a reference pixel line adjacent to the current block.

[0175] [Equation 3]

[0176] C pixel value < weight × average value of r pixels

[0177] As shown in the comparison in mathematical expression 3, if the C pixel value is smaller, the C pixel value is used as it is, and if the C pixel value is larger, the pixel value of the corresponding pixel can be replaced with the average value of r pixels. At this time, the actual restoration value of the current checking pixel does not change, and when the current checking pixel is used to generate a prediction block, only the pixel value of the reference pixel can be replaced with the average value. 1101 to 1105 of Fig. 11 can perform pixel value correction using mathematical expression 3 for each checking pixel. For the convenience of explanation, the comparison pixel existing on the left of the checking pixel in 1101 is omitted, but in reality, it can be used for reference pixel correction. In addition, the weight value can use a pre-set value in the encoding device and the decoding device without a separate transmission process.

[0178] Using mathematical expression 3, only the location information of the filtered reference pixels can be transmitted to the decoding device.

[0179] The following describes a method for performing reference pixel filtering in a decoding device. Using the reference pixel position information received from the encoding device, the previously restored reference pixel values ​​for some reference pixels can be replaced with the average value of the comparison pixels.

[0180] The number of reference pixel positions transmitted from the encoding device to the decoding device may be the same number as the number specified by a pre-set method in the encoding device and the decoding device, or the number of reference pixels filtered by the encoding device may be transmitted to the decoding device.

[0181] Alternatively, in step S902 of FIG. 9, reference pixel filtering may be performed using another method without using the reference pixel filtering process of FIG. 10, or the reference pixel filtering process may be omitted.

[0182] The step of generating a prediction block using reference pixels and prediction modes (S903) can generate a final prediction block using reference pixel-related information and prediction-related information.

[0183] After the prediction mode and reference pixel line information are determined to generate a prediction block for the current coding block, reference pixel filtering can be performed.

[0184] At this time, the prediction mode information within the screen may include directional modes and non-directional modes, as in the example of Fig. 2.

[0185] Also, for the convenience of explanation, the numbers for multiple reference pixel lines are sequentially increased from 0, 1, 2 from adjacent reference pixel lines, but they may not increase sequentially depending on the preset method of the encoding device and decoding device. For example, the numbers may be set as 0, 3, 2, 1 from adjacent reference pixel lines, or they may be set as 3, 2, 1, 0.

[0186]

[0187] Figure 12 illustrates examples of vertical mode direction (1201), upper right diagonal mode direction (1202), and non-directional mode (1203) when the current coding block is 4x4 and the reference pixel line is 2.

[0188] 1201 and 1202 of Fig. 12 indicate that each prediction pixel C within a coding block can use three reference pixels depending on the directionality of the prediction mode. For example, if the current prediction pixel is C and the prediction information of the prediction mode within the screen is vertical mode, the reference pixels can use D0, D1, and D2.

[0189] At this time, the pixel value of the reference pixel selected by the reference pixel line information is compared with the average value of three available candidate pixels, and the selected reference pixel can be used as is or replaced with the average value instead of the selected pixel value. For example, assuming that the pixel values ​​of the D0, D1, and D2 reference pixels are 12, 10, and 14, respectively, the pixel value of the selected reference pixel can be 14, and the average value can be 12. If the pixel value of the selected reference pixel is smaller than the average value, the pixel value of the reference pixel can be used as is, and if not, the pixel value can be replaced with the average value and used to generate the prediction block.

[0190] 1203 of FIG. 12 shows that when the reference pixel line information is 2 and the current screen prediction mode is a non-directional mode such as DC or PLANAR, pixels from n2 to q2 and h2 to e2 can be used to generate a prediction block. The average value of the pixel values ​​from n2 to q2, n1 to q1, and n0 to q0 is compared with the pixel values ​​from n2 to q2, and if the pixel value of each reference pixel is smaller than the average value, the pixel value of the reference pixel is used as is, and if the pixel value is otherwise replaced with the average value, the pixel value can be used to generate a prediction block. The average value of the pixel values ​​from h2 to e2, h1 to e1, and h0 to e0 is compared with the pixel values ​​from h2 to e2, and if the pixel value of each reference pixel is smaller than the average value, the pixel value of the reference pixel is used as is, and if the pixel value is otherwise replaced with the average value, the pixel value can be used to generate a prediction block. Alternatively, the average value of the pixel values ​​from n2 to q2, n1 to q1, n0 to q0, h2 to e2, h1 to e1, and h0 to e0 can be compared with the pixel values ​​from n2 to q2 and h2 to e2. If the pixel value of each reference pixel is smaller than the average value, the pixel value of the reference pixel can be used as is, and if the opposite is true, the pixel value can be replaced with the average value and used to generate a prediction block.

[0191] Alternatively, reference pixel filtering may be performed after the prediction mode information or reference pixel line information is determined. Alternatively, the reference pixel filtering process after the reference pixel information and prediction information are determined may be omitted.

[0192]

[0193] Next, the directional prediction mode (203) of Fig. 2 is additionally described.

[0194] N modes can be used as prediction blocks for the current block. The number of directional prediction modes can be the same as that of the encoding device and the decoding device according to a pre-configured method.

[0195] If the width of the current block is greater than the height, only the vertical and vertical-direction neighboring prediction modes are used. Conversely, if the height is greater than the width, only the horizontal and horizontal-direction neighboring prediction modes are used. Alternatively, the number of available directional prediction modes may vary depending on the block size. For example, if the current block size is MxN or less, only 9 directional prediction modes are used. If the block size is greater than MxN, N directional prediction modes, which are more than 9 directions, can be used. In other words, the larger the block, the more sophisticated the directional prediction modes can be used.

[0196] Conversely, if the current block size is less than or equal to MxN, N directional prediction modes can be used, and if it exceeds MxN, a number of directional prediction modes less than N can be used.

[0197] Alternatively, the available prediction modes may vary depending on whether the current block's surrounding locations are available as reference pixels. For example, if the top or left of the current block is the image boundary, reference pixels in that area may not exist and thus may not be available. In other words, the prediction mode in that direction may not be available. Alternatively, it may be a tile boundary, a slice boundary, or an area that has not yet been restored in the coding (encoding / decoding) process.

[0198] If the upper reference pixel line is determined to be unavailable, the pixel values ​​of the upper reference pixel line can be replaced with the median pixel value and used. Alternatively, horizontal prediction can be performed using only the left reference pixel line, without using the upper reference pixel line.

[0199] Conversely, if the left reference pixel line is determined to be unavailable, the pixel values ​​of the left reference pixel line can be replaced with the median pixel value and used. Alternatively, vertical prediction can be performed using only the upper reference pixel line, without using the left reference pixel line.

[0200] Alternatively, if both the top and left reference pixel lines are unavailable, the median value of the pixels can be substituted.

[0201] Additionally, for multiple reference pixel lines, they may only be used in directional modes and not in non-directional modes. Alternatively, for multiple reference pixel lines, they may only be used in non-directional modes and not in directional modes. Alternatively, multiple reference pixel lines may not be used in some non-directional modes.

[0202] Depending on the prediction mode information within the screen, block size, and surrounding circumstances of the coding block, multiple reference pixel lines may not be available or may only be partially available.

[0203] Whether multiple reference pixel lines are available, or whether some multiple reference pixel lines are available, can be determined by a method preset in the encoding device and the decoding device, or the information can be transmitted in the upper header. Alternatively, it can be transmitted in units of coding blocks.

[0204]

[0205] The following provides additional explanation of the DC (201) mode of Fig. 2.

[0206] DC mode fills all predicted pixels within the prediction block of the current coding block with the average value. One method is to fill all predicted pixels with the average value of the pixel values ​​from a0 to q0 of 201. Alternatively, the average value of the pixel values ​​from a0 to h0 and j0 to q0 can be used as the predicted pixel value.

[0207] For the current prediction block, a prediction block can be generated in units of sub-blocks. At this time, the size of the sub-block is determined by a method preset in the encoding and decoding devices as AxB, or the size of the sub-block can be transmitted in units of the current block or from a layer higher than the current coding block, so that the encoding and decoding devices can use sub-blocks of the same size.

[0208] If the size of the sub-block is 4x4, the current prediction block can be divided into N sub-blocks, and a prediction sub-block can be generated for each sub-block in turn.

[0209] If the current block is 8x8, four sub-blocks are created. Prediction can be performed starting from the sub-block located at the upper left. In the case of the sub-block located at the upper left, since there are restored pixels at the top and left, the average of these restored pixels can be used as the predicted pixel value of the sub-block. Moving on to the next sub-block, the next sub-block may have restored pixels at the top, no restored pixels at the left, and the predicted pixels from the previous sub-block. Accordingly, the average of the restored pixels adjacent to the top and the predicted pixels adjacent to the left can be used as the predicted pixel value of the current sub-block. Moving on to the next sub-block, the next sub-block may have restored pixels at the left, no restored pixels at the top, and the predicted pixels from the previous sub-block. Accordingly, the average of the restored pixels adjacent to the left and the predicted pixels adjacent to the top can be used as the predicted pixel value of the current sub-block. Moving on to the next sub-block, the next sub-block may not have restored pixels at the top and left, but only the predicted pixels. Accordingly, the average value of these predicted pixels can be used as the pixel value of the current sub-block.

[0210] Alternatively, the available reference pixels may vary depending on the shape and size of the block. For example, if the width is greater than the height, the average of the reference pixels on the top can be used as the prediction pixel, and conversely, if the height is greater than the width, the average of the reference pixels on the left can be used as the prediction pixel. If the width and height are the same, the prediction can be performed using the average of the reference pixels on the top and the left. Alternatively, if the block size is MxN or less, the average can be obtained using both the top and left reference pixels, and if it exceeds MxN, the width and height can be compared, and if the lengths are the same, both the top and left reference pixels can be used, if the height is long, only the left reference pixels can be used, and if the width is long, only the top reference pixels can be used to perform the prediction. Alternatively, if the block size exceeds MxN, the average value can be obtained by using both the top and left reference pixels, and if the block size is less than MxN, the horizontal length and the vertical length can be compared, and if the lengths are the same, both the top and left reference pixels can be used, if the vertical length is long, only the left reference pixels can be used, and if the horizontal length is long, only the top reference pixels can be used to perform prediction. At this time, the MxN size information can be determined by a method preset in the encoding device and the decoding device or transmitted in the upper header.

[0211] Alternatively, the available reference pixels for DC mode may vary depending on whether the current block's surrounding locations are available as reference pixels. For example, if the top or left of the current block is the image border, reference pixels in that area may not exist and therefore cannot be used. Alternatively, it may be a tile border, a slice border, or an area that has not yet been restored in the coding (encoding / decoding) process.

[0212] If the upper reference pixel line is determined to be unavailable, the pixel values ​​of the upper reference pixel line can be replaced with the median pixel value and used. Alternatively, the upper reference pixel line can be omitted and prediction can be performed using only the left reference pixel line. In other words, only the pixel values ​​from a0 to h0 in 201 of Fig. 2 can be used.

[0213] Conversely, if the left reference pixel line is determined to be unavailable, the pixel values ​​of the left reference pixel line can all be replaced with the median pixel value. Alternatively, prediction can be performed using only the upper reference pixel line, without using the left reference pixel line. In other words, only the pixel values ​​from j0 to q0 in 201 of Figure 2 can be used.

[0214] Alternatively, if both the top and left reference pixel lines are unavailable, the median value of the pixels can be substituted.

[0215] Additionally, for multiple reference pixel lines, they may only be used in directional modes and not in non-directional modes. Alternatively, for multiple reference pixel lines, they may only be used in non-directional modes and not in directional modes. Alternatively, multiple reference pixel lines may not be used in some non-directional modes.

[0216] Depending on the prediction mode information within the screen, block size, and surrounding circumstances of the coding block, multiple reference pixel lines may not be available or may only be partially available.

[0217] Whether multiple reference pixel lines are available, or whether some multiple reference pixel lines are available, can be determined by a method preset in the encoding device and the decoding device, or the information can be transmitted in the upper header. Alternatively, it can be transmitted in units of coding blocks.

[0218]

[0219] Next, the PLANAR (202) mode of Fig. 2 is additionally described.

[0220] For each pixel in the current prediction block, there may be a pixel value obtained by the ratio of the distance between the reference pixel at the top and the reference pixel at the bottom, and a pixel value obtained by the ratio of the distance between the reference pixel at the left and the reference pixel at the right. The average of these two pixel values ​​may be used as the pixel value of the prediction pixel. In 202 of Fig. 2, if the current pixel is C, the n0 pixel at the top and the V pixel at the bottom may be used, and the d0 pixel at the left and the H pixel at the right may be used. At this time, depending on the position of the current pixel C, the positions and reference pixel values ​​of the upper and left reference pixels may change, but the reference pixels at the right and bottom may not change. The lower reference pixel may use a pixel located in the lower left direction based on the lower left pixel of the current block, and the right reference pixel may use a pixel located in the upper right direction based on the upper right pixel of the current block.

[0221] Alternatively, the available reference pixels and the generation of prediction blocks may vary depending on the shape and size of the block. For example, if the width of the current block is greater than the height, a prediction block may be generated using only pixels existing in the vertical direction. Conversely, if the height is greater than the width, a prediction block may be generated using only pixels existing in the horizontal direction. For example, if only pixels existing in the vertical direction are used, the prediction block of the current block may use a pixel value using the distance ratio between the upper reference pixel and the lower reference pixel as the pixel value of the prediction pixel. Conversely, if only pixels existing in the horizontal direction are used, the prediction block of the current block may use a pixel value using the distance ratio between the left reference pixel and the right reference pixel as the pixel value of the prediction pixel. If the width and height are the same, a prediction block may be generated using both vertical reference pixels and horizontal reference pixels. Alternatively, if the block size is less than or equal to MxN, a prediction block may be generated using both vertical and horizontal reference pixels, and if it exceeds MxN, the length of the width and the length of the height may be compared, and if the lengths are the same, prediction may be performed using both vertical and horizontal reference pixels, if the length is long, prediction may be performed using only horizontal reference pixels, and if the width is long, prediction may be performed using only vertical reference pixels. Alternatively, if the block size exceeds MxN, a prediction block may be generated using both vertical and horizontal reference pixels, and if it is less than or equal to MxN, prediction may be performed using both vertical and horizontal reference pixels, and if the lengths are the same, prediction may be performed using only horizontal reference pixels, and if the height is long, prediction may be performed using only vertical reference pixels. At this time, the MxN size information may be determined by a method preset in the encoding device and the decoding device or may be transmitted in the upper header.

[0222] Alternatively, the available reference pixels for PLANAR mode may vary depending on whether the current block's surrounding locations are available as reference pixels. For example, if the top or left of the current block is the image boundary, reference pixels in that area may not exist and therefore be unavailable. Alternatively, it may be the boundary of a tile, the boundary of a slice, or an area that has not yet been restored in the coding (encoding / decoding) process.

[0223] If it is determined that the upper reference pixel line is unavailable, the pixel values ​​of the upper reference pixel line can be replaced with the median value of the pixels and used. Alternatively, the upper reference pixel line can be omitted and horizontal prediction can be performed using only the left reference pixel line. However, the median value of the pixels can be used as the reference pixel on the right side of the current block, or the same pixel value can be replaced using a method preset in the encoding device and decoding device.

[0224] Conversely, if the left reference pixel line is determined to be unavailable, the pixel values ​​of the left reference pixel line can all be replaced with the median value of the pixels and used. Alternatively, the left reference pixel line is not used and only the upper reference pixel line is used to perform vertical prediction. However, the median value of the pixels can be used as the reference pixel at the bottom of the current block, or the same pixel value can be replaced using a method preset in the encoding device and decoding device.

[0225] Alternatively, if both the top and left reference pixel lines are unavailable, the median value of the pixels can be substituted.

[0226] Additionally, for multiple reference pixel lines, they may only be used in directional modes and not in non-directional modes. Alternatively, for multiple reference pixel lines, they may only be used in non-directional modes and not in directional modes. Alternatively, multiple reference pixel lines may not be used in some non-directional modes.

[0227] Depending on the prediction mode information within the screen, block size, and surrounding circumstances of the coding block, multiple reference pixel lines may not be available or may only be partially available.

[0228] Whether multiple reference pixel lines are available, or whether some multiple reference pixel lines are available, is determined by a method preset in the encoding device and decoding device, or information is transmitted in the upper header. Alternatively, it can be transmitted in units of coding blocks.

[0229]

[0230] Figures 13 and 14 are flowcharts showing the process of encoding / decoding prediction information for generating prediction blocks.

[0231] Here, the prediction information may include reference pixel correction information, including information used for prediction within the existing screen.

[0232] First, it may be a process of encoding prediction information in which the encoding device checks whether reference pixels adjacent to the current block are corrected for all reference pixels.

[0233] A prediction mode for the current block can be encoded (S1301). If inter-screen prediction is used as the prediction mode of the current block (S1302), existing inter-screen prediction information can be encoded (S1303). If intra-screen prediction is used as the prediction mode of the current block (S1302), a reference pixel correction flag can be encoded to inform a decoding device whether reference pixel correction has been performed for the current block (S1304). Reference pixel line information that indicates whether an adjacent reference pixel line or a non-adjacent reference pixel line is to be used can be encoded (S1306).

[0234] After that, the prediction information used for prediction within the screen can be encoded (S1307), and the process can be terminated.

[0235] The following may be an encoding process in which reference pixel location information is transmitted to perform filtering on only some reference pixels.

[0236] A prediction mode for the current block can be encoded (S1301). If inter-screen prediction is used as the prediction mode of the current block (S1302), existing inter-screen prediction information can be encoded (S1303). If intra-screen prediction is used as the prediction mode of the current block (S1302), a reference pixel correction flag can be encoded to inform a decoding device whether reference pixel correction has been performed for the current block (S1304). After that, a reference pixel correction index can be encoded to inform position information on which pixel among the reference pixels of the current block is to be corrected (S1305). Reference pixel line information indicating whether an adjacent reference pixel line or a non-adjacent reference pixel line is to be used can be encoded (S1306). Finally, prediction information used for intra-screen prediction can be encoded (S1307), and the process can be terminated.

[0237] It may be a process of decoding prediction information in which the decoding device checks whether reference pixels adjacent to the current block are corrected.

[0238] The prediction mode for the current block can be decoded (S1401). If inter-screen prediction is used as the prediction mode of the current block (S1402), existing inter-screen prediction information is decoded (S1403). If intra-screen prediction is used as the prediction mode of the current block (S1402), the reference pixel correction flag can be decoded to determine whether reference pixel correction has been performed for the current block (S1404). Reference pixel line information indicating whether to use an adjacent reference pixel line or a non-adjacent reference pixel line can be decoded (S1406).

[0239] After that, the prediction information used for the prediction on the screen can be decrypted (S1407), and the process can be terminated.

[0240] The following may be a decoding process in which reference pixel location information is transmitted to perform filtering on only some reference pixels.

[0241] The prediction mode for the current block can be decoded (S1401). If inter-screen prediction is used as the prediction mode of the current block (S1402), existing inter-screen prediction information is decoded (S1403). If intra-screen prediction is used as the prediction mode of the current block (S1402), the reference pixel correction flag can be decoded to determine whether reference pixel correction has been performed for the current block (S1404). After that, the reference pixel correction index can be decoded to determine the location information of which pixel among the reference pixels of the current block is to be corrected (S1405). Reference pixel line information indicating whether to use an adjacent reference pixel line or a non-adjacent reference pixel line can be decoded (S1406). Finally, the prediction information used for intra-screen prediction can be decoded (S1407), and the process can be terminated.

[0242]

[0243] [Example 2]

[0244] This embodiment describes a method for segmenting a current image (picture) and a process for generating a prediction block by applying intra-screen prediction to a block currently being coded (encoded or decoded) (= coding block). At this time, adjacent restored pixels and non-adjacent restored pixels surrounding the current coding block can be used as reference pixels.

[0245] Figure 7 is a diagram showing a process of dividing an input image into coding block units in three steps.

[0246] Figures 8a to 8c are exemplary drawings showing the division process for each step in Figure 7.

[0247] Below, the image segmentation process is explained using FIG. 7 and FIG. 8a to FIG. 8c.

[0248] The input image of Fig. 1 can be a general rectangular YUV (YCbCr) or RGB format image. Alternatively, if the input image is not rectangular, such as a 360-degree video image or game content image, or if the image format is not YUV or RGB format, the input image can be converted into a format that can be compressed through a separate process.

[0249] The restored image of Fig. 5 can be output as a typical square-shaped YUV (YCbCr) or RGB format image. Additionally, depending on the output image, an additional process may be performed to change the image format or convert it into a 360-degree video image, game content image, etc. The process of Fig. 7 can be performed in the block division unit of Fig. 1 or the entropy decoding unit of Fig. 5.

[0250] Block segmentation using the first segmentation information (S701) may be the first step of segmenting an image. Block segmentation using the first segmentation information may be generating a first block by dividing the input image into one or more blocks. The image may be equally segmented into MxN blocks, or may be segmented into blocks of different sizes. Alternatively, the segmentation shape may not be rectangular. In addition, blocks already segmented from an image may be further segmented. That is, segmentation may be performed recursively. Alternatively, the image may not be segmented. Sub-blocks segmented using the first segmentation information may be referred to as a first block. The first block may be a slice, a tile, or the like. The first segmentation information may be transmitted from an encoding device to a decoding device or may be derived without information transmission using a pre-configured method.

[0251] 801 to 804 of FIG. 8A may be examples of dividing an input image into a first block using at least one of the first information or the second information included in the first segmentation information.

[0252] 801 is an example of creating a first block by dividing the first block into equal sizes using the first information, 802 is an example of creating a first block by dividing the first block into unequal sizes using the first information, and 803 is an example of dividing an image into three first blocks in a non-rectangular shape using the second information.

[0253] 804 may be an example of creating a first block by splitting a block using second information and then further splitting the block using first information. Conversely, it may also be possible to create a first block by splitting a block using first information and then further splitting the block using second information.

[0254] Block division using second division information (S702) may be a step of division of the first block (the block obtained in block division using the first division information (S701)). The second division information may divide the input first block into one or more sub-blocks to generate a second block. The block may be divided evenly into MxN blocks, or may be divided into blocks of different sizes. In addition, the division shape may not be rectangular. In addition, blocks divided from the first block may be further divided. That is, division may be performed recursively. Alternatively, the first block may not be divided. Sub-blocks divided using the second division information may be referred to as second blocks. The second block may be a CTU, a super block, etc. The second division information may be transmitted from an encoding device to a decoding device or may be derived without information transmission by a preset method.

[0255] 805 to 808 of FIG. 8b may be examples of dividing an input first block into a second block using at least one of the first information or the second information included in the second division information.

[0256] 805 is an example of creating a second block by dividing a second block into equal-sized blocks using first information, 806 is an example of creating a second block by dividing a second block into unequal-sized blocks using first information, and 807 is an example of dividing a first block into three second blocks in a non-rectangular shape using second information.

[0257] 808 may be an example of creating a second block by splitting a block using second information and then further splitting the block using first information. Conversely, it may also be possible to create a second block by splitting a block using first information and then further splitting the block using second information.

[0258] Block segmentation using the third segmentation information (S703) may be a step of segmenting the second block (the block obtained in block segmentation using the second segmentation information (S702)). The third segmentation information may generate a third block by segmenting the input second block into one or more sub-blocks. The block may be segmented evenly in a size of MxN, or may be segmented in different sizes. In addition, the segmentation shape may not be rectangular. In addition, a block segmented from the second block may be further segmented. That is, segmentation may be performed recursively. Alternatively, the second block may not be segmented. Sub-blocks segmented using the third segmentation information may be referred to as a third block. The third block may be a CU, PU, ​​TU, macroblock, etc. Alternatively, the third segmentation information may include CU segmentation information, PU segmentation information, TU segmentation information, etc. The third segmentation information may be transmitted from an encoding device to a decoding device or may be derived without information transmission by a preset method.

[0259] 809 to 812 of FIG. 8c may be examples of dividing the input second block into third blocks using third division information.

[0260] 809 can be an example of an even partition. Specifically, after splitting once with QT to create four sub-blocks, each sub-block can be further split with QT to create four sub-blocks for each sub-block. In other words, even partitioning can be performed recursively. Alternatively, partitioning can be performed by dividing the second block into MxN-sized sub-blocks (e.g., 16 sub-blocks of 809) all at once.

[0261] 810 may be obtained by dividing the second block into four sub-blocks (A, B, C, D) by QT division, and then performing QT division once more on the first sub-block A to divide it into four sub-blocks. The second sub-block B may be obtained by performing vertical TT division. In this case, the TT division may be performed by dividing the horizontal length of the block in a ratio of 1:2:1. The third sub-block C may be obtained by dividing it into three sub-blocks (C-1, C-2, C-3) by TT division, and then the first sub-block C-1 of the TT division may not be divided any further, the second sub-block C-2 may be divided BT in the horizontal direction, and the third sub-block C-3 may not be divided any further. The fourth sub-block D may be divided into two sub-blocks (D-1, D-2) by performing horizontal BT division, and then, for the first sub-block D-1, after horizontal BT division (D-1-1, D-1-2), sub-block D-1-1 may not be divided any further, and sub-block D-1-2 may be subject to vertical BT division. Sub-block D-2 may be subject to vertical TT division.

[0262] 811 may be a second block divided into four sub-blocks (A, B, C, D) by QT division, and then the first sub-block A may be divided into three sub-blocks (A-1, A-2, A-3) by performing vertical TT division. Sub-block A-1 may not be divided any further, and sub-block A-2 may be divided into horizontal BT division. Sub-block A-3 may not be divided any further. The second sub-block B may be divided into vertical TT division. The third sub-block C may be divided into three sub-blocks (C-1, C-2, C-3) by performing horizontal TT division, and then each sub-block may not be divided any further. The fourth sub-block D may be divided into three sub-blocks (D-1, D-2, D-3) by performing horizontal TT division, and then the first sub-block D-1 may not be divided any further, sub-block D-2 may perform vertical BT, and sub-block D-3 may not be divided any further.

[0263] 812 may be a QT split of the second block into four sub-blocks (A, B, C, D), and then the first sub-block A may be split into three sub-blocks (A-1, A-2, A-3) by performing horizontal TT splitting. Sub-block A-1 may not be split any further, and sub-block A-2 may be split into two sub-blocks (A-1-1, A-1-2) by performing horizontal BT splitting. Sub-block A-1-1 may be split into three sub-blocks (A-1-1-1, A-1-1-2, A-1-1-3) by horizontal TT splitting. Sub-block A-1-1-1 may not be split any further, sub-block A-1-1-2 may be split vertically BT splitting, and sub-block A-1-1-3 may not be split any further. Sub-block A-1-2 may be divided into three sub-blocks (A-1-2-1, A-1-2-2, A-1-2-3) by vertical TT division. Sub-block A-1-2-1 may not be divided any further, sub-block A-1-2-2 may be divided horizontally BT, and sub-block A-1-2-3 may not be divided any further. Sub-block A-3 may not be divided. The second sub-block B may be divided into two sub-blocks (B-1, B-2) by horizontal BT division. Sub-block B-1 may not be divided any further, and sub-block B-2 may be divided vertically BT to generate two sub-blocks (B-2-1, B-2-2). Sub-block B-2-1 may be divided into four sub-blocks by QT division, and sub-block B-2-2 may not be divided any further.The third sub-block C may be divided into three sub-blocks (C-1, C-2, C-3) by performing vertical TT division, C-1 may not be divided any further, sub-block C-2 may be divided into two sub-blocks (C-2-1, C-2-2) by performing horizontal BT division, and sub-block C-2-1 may be divided into two sub-blocks by performing horizontal BT division, and C-2-2 may be divided into two sub-blocks by performing vertical BT division. Sub-block C-3 may not be divided any further. The fourth sub-block D may be finally divided into four sub-blocks by performing QT division.

[0264] In the block division explanation using FIGS. 8A to 8C, it was explained that BT or TT division or QT division is possible after QT division, or after BT, TT division, but QT division can be prevented from being possible after BT, TT division by a method preset in the encoding device and the decoding device. In addition, after BT, TT division, additional division is made possible, but it may also be possible to prevent additional division from being possible by a method preset in the encoding device and the decoding device.

[0265] Coding (encoding and decoding) can be performed block by block in the third block. That is, the third block can be a coding block.

[0266] The following describes a method for generating a prediction block in units of current coding blocks input to the prediction unit (102 in FIG. 1, 504 in FIG. 5).

[0267] In-screen prediction can be used as a prediction mode to generate prediction blocks for the current coding block.

[0268] Figure 9 is a flowchart showing the process of generating a prediction block of a current coding block in an encoding device and a decoding device.

[0269] The reference pixel determination step (S901) of FIG. 9 may determine whether to use only adjacent reference pixel lines as reference pixels of the current coding block, or to also use pixels of reference pixel lines that are not adjacent to the adjacent reference pixel lines. Information regarding whether to use only adjacent reference pixel lines as reference pixels of the current coding block, or to also use pixels of reference pixel lines that are not adjacent to the adjacent reference pixel lines, may be included in the reference pixel line information. That is, the reference pixel line of the current coding block may include at least one of a reference pixel line adjacent to the current coding block or a reference pixel line that is not adjacent to the current coding block. For example, the reference pixel line of the current coding block may include only reference pixel lines adjacent to the current coding block. For example, the reference pixel line of the current coding block may include both a reference pixel line adjacent to the current coding block and a reference pixel line that is not adjacent to the current coding block.

[0270] This may be a step of determining a reference pixel used for generating a prediction block of the current coding block among pixels restored before the current coding block.

[0271] Figure 10 illustrates the restored pixels around the current coding block that can be used as reference pixels.

[0272] Reference pixels adjacent to the current coding block can be used as a single reference pixel line. Reference pixels located one pixel apart from the current coding block can be used as a single reference pixel line. Similarly, reference pixels located two pixels apart from the current coding block can be used as a single reference pixel line.

[0273] In order to generate a prediction block in units of coding blocks, N reference pixel lines (N=0, 1, 2, 3, 4, 5…) can be used, and the number of available reference pixel lines or whether to use a reference pixel line that is not adjacent to the current coding block is determined using a method preset in the encoding device and the decoding device, or whether to use a non-adjacent reference pixel line is transmitted from the encoding device to the decoding device at a stage higher than the current coding block, and if it is determined here to be used, whether to use a non-adjacent reference pixel line can be determined once more at the coding block stage.

[0274] Additionally, restored pixels further away than the example in Fig. 10 can also be used as reference pixels for the current coding block.

[0275]

[0276] Figure 15 illustrates the area restored before the current coding block and the area to be restored after the current coding block.

[0277] Figure 16 shows an example of the current coding block and some restored pixels in Figure 15.

[0278] As shown in Figures 15 and 16, pixels restored prior to the current block can be divided into reference area units. At this time, the reference area can be divided by the first block. Alternatively, the reference area can be divided by the second block. Alternatively, the reference area can be divided by the third block. Alternatively, the division can be performed using a method preset in the encoding and decoding devices.

[0279] The reference pixels used for generating the prediction block of the current coding block can be determined by first determining the reference region, and then transmitting information from the encoding device to the decoding device to inform which line among the reference pixel lines within the determined reference region to use. For example, as in 1601 and 1602, the reference region information can first be checked to see which region the reference pixels used in the prediction block belong to. If the reference region information is selected as reference region 0, it can mean that some of the pixels from j0 to q0 and the pixels from l1 to s1 in 1601 are used for prediction. For the convenience of explanation, only the reference region on the top or left and two lines of reference pixel lines are shown in 1601 and 1602, but all reconstructed pixels, as in Fig. 15, can be reference pixel candidates that can be selected for generating the prediction block of the current block.

[0280] After selecting a reference area, a reference pixel line can be selected. In reference area 0, j0 to q0 can be reference pixel line 0, and l1 to s1 can be reference pixel line 1. Information on one reference pixel line selected from among N reference pixel lines can be transmitted from an encoding device to a decoding device.

[0281] The following describes another method of using reference pixels. A representative reference pixel line in each reference area can be extracted and used as N reference pixel lines. For example, in 1601 of FIG. 16, pixels j0 to q0 in reference area 0 can be reference pixel line 0, pixels l1 to s1 can be reference pixel line 1, pixels j0 to q0 in reference area 1 can be reference pixel line 2, and pixels l1 to s1 can be reference pixel line 3. If 1 is selected as the optimal reference pixel line, it can mean that l1 to s1 in reference area 0 are used as reference pixels, and if 3 is selected as the reference pixel line, it can mean that l1 to s1 in reference area 1 are used as reference pixels.

[0282] All reference pixel-related information transmitted from the encoding device to the decoding device may be reference pixel information. For example, both the reference pixel line number and the reference area may be reference pixel information. Alternatively, they may be referred to as reference information.

[0283] Next, the reference pixel filtering step (S902) of FIG. 9 may be a step of correcting the pixel value of the reference pixel determined in the reference pixel determination step (S901) using surrounding restoration information, block size information, etc. At this time, whether to correct the pixel value can be determined in the same way in the encoding / decoding device using a pre-set method. Alternatively, information regarding correction can be transmitted from the encoding device to the decoding device so that pixel correction can be performed in the same way in the encoding device and the decoding device.

[0284] The following may be a specific example of how to perform reference pixel filtering. Correction may be performed on pixels in a reference pixel line directly used for generating prediction blocks by the prediction mode.

[0285]

[0286] Figure 11 is an example drawing for explaining reference pixel filtering.

[0287] Referring to FIG. 11, in reference pixel filtering, it may be desired to correct the pixel value of a reference pixel adjacent to the current block using reference pixels that are not adjacent to the current block.

[0288] Fig. 11 is an example showing reference pixels surrounding the current block. Reference pixels adjacent to the current block are reference pixel line 0, pixels one pixel apart from the current block are reference pixel line 1, and N reference pixel lines can be used in this manner. In this embodiment, for the convenience of explanation, up to three reference pixel lines can be used.

[0289] The restored pixels in Fig. 11 are pixels existing at the top of the current block, and for the sake of convenience of explanation, only the top is described, but a filtering process (distortion search process and reference pixel correction) can be performed on all pixels existing at the top, left, upper left, upper right, and lower left. That is, reference pixel correction can be performed on all reference pixels used for prediction of the current block. In 1101 of Fig. 11, C may be the current reference pixel for which correction is to be performed. In addition, the current reference pixel C may be referred to as a checking pixel. r may be a restored pixel used for correction. In addition, r may be referred to as a comparison pixel. The position and number of r pixels used as comparison pixels may be used in the same manner as preset in the encoding device and the decoding device. In the present embodiment, it may be the case that 8 surrounding restored pixels are used as comparison pixels. For example, the average of the pixel values ​​of r pixels may be obtained and compared with the pixel value of pixel C.

[0290] [Equation 4]

[0291] C pixel value < weight × average value of r pixels

[0292] As shown in the comparison in Equation 4, if the C pixel value is smaller, the C pixel value is used as is, and if the C pixel value is larger, the pixel value of the corresponding pixel can be replaced with the average value of r pixels. At this time, the actual restoration value of the current checking pixel does not change, and when the current checking pixel is used to generate a prediction block, only the pixel value of the reference pixel can be replaced with the average value. 1101 to 1105 of Fig. 11 can perform pixel value correction for each checking pixel using Equation 4. For the convenience of explanation, the comparison pixel existing on the left of the checking pixel in 1101 is omitted, but in reality, it can be used for reference pixel correction. In addition, the weight value can use a pre-set value in the encoding device and the decoding device without a separate transmission process. In addition, if there are less than 8 reference pixels available around the checking pixel, the values ​​of nearby comparison pixels can be copied to create 8 comparison pixels. Alternatively, even if there are less than 8, whether to correct the reference pixel can be determined using only the available comparison pixels.

[0293] Taking 1106 as an example, the original pixel value of the current checking pixel C is 58, and the average value of the surrounding comparison pixels can be calculated as (12+12+12+12+14+12+12+13) / 8 or (12+12+12+12+14+12+12+13) >> 3. Assuming that the pixel value of the checking pixel and the average value of the r pixel are 58 and 12.375, respectively, and the weight is 1.2, since the pixel value of the C pixel is greater than the average value of the r pixel multiplied by the weight, the reference pixel value of the C pixel can be replaced or corrected with 12.375.

[0294] In addition, if the checking pixel is corrected with the average value of the comparison pixel rather than the restored pixel value as the reference pixel, the checking pixel can become the comparison pixel of the next checking pixel. In this case, the original restored pixel value or the corrected value can be used as the pixel value of the comparison pixel. Which value to use can be determined using a method preset in the encoding device and the decoding device.

[0295] The following describes another method of performing reference pixel filtering at step S902.

[0296] Rather than performing reference pixel filtering on all reference pixels used in predictive block generation, filtering can be performed only on specified reference pixels. The reference pixel filtering method is explained using Fig. 11.

[0297] The restored pixels in Fig. 11 are pixels existing at the top of the current block, and for the sake of convenience of explanation, only the top is described, but a filtering process (distortion search process and reference pixel correction) can be performed on all pixels existing at the top, left, upper left, upper right, and lower left. That is, reference pixel correction can be performed on all reference pixels used for prediction of the current block. In 1101 of Fig. 11, C may be the current reference pixel for which correction is to be performed. In addition, the current reference pixel C may be referred to as a checking pixel. r may be a restored pixel used for correction. In addition, r may be referred to as a comparison pixel. The position and number of r pixels used as comparison pixels can be used in the same manner as preset in the encoding device and the decoding device. In the present embodiment, it may be the case that 8 surrounding restored pixels are used as comparison pixels. The average of the pixel values ​​of the r pixels can be obtained and compared with the pixel value of the C pixel.

[0298] First, the encoding device can check Equation 5 for reference pixels within a reference pixel line adjacent to the current block using Equation 5.

[0299] [Equation 5]

[0300] C pixel value < weight × average value of r pixels

[0301] As shown in the comparison in Equation 5, if the C pixel value is smaller, the C pixel value is used as it is, and if the C pixel value is larger, the pixel value of the corresponding pixel can be replaced with the average value of r pixels. At this time, the actual restoration value of the current checking pixel does not change, and when the current checking pixel is used to generate a prediction block, only the pixel value of the reference pixel can be replaced with the average value. 1101 to 1105 of Fig. 11 can perform pixel value correction using Equation 5 for each checking pixel. For the convenience of explanation, the comparison pixel existing on the left of the checking pixel in 1101 is omitted, but in reality, it can be used for reference pixel correction. In addition, the weight value can use a pre-set value in the encoding device and the decoding device without a separate transmission process.

[0302] Using mathematical expression 5, only the location information of the filtered reference pixels can be transmitted to the decoding device.

[0303] The following describes a method for performing reference pixel filtering in a decoding device. Using the reference pixel position information received from the encoding device, the previously restored reference pixel values ​​for some reference pixels can be replaced with the average value of the comparison pixels.

[0304] The number of reference pixel positions transmitted from the encoding device to the decoding device may be the same number as the number specified by a pre-set method in the encoding device and the decoding device, or the number of reference pixels filtered by the encoding device may be transmitted to the decoding device.

[0305] Alternatively, in step S902 of FIG. 9, reference pixel filtering may be performed using another method without using the reference pixel filtering process of FIG. 10, or the reference pixel filtering process may be omitted.

[0306] The step of generating a prediction block using reference pixels and prediction modes (S903) can generate a final prediction block using reference pixel-related information and prediction-related information.

[0307] After the prediction mode and reference pixel line information are determined to generate a prediction block for the current coding block, reference pixel filtering can be performed.

[0308] At this time, the prediction mode information within the screen may include directional modes and non-directional modes, as in the example of Fig. 2.

[0309] Also, for the convenience of explanation, the numbers for multiple reference pixel lines are sequentially increased from 0, 1, 2 from adjacent reference pixel lines, but they may not increase sequentially depending on the preset method of the encoding device and decoding device. For example, the numbers may be set as 0, 3, 2, 1 from adjacent reference pixel lines, or they may be set as 3, 2, 1, 0.

[0310] Figure 12 illustrates examples of vertical mode direction (1201), upper right diagonal mode direction (1202), and non-directional mode (1203) when the current coding block is 4x4 and the reference pixel line is 2.

[0311] 1201 and 1202 of Fig. 12 indicate that each prediction pixel C within a coding block can use three reference pixels depending on the directionality of the prediction mode. For example, if the current prediction pixel is C and the prediction information of the prediction mode within the screen is vertical mode, the reference pixels can use D0, D1, and D2.

[0312] At this time, the pixel value of the reference pixel selected by the reference pixel line information is compared with the average value of three available candidate pixels, and the selected reference pixel can be used as is or replaced with the average value instead of the selected pixel value. For example, assuming that the pixel values ​​of the D0, D1, and D2 reference pixels are 12, 10, and 14, respectively, the pixel value of the selected reference pixel can be 14, and the average value can be 12. If the pixel value of the selected reference pixel is smaller than the average value, the pixel value of the reference pixel can be used as is, and if not, the pixel value can be replaced with the average value and used to generate the prediction block.

[0313] 1203 of FIG. 12 shows that when the reference pixel line information is 2 and the current screen prediction mode is a non-directional mode such as DC or PLANAR, pixels from n2 to q2 and h2 to e2 can be used to generate a prediction block. The average value of the pixel values ​​from n2 to q2, n1 to q1, and n0 to q0 is compared with the pixel values ​​from n2 to q2, and if the pixel value of each reference pixel is smaller than the average value, the pixel value of the reference pixel is used as is, and if the pixel value is otherwise replaced with the average value, the pixel value can be used to generate a prediction block. The average value of the pixel values ​​from h2 to e2, h1 to e1, and h0 to e0 is compared with the pixel values ​​from h2 to e2, and if the pixel value of each reference pixel is smaller than the average value, the pixel value of the reference pixel is used as is, and if the pixel value is otherwise replaced with the average value, the pixel value can be used to generate a prediction block. Alternatively, the average value of the pixel values ​​from n2 to q2, n1 to q1, n0 to q0, h2 to e2, h1 to e1, and h0 to e0 can be compared with the pixel values ​​from n2 to q2 and h2 to e2. If the pixel value of each reference pixel is smaller than the average value, the pixel value of the reference pixel can be used as is, and if the opposite is true, the pixel value can be replaced with the average value and used to generate a prediction block.

[0314] Alternatively, reference pixel filtering may be performed after the prediction mode information or reference pixel line information is determined. Alternatively, the reference pixel filtering process after the reference pixel information and prediction information are determined may be omitted.

[0315]

[0316] Next, the directional prediction mode (203) of Fig. 2 is additionally described.

[0317] N modes can be used as prediction blocks for the current block. The number of directional prediction modes can be the same as that of the encoding device and the decoding device according to a pre-configured method.

[0318] If the width of the current block is greater than the height, only the vertical and vertical-direction neighboring prediction modes are used. Conversely, if the height is greater than the width, only the horizontal and horizontal-direction neighboring prediction modes are used. Alternatively, the number of available directional prediction modes may vary depending on the block size. For example, if the current block size is MxN or less, only 9 directional prediction modes are used. If the block size is greater than MxN, N directional prediction modes, which are more than 9 directions, can be used. In other words, the larger the block, the more sophisticated the directional prediction modes can be used.

[0319] Conversely, if the current block size is less than or equal to MxN, N directional prediction modes can be used, and if it exceeds MxN, a number of directional prediction modes less than N can be used.

[0320] Alternatively, the available prediction modes may vary depending on whether the current block's surrounding locations are available as reference pixels. For example, if the top or left of the current block is the image boundary, reference pixels in that area may not exist and thus may not be available. In other words, the prediction mode in that direction may not be available. Alternatively, it may be a tile boundary, a slice boundary, or an area that has not yet been restored in the coding (encoding / decoding) process.

[0321] If the upper reference pixel line is determined to be unavailable, the pixel values ​​of the upper reference pixel line can be replaced with the median pixel value and used. Alternatively, horizontal prediction can be performed using only the left reference pixel line, without using the upper reference pixel line.

[0322] Conversely, if the left reference pixel line is determined to be unavailable, the pixel values ​​of the left reference pixel line can be replaced with the median pixel value and used. Alternatively, vertical prediction can be performed using only the upper reference pixel line, without using the left reference pixel line.

[0323] Alternatively, if both the top and left reference pixel lines are unavailable, the median value of the pixels can be substituted.

[0324] Additionally, for multiple reference pixel lines, they may only be used in directional modes and not in non-directional modes. Alternatively, for multiple reference pixel lines, they may only be used in non-directional modes and not in directional modes. Alternatively, multiple reference pixel lines may not be used in some non-directional modes.

[0325] Depending on the prediction mode information within the screen, block size, and surrounding circumstances of the coding block, multiple reference pixel lines may not be available or may only be partially available.

[0326] Whether multiple reference pixel lines are available, or whether some multiple reference pixel lines are available, can be determined by a method preset in the encoding device and the decoding device, or the information can be transmitted in the upper header. Alternatively, it can be transmitted in units of coding blocks.

[0327]

[0328] The following provides additional explanation of the DC (201) mode of Fig. 2.

[0329] DC mode fills all predicted pixels within the prediction block of the current coding block with the average value. One method is to fill all predicted pixels with the average value of the pixel values ​​from a0 to q0 of 201. Alternatively, the average value of the pixel values ​​from a0 to h0 and j0 to q0 can be used as the predicted pixel value.

[0330] For the current prediction block, a prediction block can be generated in units of sub-blocks. At this time, the size of the sub-block is determined by a method preset in the encoding and decoding devices as AxB, or the size of the sub-block can be transmitted in units of the current block or from a layer higher than the current coding block, so that the encoding and decoding devices can use sub-blocks of the same size.

[0331] If the size of the sub-block is 4x4, the current prediction block can be divided into N sub-blocks, and a prediction sub-block can be generated for each sub-block in turn.

[0332] If the current block is 8x8, four sub-blocks are created. Prediction can be performed starting from the sub-block located at the upper left. In the case of the sub-block located at the upper left, since there are restored pixels at the top and left, the average of these restored pixels can be used as the predicted pixel value of the sub-block. Moving on to the next sub-block, the next sub-block may have restored pixels at the top, no restored pixels at the left, and the predicted pixels from the previous sub-block. Accordingly, the average of the restored pixels adjacent to the top and the predicted pixels adjacent to the left can be used as the predicted pixel value of the current sub-block. Moving on to the next sub-block, the next sub-block may have restored pixels at the left, no restored pixels at the top, and the predicted pixels from the previous sub-block. Accordingly, the average of the restored pixels adjacent to the left and the predicted pixels adjacent to the top can be used as the predicted pixel value of the current sub-block. Moving on to the next sub-block, the next sub-block may not have restored pixels at the top and left, but only the predicted pixels. Accordingly, the average value of these predicted pixels can be used as the pixel value of the current sub-block.

[0333] Alternatively, the available reference pixels may vary depending on the shape and size of the block. For example, if the width is greater than the height, the average of the reference pixels on the top can be used as the prediction pixel, and conversely, if the height is greater than the width, the average of the reference pixels on the left can be used as the prediction pixel. If the width and height are the same, the prediction can be performed using the average of the reference pixels on the top and the left. Alternatively, if the block size is MxN or less, the average can be obtained using both the top and left reference pixels, and if it exceeds MxN, the width and height can be compared, and if the lengths are the same, both the top and left reference pixels can be used, if the height is long, only the left reference pixels can be used, and if the width is long, only the top reference pixels can be used to perform the prediction. Alternatively, if the block size exceeds MxN, the average value can be obtained by using both the top and left reference pixels, and if the block size is less than MxN, the horizontal length and the vertical length can be compared, and if the lengths are the same, both the top and left reference pixels can be used, if the vertical length is long, only the left reference pixels can be used, and if the horizontal length is long, only the top reference pixels can be used to perform prediction. At this time, the MxN size information can be determined by a method preset in the encoding device and the decoding device or transmitted in the upper header.

[0334] Alternatively, the available reference pixels for DC mode may vary depending on whether the current block's surrounding locations are available as reference pixels. For example, if the top or left of the current block is the image border, reference pixels in that area may not exist and therefore cannot be used. Alternatively, it may be a tile border, a slice border, or an area that has not yet been restored in the coding (encoding / decoding) process.

[0335] If the upper reference pixel line is determined to be unavailable, the pixel values ​​of the upper reference pixel line can be replaced with the median pixel value and used. Alternatively, the upper reference pixel line can be omitted and prediction can be performed using only the left reference pixel line. In other words, only the pixel values ​​from a0 to h0 in 201 of Fig. 2 can be used.

[0336] Conversely, if the left reference pixel line is determined to be unavailable, the pixel values ​​of the left reference pixel line can all be replaced with the median pixel value. Alternatively, prediction can be performed using only the upper reference pixel line, without using the left reference pixel line. In other words, only the pixel values ​​from j0 to q0 in 201 of Figure 2 can be used.

[0337] Alternatively, if both the top and left reference pixel lines are unavailable, the median value of the pixels can be substituted.

[0338] Additionally, for multiple reference pixel lines, they may only be used in directional modes and not in non-directional modes. Alternatively, for multiple reference pixel lines, they may only be used in non-directional modes and not in directional modes. Alternatively, multiple reference pixel lines may not be used in some non-directional modes.

[0339] Depending on the prediction mode information within the screen, block size, and surrounding circumstances of the coding block, multiple reference pixel lines may not be available or may only be partially available.

[0340] Whether multiple reference pixel lines are available, or whether some multiple reference pixel lines are available, can be determined by a method preset in the encoding device and the decoding device, or the information can be transmitted in the upper header. Alternatively, it can be transmitted in units of coding blocks.

[0341]

[0342] Next, the PLANAR (202) mode of Fig. 2 is additionally described.

[0343] For each pixel in the current prediction block, there may be a pixel value obtained by the ratio of the distance between the reference pixel at the top and the reference pixel at the bottom, and a pixel value obtained by the ratio of the distance between the reference pixel at the left and the reference pixel at the right. The average of these two pixel values ​​may be used as the pixel value of the prediction pixel. In 202 of Fig. 2, if the current pixel is C, the n0 pixel at the top and the V pixel at the bottom may be used, and the d0 pixel at the left and the H pixel at the right may be used. At this time, depending on the position of the current pixel C, the positions and reference pixel values ​​of the upper and left reference pixels may change, but the reference pixels at the right and bottom may not change. The lower reference pixel may use a pixel located in the lower left direction based on the lower left pixel of the current block, and the right reference pixel may use a pixel located in the upper right direction based on the upper right pixel of the current block.

[0344] Alternatively, the available reference pixels and the generation of prediction blocks may vary depending on the shape and size of the block. For example, if the width of the current block is greater than the height, a prediction block may be generated using only pixels existing in the vertical direction. Conversely, if the height is greater than the width, a prediction block may be generated using only pixels existing in the horizontal direction. For example, if only pixels existing in the vertical direction are used, the prediction block of the current block may use a pixel value using the distance ratio between the upper reference pixel and the lower reference pixel as the pixel value of the prediction pixel. Conversely, if only pixels existing in the horizontal direction are used, the prediction block of the current block may use a pixel value using the distance ratio between the left reference pixel and the right reference pixel as the pixel value of the prediction pixel. If the width and height are the same, a prediction block may be generated using both vertical reference pixels and horizontal reference pixels. Alternatively, if the block size is less than or equal to MxN, a prediction block may be generated using both vertical and horizontal reference pixels, and if it exceeds MxN, the length of the width and the length of the height may be compared, and if the lengths are the same, prediction may be performed using both vertical and horizontal reference pixels, if the length is long, prediction may be performed using only horizontal reference pixels, and if the width is long, prediction may be performed using only vertical reference pixels. Alternatively, if the block size exceeds MxN, a prediction block may be generated using both vertical and horizontal reference pixels, and if it is less than or equal to MxN, prediction may be performed using both vertical and horizontal reference pixels, and if the lengths are the same, prediction may be performed using only horizontal reference pixels, and if the height is long, prediction may be performed using only vertical reference pixels. At this time, the MxN size information may be determined by a method preset in the encoding device and the decoding device or may be transmitted in the upper header.

[0345] Alternatively, the available reference pixels for PLANAR mode may vary depending on whether the current block's surrounding locations are available as reference pixels. For example, if the top or left of the current block is the image boundary, reference pixels in that area may not exist and therefore be unavailable. Alternatively, it may be the boundary of a tile, the boundary of a slice, or an area that has not yet been restored in the coding (encoding / decoding) process.

[0346] If it is determined that the upper reference pixel line is unavailable, the pixel values ​​of the upper reference pixel line can be replaced with the median value of the pixels and used. Alternatively, the upper reference pixel line can be omitted and horizontal prediction can be performed using only the left reference pixel line. However, the median value of the pixels can be used as the reference pixel on the right side of the current block, or the same pixel value can be replaced using a method preset in the encoding device and decoding device.

[0347] Conversely, if the left reference pixel line is determined to be unavailable, the pixel values ​​of the left reference pixel line can all be replaced with the median value of the pixels and used. Alternatively, the left reference pixel line is not used and only the upper reference pixel line is used to perform vertical prediction. However, the median value of the pixels can be used as the reference pixel at the bottom of the current block, or the same pixel value can be replaced using a method preset in the encoding device and decoding device.

[0348] Alternatively, if both the top and left reference pixel lines are unavailable, the median value of the pixels can be substituted.

[0349] Additionally, for multiple reference pixel lines, they may only be used in directional modes and not in non-directional modes. Alternatively, for multiple reference pixel lines, they may only be used in non-directional modes and not in directional modes. Alternatively, multiple reference pixel lines may not be used in some non-directional modes.

[0350] Depending on the prediction mode information within the screen, block size, and surrounding circumstances of the coding block, multiple reference pixel lines may not be available or may only be partially available.

[0351] Whether multiple reference pixel lines are available, or whether some multiple reference pixel lines are available, is determined by a method preset in the encoding device and decoding device, or information is transmitted in the upper header. Alternatively, it can be transmitted in units of coding blocks.

[0352]

[0353] Figures 17 and 18 are flowcharts showing the process of encoding / decoding prediction information for generating prediction blocks.

[0354] Here, the prediction information may include information about a method for determining a reference pixel among reference pixel candidates, including information used for prediction within an existing screen.

[0355] First, in Fig. 17, the prediction mode for the current block can be encoded (S1701). If inter-screen prediction is used as the prediction mode of the current block (S1702), existing inter-screen prediction information can be encoded (S1703), and if intra-screen prediction is used as the prediction mode of the current block (S1702), reference pixel related information can be encoded first (S1704). If the reference areas of Figs. 15 and 16 are used as reference pixel candidates, the reference area information can be encoded, and then reference pixel line information can be encoded. Alternatively, if N lines adjacent to the current block are used as reference pixel candidates, or if representative reference pixel lines for each reference area are extracted and used, only the reference pixel line information can be encoded.

[0356] Thereafter, the prediction information used for on-screen prediction can be encoded (S1705). The on-screen prediction information refers to the directional prediction mode and non-directional prediction mode of FIG. 2, and information on one of the selected prediction modes can be transmitted. The process can then be terminated.

[0357] The following may be a decoding process. In Fig. 18, the prediction mode for the current block can be decoded (S1801). If inter-screen prediction is used as the prediction mode of the current block (S1802), the existing inter-screen prediction information is decoded (S1803), and if intra-screen prediction is used as the prediction mode of the current block (S1802), reference pixel related information can be decoded first (S1804). If the reference areas of Figs. 15 and 16 are used as reference pixel candidates, the reference area information can be decoded, and then the reference pixel line information can be decoded. Alternatively, if N lines adjacent to the current block are used as reference pixel candidates, or if representative reference pixel lines are extracted and used for each reference area, only the reference pixel line information can be decoded.

[0358] Thereafter, the prediction information used for the on-screen prediction can be decoded (S1805). The on-screen prediction information refers to the directional prediction mode and non-directional prediction mode of FIG. 2, and information on one of the selected prediction modes can be decoded from the bitstream. This process ends.

[0359]

[0360] [Example 3]

[0361] This embodiment describes a method for segmenting a current image (picture) and a process for generating a prediction block by applying intra-screen prediction to a block currently being coded (encoded or decoded) (= coding block). At this time, adjacent restored pixels and non-adjacent restored pixels surrounding the current coding block can be used as reference pixels.

[0362] Figure 7 is a diagram showing a process of dividing an input image into coding block units in three steps.

[0363] Figures 8a to 8c are exemplary drawings showing the division process for each step in Figure 7.

[0364] Below, the image segmentation process is explained using FIG. 7 and FIG. 8a to FIG. 8c.

[0365] The input image of Fig. 1 can be a general rectangular YUV (YCbCr) or RGB format image. Alternatively, if the input image is not rectangular, such as a 360-degree video image or game content image, or if the image format is not YUV or RGB format, the input image can be converted into a format that can be compressed through a separate process.

[0366] The restored image of Fig. 5 can be output as a typical square-shaped YUV (YCbCr) or RGB format image. Additionally, depending on the output image, an additional process may be performed to change the image format or convert it into a 360-degree video image, game content image, etc. The process of Fig. 7 can be performed in the block division unit of Fig. 1 or the entropy decoding unit of Fig. 5.

[0367] Block segmentation using the first segmentation information (S701) may be the first step of segmenting an image. Block segmentation using the first segmentation information may be generating a first block by dividing the input image into one or more blocks. The image may be equally segmented into MxN blocks, or may be segmented into blocks of different sizes. Alternatively, the segmentation shape may not be rectangular. In addition, blocks already segmented from an image may be further segmented. That is, segmentation may be performed recursively. Alternatively, the image may not be segmented. Sub-blocks segmented using the first segmentation information may be referred to as a first block. The first block may be a slice, a tile, or the like. The first segmentation information may be transmitted from an encoding device to a decoding device or may be derived without information transmission using a pre-configured method.

[0368] 801 to 804 of FIG. 8A may be examples of dividing an input image into a first block using at least one of the first information or the second information included in the first segmentation information.

[0369] 801 is an example of creating a first block by dividing the first block into equal sizes using the first information, 802 is an example of creating a first block by dividing the first block into unequal sizes using the first information, and 803 is an example of dividing an image into three first blocks in a non-rectangular shape using the second information.

[0370] 804 may be an example of creating a first block by splitting a block using second information and then further splitting the block using first information. Conversely, it may also be possible to create a first block by splitting a block using first information and then further splitting the block using second information.

[0371] Block division using second division information (S702) may be a step of division of the first block (the block obtained in block division using the first division information (S701)). The second division information may divide the input first block into one or more sub-blocks to generate a second block. The block may be divided evenly into MxN blocks, or may be divided into blocks of different sizes. In addition, the division shape may not be rectangular. In addition, blocks divided from the first block may be further divided. That is, division may be performed recursively. Alternatively, the first block may not be divided. Sub-blocks divided using the second division information may be referred to as second blocks. The second block may be a CTU, a super block, etc. The second division information may be transmitted from an encoding device to a decoding device or may be derived without information transmission by a preset method.

[0372] 805 to 808 of FIG. 8b may be examples of dividing an input first block into a second block using at least one of the first information or the second information included in the second division information.

[0373] 805 is an example of creating a second block by dividing a second block into equal-sized blocks using first information, 806 is an example of creating a second block by dividing a second block into unequal-sized blocks using first information, and 807 is an example of dividing a first block into three second blocks in a non-rectangular shape using second information.

[0374] 808 may be an example of creating a second block by splitting a block using second information and then further splitting the block using first information. Conversely, it may also be possible to create a second block by splitting a block using first information and then further splitting the block using second information.

[0375] Block segmentation using the third segmentation information (S703) may be a step of segmenting the second block (the block obtained in block segmentation using the second segmentation information (S702)). The third segmentation information may generate a third block by segmenting the input second block into one or more sub-blocks. The block may be segmented evenly in a size of MxN, or may be segmented in different sizes. In addition, the segmentation shape may not be rectangular. In addition, a block segmented from the second block may be further segmented. That is, segmentation may be performed recursively. Alternatively, the second block may not be segmented. Sub-blocks segmented using the third segmentation information may be referred to as a third block. The third block may be a CU, PU, ​​TU, macroblock, etc. Alternatively, the third segmentation information may include CU segmentation information, PU segmentation information, TU segmentation information, etc. The third segmentation information may be transmitted from an encoding device to a decoding device or may be derived without information transmission by a preset method.

[0376] 809 to 812 of FIG. 8c may be examples of dividing the input second block into third blocks using third division information.

[0377] 809 can be an example of an even partition. Specifically, after splitting once with QT to create four sub-blocks, each sub-block can be further split with QT to create four sub-blocks for each sub-block. In other words, even partitioning can be performed recursively. Alternatively, partitioning can be performed by dividing the second block into MxN-sized sub-blocks (e.g., 16 sub-blocks of 809) all at once.

[0378] 810 may be obtained by dividing the second block into four sub-blocks (A, B, C, D) by QT division, and then performing QT division once more on the first sub-block A to divide it into four sub-blocks. The second sub-block B may be obtained by performing vertical TT division. In this case, the TT division may be performed by dividing the horizontal length of the block in a ratio of 1:2:1. The third sub-block C may be obtained by dividing it into three sub-blocks (C-1, C-2, C-3) by TT division, and then the first sub-block C-1 of the TT division may not be divided any further, the second sub-block C-2 may be divided BT in the horizontal direction, and the third sub-block C-3 may not be divided any further. The fourth sub-block D may be divided into two sub-blocks (D-1, D-2) by performing horizontal BT division, and then, for the first sub-block D-1, after horizontal BT division (D-1-1, D-1-2), sub-block D-1-1 may not be divided any further, and sub-block D-1-2 may be subject to vertical BT division. Sub-block D-2 may be subject to vertical TT division.

[0379] 811 may be a second block divided into four sub-blocks (A, B, C, D) by QT division, and then the first sub-block A may be divided into three sub-blocks (A-1, A-2, A-3) by performing vertical TT division. Sub-block A-1 may not be divided any further, and sub-block A-2 may be divided into horizontal BT division. Sub-block A-3 may not be divided any further. The second sub-block B may be divided into vertical TT division. The third sub-block C may be divided into three sub-blocks (C-1, C-2, C-3) by performing horizontal TT division, and then each sub-block may not be divided any further. The fourth sub-block D may be divided into three sub-blocks (D-1, D-2, D-3) by performing horizontal TT division, and then the first sub-block D-1 may not be divided any further, sub-block D-2 may perform vertical BT, and sub-block D-3 may not be divided any further.

[0380] 812 may be a QT split of the second block into four sub-blocks (A, B, C, D), and then the first sub-block A may be split into three sub-blocks (A-1, A-2, A-3) by performing horizontal TT splitting. Sub-block A-1 may not be split any further, and sub-block A-2 may be split into two sub-blocks (A-1-1, A-1-2) by performing horizontal BT splitting. Sub-block A-1-1 may be split into three sub-blocks (A-1-1-1, A-1-1-2, A-1-1-3) by horizontal TT splitting. Sub-block A-1-1-1 may not be split any further, sub-block A-1-1-2 may be split vertically BT splitting, and sub-block A-1-1-3 may not be split any further. Sub-block A-1-2 may be divided into three sub-blocks (A-1-2-1, A-1-2-2, A-1-2-3) by vertical TT division. Sub-block A-1-2-1 may not be divided any further, sub-block A-1-2-2 may be divided horizontally BT, and sub-block A-1-2-3 may not be divided any further. Sub-block A-3 may not be divided. The second sub-block B may be divided into two sub-blocks (B-1, B-2) by horizontal BT division. Sub-block B-1 may not be divided any further, and sub-block B-2 may be divided vertically BT to generate two sub-blocks (B-2-1, B-2-2). Sub-block B-2-1 may be divided into four sub-blocks by QT division, and sub-block B-2-2 may not be divided any further.The third sub-block C may be divided into three sub-blocks (C-1, C-2, C-3) by performing vertical TT division, C-1 may not be divided any further, sub-block C-2 may be divided into two sub-blocks (C-2-1, C-2-2) by performing horizontal BT division, and sub-block C-2-1 may be divided into two sub-blocks by performing horizontal BT division, and C-2-2 may be divided into two sub-blocks by performing vertical BT division. Sub-block C-3 may not be divided any further. The fourth sub-block D may be finally divided into four sub-blocks by performing QT division.

[0381] In the block division explanation using FIGS. 8A to 8C, it was explained that BT or TT division or QT division is possible after QT division, or after BT, TT division, but QT division can be prevented from being possible after BT, TT division by a method preset in the encoding device and the decoding device. In addition, after BT, TT division, additional division is made possible, but it may also be possible to prevent additional division from being possible by a method preset in the encoding device and the decoding device.

[0382] Coding (encoding and decoding) can be performed block by block in the third block. That is, the third block can be a coding block.

[0383] The following describes a method for generating a prediction block in units of current coding blocks input to the prediction unit (102 in FIG. 1, 504 in FIG. 5).

[0384] In-screen prediction can be used as a prediction mode to generate prediction blocks for the current coding block.

[0385]

[0386] Figure 19 is a flowchart showing the process of generating a prediction block of a current coding block in an encoding device and a decoding device.

[0387] The reference pixel determination step (S1901) of FIG. 19 may determine reference pixel candidates of the current coding block, and may determine whether to use only adjacent reference pixel lines among the reference pixel candidates as reference pixels used for generating a prediction block, or to also use pixels of reference pixel lines that are not adjacent to the adjacent reference pixel lines. In other words, this may be a step for determining reference pixels used for generating a prediction block of the current coding block among pixels restored before the current coding block.

[0388] Figure 10 illustrates the restored pixels around the current coding block that can be used as reference pixels.

[0389] Reference pixels adjacent to the current coding block can be used as a single reference pixel line. Reference pixels located one pixel apart from the current coding block can be used as a single reference pixel line. Similarly, reference pixels located two pixels apart from the current coding block can be used as a single reference pixel line.

[0390] In order to generate a prediction block in units of coding blocks, N reference pixel lines (N=0, 1, 2, 3, 4, 5…) can be used, and the number of available reference pixel lines or whether to use a reference pixel line that is not adjacent to the current coding block is determined using a method preset in the encoding device and the decoding device, or whether to use a non-adjacent reference pixel line is transmitted from the encoding device to the decoding device at a stage higher than the current coding block, and if it is determined here to be used, whether to use a non-adjacent reference pixel line can be determined once more at the coding block stage.

[0391] Additionally, restored pixels further away than the example in Fig. 10 can also be used as reference pixels for the current coding block.

[0392] Figure 15 illustrates the area restored before the current coding block and the area to be restored after the current coding block.

[0393] Figure 16 shows an example of the current coding block and some restored pixels in Figure 15.

[0394] As shown in Figures 15 and 16, pixels restored prior to the current block can be divided into reference area units. At this time, the reference area can be divided by the first block. Alternatively, the reference area can be divided by the second block. Alternatively, the reference area can be divided by the third block. Alternatively, the division can be performed using a method preset in the encoding and decoding devices.

[0395] The reference pixels used for generating the prediction block of the current coding block can be determined by first determining the reference region, and then transmitting information from the encoding device to the decoding device to inform which line among the reference pixel lines within the determined reference region to use. For example, as in 1601 and 1602, the reference region information can first be checked to determine which region the reference pixels used in the prediction block belong to. If the reference region information is selected as reference region 0, this may mean that some of the pixels from j0 to q0 and the pixels from l1 to s1 in 1601 are used for prediction. For the convenience of explanation, only the reference region on the top or left and two lines of reference pixel lines are shown in 1601 and 1602, but all reconstructed pixels, as in Fig. 15, may be reference pixel candidates that can be selected for generating the prediction block of the current block.

[0396] After selecting a reference area, a reference pixel line can be selected. In reference area 0, j0 to q0 can be reference pixel line 0, and l1 to s1 can be reference pixel line 1. Information on one reference pixel line selected from among N reference pixel lines can be transmitted from an encoding device to a decoding device.

[0397] The following describes another method of using reference pixels. A representative reference pixel line in each reference area can be extracted and used as N reference pixel lines. For example, in 1601 of FIG. 16, pixels j0 to q0 in reference area 0 can be reference pixel line 0, pixels l1 to s1 can be reference pixel line 1, pixels j0 to q0 in reference area 1 can be reference pixel line 2, and pixels l1 to s1 can be reference pixel line 3. If 1 is selected as the optimal reference pixel line, it can mean that l1 to s1 in reference area 0 are used as reference pixels, and if 3 is selected as the reference pixel line, it can mean that l1 to s1 in reference area 1 are used as reference pixels.

[0398] All reference pixel-related information transmitted from the encoding device to the decoding device may be reference pixel information. For example, both the reference pixel line number and the reference area may be reference pixel information. Alternatively, they may be referred to as reference information.

[0399]

[0400] FIG. 20 and FIG. 21 are diagrams for explaining another method regarding a reference pixel candidate determination process and a method for determining a reference pixel used for prediction among the reference pixel candidates.

[0401] Fig. 20 is an example of dividing the first block and the second block, which are upper blocks than the current coding block, and Fig. 21 may be an example drawing showing a case where the upper reference pixel line and the left reference pixel line are set differently.

[0402] Block division example 1 (2001) may be a case where the current coding block is adjacent to the boundary of the first block. Since the restoration pixels surrounding the current coding block cross the boundary of the first block, the surrounding restoration pixels may not be used as reference pixels, and the reference pixels may be replaced with median values ​​that the current restoration pixels may have. Alternatively, only the reference pixel line adjacent to the current coding block may be used as reference pixels, and the pixel values ​​of the remaining reference pixel lines may be replaced with median values. Alternatively, only the adjacent reference pixel lines may be used as reference pixels. For example, if the pixel values ​​that the restoration pixels may have are between 0 and 255, the median value may be 128, and if the pixel values ​​that the restoration pixels may have are between 0 and 1023, the median value may be 512. In addition, the first block may be a tile or a slice, etc. Block division example 2 (2002) may be a case where the current coding block is adjacent to the boundary of the second block. Since the restored pixels around the current coding block cross the boundary of the second block, instead of using the surrounding restored pixels as reference pixels, the reference pixels can be replaced with the median value of the values ​​that the current restored pixels can have. Alternatively, only the reference pixel line adjacent to the current coding block can be used as reference pixels, and the pixel values ​​of the remaining reference pixel lines can be replaced with the median value. Alternatively, only the adjacent reference pixel lines can be used as reference pixels. For example, if the pixel values ​​that the restored pixels can have are between 0 and 255, the median value can be 128, and if the pixel values ​​that the restored pixels can have are between 0 and 1023, the median value can be 512. In addition, the second block can be a CTU, a Super block, etc., and can mean a higher layer than the current coding block.

[0403] Block segmentation example 3 (2003) may be a case where the upper boundary of the current coding block is adjacent to the boundary of the second block. Since the upper reconstructed pixels of the current coding block cross the boundary of the second block, the upper reconstructed pixels may not be used as reference pixels, and the reference pixels may be replaced with the median value of the values ​​that the current reconstructed pixels can have. Alternatively, only the reconstructed pixels adjacent to the upper end of the current coding block may be used as reference pixels, and the median value of non-adjacent reconstructed pixels may be replaced. Alternatively, only the reference pixel lines adjacent to the upper end of the current block may be used, and non-adjacent reference pixel lines may not be used. For example, if the pixel values ​​that the reconstructed pixels can have are between 0 and 255, the median value may be 128, and if the pixel values ​​that the reconstructed pixels can have are between 0 and 1023, the median value may be 512. In addition, the second block may be a CTU, a Super block, etc., and may mean a higher layer than the current coding block. Additionally, since the left side of the current block is not adjacent to the boundaries of the first and second blocks, N reference pixel lines can be used.

[0404] Block segmentation example 4 (2004) may be a case where the left boundary of the current coding block is adjacent to the boundary of the second block. Since the left restoration pixels of the current coding block cross the boundary of the second block, the left restoration pixels may not be used as reference pixels, and the reference pixels may be replaced with the median value of the values ​​that the current restoration pixels can have. Alternatively, only the restoration pixels adjacent to the left side of the current coding block may be used as reference pixels, and the median value of the non-adjacent restoration pixels may be replaced. Alternatively, only the reference pixel lines adjacent to the left side of the current block may be used, and the non-adjacent reference pixel lines may not be used. For example, if the pixel values ​​that the restoration pixels can have are between 0 and 255, the median value may be 128, and if the pixel values ​​that the restoration pixels can have are between 0 and 1023, the median value may be 512. In addition, the second block may be a CTU, a Super block, etc., and may mean a higher layer than the current coding block. Additionally, since the top of the current block is not adjacent to the boundaries of the first and second blocks, N reference pixel lines can be used.

[0405] The following describes another method of generating a prediction block in step S903 of FIG. 9 using block division example 3 (2003) and block division example 4 (2004).

[0406] Only the upper part of the current coding block may be adjacent to the boundary of the second block, or only the left part of the current coding block may be adjacent to the boundary of the second block, or the upper and left parts of the current coding block may be adjacent to the boundary of the second block. In this case, if the left side of the current coding block is adjacent to the boundary of the second block, the left reference pixel may be used as is with the N reference pixel lines. If the upper part of the current coding block is adjacent to the boundary of the second block, the upper reference pixel may only use the adjacent reference pixels.

[0407]

[0408] Figure 21 illustrates the reference pixel lines available at the top and left when there are multiple reference pixel lines and the current coding block is in the upper left diagonal prediction mode.

[0409] The process of generating a prediction block is described in detail using Fig. 21. For example, if the left and top of the current coding block are adjacent to the boundary of the second block, and the prediction mode of the prediction within the current screen is a mode indicating the upper left diagonal direction, and the reference pixel line uses 2, the left reference pixels are L2 to S2, and the top reference pixel line uses adjacent reference pixels from A0 to K0 to generate a prediction block. For convenience of explanation, the top reference pixel line is referred to as A0 to K0, but the top reference pixels from C0 to K0 can be used, and the left reference pixel lines from A1, A0, and L2 to S2 can be used.

[0410] The following describes another method for determining reference pixel candidates and reference pixels used in prediction blocks among the reference pixel candidates.

[0411]

[0412] FIG. 22a and FIG. 22b are drawings for explaining a method of using diagonal reference pixel lines in multiple directions, including vertical and horizontal directions, rather than selecting reference pixel lines only in the vertical or horizontal direction.

[0413] You can select the upper reference pixel line and the left reference pixel line separately, or select the upper and left reference pixel lines as one reference pixel line.

[0414] 2201 and 2202 may be example drawings for selecting horizontal and vertical reference pixel lines while selecting an upper reference pixel line and a left reference pixel line, respectively. The upper reference pixel line of 2201 and 2202 may be a reference pixel line consisting of pixels between the pixel pointed to by the upper left pixel and the pixel pointed to by the upper right pixel of the current coding block. The left reference pixel line may be a reference pixel line consisting of pixels between the pixel pointed to by the upper left pixel and the pixel pointed to by the lower left pixel of the current coding block. The upper reference pixel line and the left reference pixel line may completely overlap or may only partially overlap, and information regarding which reference pixels among reference pixel candidates are selected may need to be transmitted from the encoding device to the decoding device. For example, the upper reference pixel line may transmit position information of the pixel pointed to by the upper left pixel and position information of the pixel pointed to by the upper right pixel within the current coding block. The left reference pixel line can transmit position information of the pixel pointed to by the upper left pixel and the pixel pointed to by the lower left pixel within the current coding block. Additionally, the positions of the pixels pointed to by the upper left, upper right, and lower left pixels may not be integer positions.

[0415] 2203 and 2204 are example drawings in which diagonal reference pixel lines are selected. In 2203, only one reference pixel line is selected, and in 2204, the upper reference pixel line and the left reference pixel line may be selected, respectively.

[0416] The upper reference pixel line of 2204 may be a reference pixel line consisting of pixels between the pixel pointed to by the upper left pixel and the pixel pointed to by the upper right pixel of the current coding block. The left reference pixel line may be a reference pixel line consisting of pixels between the pixel pointed to by the upper left pixel and the pixel pointed to by the lower left pixel of the current coding block. The upper reference pixel line and the left reference pixel line may completely overlap or may only partially overlap, and may need to transmit information from the encoding device to the decoding device regarding which reference pixels among the reference pixel candidates are selected. For example, the upper reference pixel line may transmit position information of the pixel pointed to by the upper left pixel and position information of the pixel pointed to by the upper right pixel within the current coding block. The left reference pixel line may transmit position information of the pixel pointed to by the upper left pixel and position information of the pixel pointed to by the lower left pixel within the current coding block. In addition, the positions of the pixels pointed to by the upper left, upper right, and lower left pixels may not be integer positions.

[0417] The reference pixel line of 2203 can be a single reference pixel line consisting of pixels between the pixel pointed to by the upper left pixel and the pixel pointed to by the upper right pixel of the current coding block, and pixels between the pixel pointed to by the upper left pixel and the pixel pointed to by the lower left pixel of the current coding block. The pixels between the pixel pointed to by the lower left pixel, the pixel pointed to by the upper left pixel, and the pixel pointed to by the upper right pixel may be a straight line, or may be in a bent shape as in 2203. Information regarding which reference pixels among the reference pixel candidates are selected may need to be transmitted from the encoding device to the decoding device. For example, information regarding the position of the pixel pointed to by the upper left pixel, the pixel pointed to by the upper right pixel, and the pixel pointed to by the lower left pixel in the current coding block may be transmitted. In addition, the positions of the pixels pointed to by the upper left, upper right, and lower left pixels may not be integer positions.

[0418] When generating a prediction block using 2201 to 2204, directional prediction mode or non-directional prediction mode can be used. In the case of directional prediction mode, depending on the direction of the selected prediction mode, the pixel values ​​at integer positions within the reference pixel line can be used as they are, or if the positions are not integers, interpolation can be performed using surrounding reference pixels. Interpolation can use a ratio of distances, the pixel value of the nearest integer pixel, or a cubic filter, etc. In the case of non-directional mode, all pixels within the reference pixel line can be used, or pixel values ​​equal to the number of horizontal pixels or the number of vertical pixels of the current coding block can be used. For example, if the current block is 4x4, four pixel values ​​from the upper reference pixel line and four pixel values ​​from the left reference pixel line can be used. Since the pixels at both ends of the reference pixel line are the points pointed to by the upper left and upper right pixels within the coding block, or the points pointed to by the upper left and lower left pixels, only the two pixel values ​​in the middle need to be found. At this time, the two pixel values ​​can be used by dividing the distance between the pixels at both ends into four equal parts and using the two pixel positions in the middle. If the pixel positions are integer pixel positions, the pixel values ​​can be used as is, and if they are not integer pixel positions, interpolation can be performed to obtain the pixel values. At this time, the interpolation method determined can be performed using a method preset in the encoding device and decoding device.

[0419] Next, the reference pixel filtering step (S1902) of FIG. 19 may be a step of correcting the pixel value of the reference pixel determined in the reference pixel determination step (S1901) using surrounding restoration information, block size information, etc. At this time, whether to correct the pixel value can be determined in the same way in the encoding / decoding device by a pre-set method. Alternatively, information regarding the correction can be transmitted from the encoding device to the decoding device, so that pixel correction can be performed in the same way in the encoding device and the decoding device.

[0420] The following is a specific example of how to perform reference pixel filtering. Correction can be performed on pixels in the reference pixel line directly used to generate prediction blocks by the prediction mode.

[0421]

[0422] Figure 11 is an example drawing for explaining reference pixel filtering.

[0423] Referring to FIG. 11, in reference pixel filtering, it may be desired to correct the pixel value of a reference pixel adjacent to the current block using reference pixels that are not adjacent to the current block.

[0424] Fig. 11 is an example showing reference pixels surrounding the current block. Reference pixels adjacent to the current block are reference pixel line 0, pixels one pixel apart from the current block are reference pixel line 1, and N reference pixel lines can be used in this manner. In this embodiment, for the convenience of explanation, up to three reference pixel lines can be used.

[0425] The restored pixels in Fig. 11 are pixels existing at the top of the current block, and for the sake of convenience of explanation, only the top is described, but a filtering process (distortion search process and reference pixel correction) can be performed on all pixels existing at the top, left, upper left, upper right, and lower left. That is, reference pixel correction can be performed on all reference pixels used for prediction of the current block. In 1101 of Fig. 11, C may be the current reference pixel for which correction is to be performed. In addition, the current reference pixel C may be referred to as a checking pixel. r may be a restored pixel used for correction. In addition, r may be referred to as a comparison pixel. The position and number of r pixels used as comparison pixels may be used in the same manner as preset in the encoding device and the decoding device. In the present embodiment, it may be the case that 8 surrounding restored pixels are used as comparison pixels. For example, the average of the pixel values ​​of r pixels may be obtained and compared with the pixel value of pixel C.

[0426] [Equation 6]

[0427] c pixel value < weight × average value of r pixels

[0428] As shown in the comparison in Equation 6, if the C pixel value is smaller, the C pixel value is used as is, and if the C pixel value is larger, the pixel value of the corresponding pixel can be replaced with the average value of r pixels. At this time, the actual restoration value of the current checking pixel does not change, and when the current checking pixel is used to generate a prediction block, only the pixel value of the reference pixel can be replaced with the average value. 1101 to 1105 of Fig. 11 can perform pixel value correction for each checking pixel using Equation 6. For the convenience of explanation, the comparison pixel existing on the left of the checking pixel in 1101 is omitted, but in reality, it can be used for reference pixel correction. In addition, the weight value can use a pre-set value in the encoding device and the decoding device without a separate transmission process. In addition, if there are less than 8 reference pixels available around the checking pixel, the values ​​of nearby comparison pixels can be copied to create 8 comparison pixels. Alternatively, even if there are less than 8, whether to correct the reference pixel can be determined using only the available comparison pixels.

[0429] Taking 1106 as an example, the original pixel value of the current checking pixel C is 58, and the average value of the surrounding comparison pixels can be calculated as (12+12+12+12+14+12+12+13) / 8 or (12+12+12+12+14+12+12+13) >> 3. Assuming that the pixel value of the checking pixel and the average value of the r pixel are 58 and 12.375, respectively, and the weight is 1.2, since the pixel value of the C pixel is greater than the average value of the r pixel multiplied by the weight, the reference pixel value of the C pixel can be replaced or corrected with 12.375.

[0430] In addition, if the checking pixel is corrected with the average value of the comparison pixel rather than the restored pixel value as the reference pixel, the checking pixel can become the comparison pixel of the next checking pixel. In this case, the original restored pixel value or the corrected value can be used as the pixel value of the comparison pixel. Which value to use can be determined using a method preset in the encoding device and the decoding device.

[0431] The following describes another method of performing reference pixel filtering at step S902.

[0432] Rather than performing reference pixel filtering on all reference pixels used in predictive block generation, filtering can be performed only on specified reference pixels. The reference pixel filtering method is explained using Fig. 11.

[0433] The restored pixels in Fig. 11 are pixels existing at the top of the current block, and for the sake of convenience of explanation, only the top is described, but a filtering process (distortion search process and reference pixel correction) can be performed on all pixels existing at the top, left, upper left, upper right, and lower left. That is, reference pixel correction can be performed on all reference pixels used for prediction of the current block. In 1101 of Fig. 11, C may be the current reference pixel for which correction is to be performed. In addition, the current reference pixel C may be referred to as a checking pixel. r may be a restored pixel used for correction. In addition, r may be referred to as a comparison pixel. The position and number of r pixels used as comparison pixels can be used in the same manner as preset in the encoding device and the decoding device. In the present embodiment, it may be the case that 8 surrounding restored pixels are used as comparison pixels. The average of the pixel values ​​of the r pixels can be obtained and compared with the pixel value of the C pixel.

[0434]

[0435] First, the encoding device can check Equation 7 for reference pixels within a reference pixel line adjacent to the current block using Equation 7.

[0436] [Equation 7]

[0437] c pixel value < weight × average value of r pixels

[0438] As shown in the comparison in Equation 7, if the C pixel value is smaller, the C pixel value is used as it is, and if the C pixel value is larger, the pixel value of the corresponding pixel can be replaced with the average value of r pixels. At this time, the actual restoration value of the current checking pixel does not change, and when the current checking pixel is used to generate a prediction block, only the pixel value of the reference pixel can be replaced with the average value. 1101 to 1105 of Fig. 11 can perform pixel value correction for each checking pixel using Equation 7. For the convenience of explanation, the comparison pixel existing on the left of the checking pixel in 1101 is omitted, but in reality, it can be used for reference pixel correction. In addition, the weight value can use a pre-set value in the encoding device and the decoding device without a separate transmission process.

[0439] Using mathematical expression 7, only the location information of the filtered reference pixels can be transmitted to the decoding device.

[0440] The following describes a method for performing reference pixel filtering in a decoding device. Using the reference pixel position information received from the encoding device, the previously restored reference pixel values ​​for some reference pixels can be replaced with the average value of the comparison pixels.

[0441] The number of reference pixel positions transmitted from the encoding device to the decoding device may be the same number as the number specified by a pre-set method in the encoding device and the decoding device, or the number of reference pixels filtered by the encoding device may be transmitted to the decoding device.

[0442] Alternatively, in step S902 of FIG. 9, reference pixel filtering may be performed using another method without using the reference pixel filtering process of FIG. 10, or the reference pixel filtering process may be omitted.

[0443] The step of generating a prediction block using reference pixels and prediction modes (S903) can generate a final prediction block using reference pixel-related information and prediction-related information.

[0444] After the prediction mode and reference pixel line information are determined to generate a prediction block for the current coding block, reference pixel filtering can be performed.

[0445] At this time, the prediction mode information within the screen may include directional modes and non-directional modes, as in the example of Fig. 2.

[0446] Also, for the convenience of explanation, the numbers for multiple reference pixel lines are sequentially increased from 0, 1, 2 from adjacent reference pixel lines, but they may not increase sequentially depending on the preset method of the encoding device and decoding device. For example, the numbers may be set as 0, 3, 2, 1 from adjacent reference pixel lines, or they may be set as 3, 2, 1, 0.

[0447] Figure 12 illustrates examples of vertical mode direction (1201), upper right diagonal mode direction (1202), and non-directional mode (1203) when the current coding block is 4x4 and the reference pixel line is 2.

[0448] 1201 and 1202 of Fig. 12 indicate that each prediction pixel C within a coding block can use three reference pixels depending on the directionality of the prediction mode. For example, if the current prediction pixel is C and the prediction information of the prediction mode within the screen is vertical mode, the reference pixels can use D0, D1, and D2.

[0449] At this time, the pixel value of the reference pixel selected by the reference pixel line information is compared with the average value of three available candidate pixels, and the selected reference pixel can be used as is or replaced with the average value instead of the selected pixel value. For example, assuming that the pixel values ​​of the D0, D1, and D2 reference pixels are 12, 10, and 14, respectively, the pixel value of the selected reference pixel can be 14, and the average value can be 12. If the pixel value of the selected reference pixel is smaller than the average value, the pixel value of the reference pixel can be used as is, and if not, the pixel value can be replaced with the average value and used to generate the prediction block.

[0450] 1203 of FIG. 12 shows that when the reference pixel line information is 2 and the current screen prediction mode is a non-directional mode such as DC or PLANAR, pixels from n2 to q2 and h2 to e2 can be used to generate a prediction block. The average value of the pixel values ​​from n2 to q2, n1 to q1, and n0 to q0 is compared with the pixel values ​​from n2 to q2, and if the pixel value of each reference pixel is smaller than the average value, the pixel value of the reference pixel is used as is, and if the pixel value is otherwise replaced with the average value, the pixel value can be used to generate a prediction block. The average value of the pixel values ​​from h2 to e2, h1 to e1, and h0 to e0 is compared with the pixel values ​​from h2 to e2, and if the pixel value of each reference pixel is smaller than the average value, the pixel value of the reference pixel is used as is, and if the pixel value is otherwise replaced with the average value, the pixel value can be used to generate a prediction block. Alternatively, the average value of the pixel values ​​from n2 to q2, n1 to q1, n0 to q0, h2 to e2, h1 to e1, and h0 to e0 can be compared with the pixel values ​​from n2 to q2 and h2 to e2. If the pixel value of each reference pixel is smaller than the average value, the pixel value of the reference pixel can be used as is, and if the opposite is true, the pixel value can be replaced with the average value and used to generate a prediction block.

[0451] Alternatively, reference pixel filtering may be performed after the prediction mode information or reference pixel line information is determined. Alternatively, the reference pixel filtering process after the reference pixel information and prediction information are determined may be omitted.

[0452]

[0453] The following describes various methods for generating prediction blocks using directional prediction modes and reference pixels.

[0454] Figures 30a and 30b are diagrams showing various reference pixel lines and prediction modes.

[0455] A method for setting a reference pixel line and a directional prediction mode used in a current prediction block at once is described. Based on the reference pixel pointed to by the upper left pixel of the current prediction block and the reference pixel pointed to by the upper right pixel of the current prediction block, the pixels existing between them can be used as reference pixel lines. At this time, the reference pixel line can be divided into equal parts by the number of horizontal pixels of the current block so that each prediction pixel points to a reference pixel. For example, in 3001 and 3003 of FIGS. 30A and 30B , when the current block is a 4x4 block, the reference pixel pointed to by the upper left pixel and the reference pixel pointed to by the upper right pixel are set as both ends of a reference pixel line, and multiple reference pixels can exist between them. Since the number of horizontal pixels of the current block is 4, the reference pixel line can be divided into four parts so that each prediction pixel points to a reference pixel. At this time, the reference pixel pointed to by the current prediction pixel may or may not be an integer position. If it is not an integer position, interpolation can be performed using pixels at surrounding integer positions. As in 3002 and 3004 of FIGS. 30A and 30B, the pixel values ​​of the prediction pixels of the next line may need to be obtained. At that time, by comparing the horizontal length of the reference pixel line and the block, the angle of the prediction direction pointed by each pixel can be slightly adjusted. For example, if the length of the reference pixel line is longer than the horizontal length of the block, the angle of the prediction direction used for each prediction pixel can be narrowed. Conversely, if the length of the reference pixel line is shorter than the horizontal length of the block, the angle of the prediction direction used for each prediction pixel can be widened. In addition, if the horizontal length of the block and the length of the reference pixel line are the same, the prediction direction angle can be used as is without adjusting it.

[0456] That is, once the reference pixel line is determined, the prediction direction can be automatically determined for each prediction pixel. At this time, additional prediction direction adjustment information can also be transmitted. For example, information can be transmitted to adjust the prediction direction of each current prediction pixel to slightly more to the right.

[0457] Next, we describe another method for generating prediction blocks.

[0458] As in 3001 of Fig. 30a, a reference pixel line can be generated using a reference pixel pointed to by the upper left pixel of the current block and a reference pixel pointed to by the upper right pixel. At this time, the directional prediction mode can only use a prediction mode in a direction that uses the determined reference pixel line. For example, if the reference pixel line is determined horizontally at the top of the current block, only a vertical mode can be used, and if the reference pixel line is determined vertically to the left of the current block, only a horizontal mode can be used. Alternatively, if a diagonal reference pixel line is generated, as in 2203 of Fig. 22b, a diagonal prediction mode can be used, and if there are two or more reference pixel lines, as in 2204, a prediction block can be generated using each reference pixel line and the prediction mode pointing to the reference pixel line, and then the prediction blocks can be weighted and combined to generate a final prediction block.

[0459]

[0460] The following provides additional explanation of the DC (201) mode of Fig. 2.

[0461] DC mode fills all predicted pixels within the prediction block of the current coding block with the average value. One method is to fill all predicted pixels with the average value of the pixel values ​​from a0 to q0 of 201. Alternatively, the average value of the pixel values ​​from a0 to h0 and j0 to q0 can be used as the predicted pixel value.

[0462] Figures 31a to 31c are drawings for explaining a method of generating a prediction block in sub-block units using non-adjacent reference pixel lines.

[0463] For the current prediction block, a prediction block can be generated in units of sub-blocks. At this time, the size of the sub-block is determined by a method preset in the encoding and decoding devices as AxB, or the size of the sub-block can be transmitted in units of the current block or from a layer higher than the current coding block, so that the encoding and decoding devices can use sub-blocks of the same size.

[0464] 3101 to 3104 of FIGS. 31a and 31b may represent a process for explaining a case where the current block is 8x8, the sub-block is 4x4, and number 1 is selected as the reference pixel line.

[0465] If the size of the sub-block is 4x4, the current prediction block can be divided into N sub-blocks, and a prediction sub-block can be generated for each sub-block in turn.

[0466] If the current block is 8x8, 4 sub-blocks can be created.

[0467] As in 3101, prediction can be performed starting from the sub-block located at the upper left. In the case of the sub-block located at the upper left, since there are restored pixels at the top and left, the average value of pixels e1 to h1 and l1 to o1 can be used as the predicted pixel value of the sub-block. Moving on to the next sub-block, the current sub-block of 3102 has restored pixels at the top, no restored pixels at the left, and a predicted pixel from the previous sub-block may exist. The average value of pixels p1 to s1 at the top and the predicted pixels C3, C7, C11, and C15 adjacent to the left can be used as the predicted pixel value of the current sub-block. Moving on to the next sub-block, the current sub-block of 3103 has restored pixels at the left, no restored pixels at the top, and a predicted pixel from the previous sub-block may exist. The average value of pixels a1 to d1 at the left and the predicted pixels C12 to C15 adjacent to the top are used as the predicted pixel value of the current sub-block. Moving on to the next sub-block, the current sub-block 3104 may have only predicted pixels, without restoration pixels on the top and left. The average value of the predicted pixels from C12 to C15, D3, D7, D11, and D15 can be used as the pixel value of the current sub-block.

[0468]

[0469] The following describes a prediction mode using the DC (201) mode of Fig. 2.

[0470] For the current prediction block, a prediction block can be generated in units of sub-blocks. At this time, the size of the sub-block is determined by a method preset in the encoding and decoding devices as AxB, or the size of the sub-block can be transmitted in units of the current block or from a layer higher than the current coding block, so that the encoding and decoding devices can use sub-blocks of the same size.

[0471] As shown in 3102 of Fig. 31a, the size of the sub-block can be set to 4x4. At this time, in order to obtain the pixel value of each prediction pixel, reference pixels around the current coding block can be used. In order to obtain the pixel value of the C0 pixel, the average value of the e0 reference pixel to the h0 reference pixel can be obtained. At this time, instead of simply taking the average, a weighted sum is performed using a weight coefficient. For example, the e0 pixel value times the weight coefficient information 1, the f0 pixel value times the weight coefficient information 2, the g0 pixel value times the weight coefficient information 3, and in this way, the pixel values ​​up to m0 are multiplied by the weight coefficient value to obtain the average. At this time, the pixel existing at the lower left of the current sub-block may be used. Each weight coefficient may have a different value depending on the location of the sub-block, the distance from the reference pixel, etc., and there may be a weight coefficient set by a method preset in the encoding device and the decoding device. Alternatively, the weight coefficient information may be transmitted.

[0472] After generating a pixel value for pixel C0 using pixels from e0 to m0 and weight coefficients, the same process can be repeated for the next predicted pixel (C1). At this time, the weight coefficients of C1 and C0 can be different. That is, C1 and C0 can use different sets of weight coefficients. Different sets of weight coefficients can be used for each predicted pixel position. After generating a predicted pixel value for predicted pixels from C0 to C15, the process can be moved to the next sub-block. The same process as the previous sub-block can be performed in the next sub-block. At this time, if there are no restored pixels around the sub-block, the predicted pixels can be used. The same process can be performed for each sub-block to generate one predicted block.

[0473] Additionally, as in 3105 and 3106 of Fig. 31c, a prediction block of a sub-block can be generated using non-adjacent reference pixels for each sub-block.

[0474] Alternatively, the available reference pixels may vary depending on the shape and size of the block. For example, if the width is greater than the height, the average of the reference pixels on the top can be used as the prediction pixel, and conversely, if the height is greater than the width, the average of the reference pixels on the left can be used as the prediction pixel. If the width and height are the same, the prediction can be performed using the average of the reference pixels on the top and the left. Alternatively, if the block size is MxN or less, the average can be obtained using both the top and left reference pixels, and if it exceeds MxN, the width and height can be compared, and if the lengths are the same, both the top and left reference pixels can be used, if the height is long, only the left reference pixels can be used, and if the width is long, only the top reference pixels can be used to perform the prediction. Alternatively, if the block size exceeds MxN, the average value can be obtained by using both the top and left reference pixels, and if the block size is less than MxN, the horizontal length and the vertical length can be compared, and if the lengths are the same, both the top and left reference pixels can be used, if the vertical length is long, only the left reference pixels can be used, and if the horizontal length is long, only the top reference pixels can be used to perform prediction. At this time, the MxN size information can be determined by a method preset in the encoding device and the decoding device or transmitted in the upper header.

[0475] Alternatively, the available reference pixels for DC mode may vary depending on whether the current block's surrounding locations are available as reference pixels. For example, if the top or left of the current block is the image border, reference pixels in that area may not exist and therefore cannot be used. Alternatively, it may be a tile border, a slice border, or an area that has not yet been restored in the coding (encoding / decoding) process.

[0476] If the upper reference pixel line is determined to be unavailable, the pixel values ​​of the upper reference pixel line can be replaced with the median pixel value and used. Alternatively, the upper reference pixel line can be omitted and prediction can be performed using only the left reference pixel line. In other words, only the pixel values ​​from a0 to h0 in 201 of Fig. 2 can be used.

[0477] Conversely, if the left reference pixel line is determined to be unavailable, the pixel values ​​of the left reference pixel line can all be replaced with the median pixel value. Alternatively, the left reference pixel line is not used, and prediction is performed using only the upper reference pixel line. In other words, only the pixel values ​​from j0 to q0 in 201 of Figure 2 can be used.

[0478] Alternatively, if both the top and left reference pixel lines are unavailable, the median value of the pixels can be substituted.

[0479]

[0480] The following describes various methods for generating prediction blocks using the PLANAR prediction mode.

[0481] Figures 32a to 32e are diagrams showing reference pixel lines used in PLANAR prediction mode and reference pixels used on the right and bottom.

[0482] 3201 and 3202 of FIGS. 32a and 32b may represent cases where non-adjacent reference pixel lines are used as reference pixels of the current block. If a non-adjacent reference pixel line is selected to generate a predicted pixel value for the current predicted pixel C, the predicted pixel C, the upper reference pixel, and the left reference pixel may all be determined to be reference pixels that are not adjacent to the current block. In addition, the distance by which the selected reference pixel is separated from the current block may be additionally calculated. That is, since the distance between p1 and C in the upper reference pixel is longer than the distance between n0 and C, the corresponding increased distance may need to be reflected in the distance ratio.

[0483] At this time, even if non-adjacent reference pixels are used for the upper and left reference pixels, adjacent reference pixels V0 and H0 are used for the reference pixels used at the bottom and right (3201). Alternatively, as in 3202 of Fig. 32, when non-adjacent reference pixels are used as the upper or left reference pixels, the lower left pixel and upper right pixel existing in the same reference pixel line can be used as the lower and right reference pixels. For example, since non-adjacent reference pixels p1 and d1 are used for the current prediction pixel C, the pixels used at the right and bottom can be pixels of H1 and V1 existing in the same reference pixel line.

[0484] 3203 and 3204 of FIGS. 32c and 32d may be processes for generating prediction blocks using only one direction, rather than using both vertical and horizontal directions when generating prediction blocks using the PLANAR mode. Prediction information may be transmitted from the encoding device to the decoding device so that only one direction of reference pixels may be used in the PLANAR prediction mode. Alternatively, only one direction may be used by a pre-set method in the encoding device and the decoding device. For example, if the upper or left boundary of the current block is adjacent to the boundary of the image, only one direction may be used. In cases where only vertical or horizontal reference pixels are used, as in 3203 and 3204, the lower or right reference pixels used may vary depending on whether reference pixels adjacent to the block are used. For example, when non-adjacent reference pixels are used, that is, when reference pixel line 1 is selected, the lower reference pixel V may be selected as V1. Or, the right reference pixel H may be used as pixel H1.

[0485] Alternatively, regardless of the reference pixel lines used on the top and left, the bottom reference pixel V and the right reference pixel H may only use adjacent reference pixels. That is, only adjacent reference pixels V0 and H0 may be used.

[0486] 3205 of Fig. 32e may represent a case where a diagonal reference pixel line is used, rather than just a vertical or horizontal reference pixel line. The two end pixels of the reference pixel line existing at the top may be the pixels pointed to by the upper left pixel and the upper right pixel of the current block, and the two end pixels of the pixel line existing on the left may be the pixels pointed to by the upper left pixel and the lower left pixel of the current block. At this time, the position of the pixel pointed to by each pixel may or may not be an integer position. If it is not an integer position, the pixel value can be generated by performing interpolation using the pixels of the surrounding integer positions.

[0487] The positions of the reference pixels used to generate the prediction block within each reference pixel line can be selected as positions equally divided by the number of horizontal pixels and the number of vertical pixels. In other words, the positions of the reference pixels pointed to by the prediction pixels may not be integer positions. The reference pixels used on the right and bottom can be taken from adjacent reference pixel lines. As in 3205, the upper and lower reference pixels used with the current pixel C, or the left and right reference pixels used with the current pixel C may not exist in a straight line. In this case, the prediction pixels can be generated using the ratio of the distances, assuming that they are in a straight line.

[0488]

[0489] For multiple reference pixel lines, they may be used only in directional modes and not in non-directional modes. Alternatively, for multiple reference pixel lines, they may be used only in non-directional modes and not in directional modes. Alternatively, multiple reference pixel lines may not be used in some non-directional modes.

[0490] Depending on the prediction mode information within the screen, block size, and surrounding circumstances of the coding block, multiple reference pixel lines may not be available or may only be partially available.

[0491] Whether multiple reference pixel lines are available, or whether some multiple reference pixel lines are available, can be determined by a method preset in the encoding device and the decoding device, or the information can be transmitted in the upper header. Alternatively, it can be transmitted in units of coding blocks.

[0492] Finally, filtering can be performed on the prediction block generated in S1903 (S1904). That is, prediction pixel value correction can be performed on the prediction block using surrounding restoration information. The surrounding restoration information can include restored pixel values ​​prior to the current coding block, prediction mode information, or pixels within the prediction block of the current coding block.

[0493] Prediction block filtering may include a first prediction block filtering method and a second prediction block filtering method. For convenience of explanation, only two filtering methods are described; more prediction block filtering techniques may exist. In addition, the first prediction block filtering and the second prediction block filtering may be applied sequentially to the prediction block, or only one filtering method may be applied. Whether filtering is applied to the prediction block may be determined by a method preset in the encoding device and the decoding device. For example, it may be determined by the block size, prediction mode information within the screen, the presence or absence of reference pixels, etc. Alternatively, information on whether filtering is applied may be transmitted from the encoding device to the decoding device.

[0494]

[0495] Figure 23 is a diagram for explaining the first prediction block filtering method.

[0496] Prediction pixel value correction can be performed on all prediction pixels in the prediction block of the current coding block, or correction can be performed on only some prediction pixel values. For example, the current block can be divided into sub-blocks of MxN size, and prediction pixel value correction can be performed for each sub-block. Here, M and N can be sizes determined by a method preset in the encoding device and the decoding device. Alternatively, M and N can be transmitted from the encoding device to the decoding device. For example, regardless of the block size, sub-blocks of 4x4 or 4x2 size can be used, or depending on the block size, if the block is larger than 16x16, the sub-block size can be 4x4, and if the block size is 16x16 or less, the sub-block size can be 4x2.

[0497] 2301 of FIG. 23 may be a diagram showing a 4x4 sub-block and surrounding pixels. C0 to c15 may be predicted pixel values, and r0 to r9 may be surrounding pixel values ​​of the current sub-block. 2302 may be a current prediction block with a size of 4x4 and a sub-block with a size of 4x4. 2303 may be a current prediction block with a size of 8x8 and a sub-block with a size of 4x4. 2304 may be a current prediction block with a size of 16x8 and a sub-block with a size of 4x4.

[0498] It is possible to determine whether correction is to be performed for each predicted pixel c0 to c15 within a sub-block. If it is determined that correction is to be performed for the current pixel within the sub-block, correction of the predicted pixel value can be performed using r0 to r9 around the current sub-block.

[0499] The following describes the correction method in detail. After obtaining the difference by subtracting the pixel value of c0 from the pixel value of r0, the offset value can be obtained by multiplying the difference by a weighting coefficient. Next, using the same process, the difference by subtracting the pixel value of c0 from the pixel value of r1 can be obtained, and the offset value can be obtained by multiplying the difference by a weighting coefficient. Next, using the same process, the difference by subtracting the pixel value of c0 from the pixel value of r2 can be obtained, and the offset value can be obtained by multiplying the difference by a weighting coefficient. The offsets for the reference pixels r3 to r8 can be obtained using the same process. This generates nine offset values ​​for c0, and these nine offset values ​​can be averaged to generate a final offset value. The final generated offset value can then be added to the pixel value of c0 to obtain the final corrected pixel value. If it is determined that correction should be performed on the currently predicted pixels for c1 to c15, the correction process for c0 can be repeated in the same manner. At this time, the weight coefficient can exist as many as the number of surrounding pixels used for correction for the prediction pixel in each sub-block. For example, in the current example 2301, the sub-block is 4x4 and the number of surrounding reference pixels is 9, so the number of weight coefficients can be 144. Among the multiple weight coefficients, one of the weight prediction coefficients used for the current prediction pixel can be determined according to the position of the current reference pixel. In addition, the value of the weight coefficient can vary depending on the distance between the current prediction pixel and the reference pixel. For example, since the pixel distance between c0 and r5 is closer than the pixel distance between c1 and r5, the value of the weight coefficient applied to r5 of c0 can be greater than the value of the weight coefficient applied to r5 of c1. In addition, since the pixel distance between c1 and r2 is closer than the pixel distance between c1 and r5, the weight coefficient of r2 can be greater than that of r5.Additionally, the weight coefficients may have values ​​of weight coefficients of reference pixels existing in the vertical and horizontal directions that are greater than the values ​​of weight coefficients existing in the diagonal direction.

[0500] After performing the predicted pixel value correction for the current sub-block, the correction method used in 2301 can be used for the next sub-block. However, the pixel values ​​of r0 to r9 used as reference pixels become the restored pixel values ​​around the current coding block when the sub-block is adjacent to the boundary of the current block, and when the sub-block is not adjacent to the boundary of the current block, the pixels existing at the right boundary or the bottom boundary of the previous sub-block can become the reference pixels.

[0501] Also, depending on the size of the current block, the S1903 process may be omitted. Also, depending on the size of the current block and the position of the sub-block, the correction process may be omitted. Also, depending on the size of the current block and the position of the sub-block and a specific threshold, the correction process may be omitted. The specific thresholds may be set separately for the horizontal and vertical thresholds of the block. The horizontal threshold may be set to 1 if the horizontal length of the current block is 4, 2 if it is 8, and 4 if it is 16 or more. The vertical threshold may be set to 1 if the horizontal length of the current block is 4, 2 if it is 8, and 4 if it is 16 or more. For example, if the vertical or horizontal size of the current block is 32 or more, the prediction block filtering process of the current coding block may be omitted. For the convenience of explanation, 32 is specified, but if it is a specific size other than 32, the process may be omitted. In addition, if the width and height of the current block are less than 32 and greater than 8, filtering can be performed only on the sub-blocks existing on the left and top within the current prediction block. As in the example of 2304, filtering can be performed only on the sub-blocks existing on the top and left, and the remaining sub-blocks can be omitted without performing filtering. Alternatively, if the current block is 8x8 or smaller, filtering can be performed only on the sub-blocks existing on the left and top within the current block. In addition, filtering can be performed only on pixel positions smaller than a specific threshold among the pixels within the sub-blocks determined to be filtered. In the case where the current block is 4x4, as in 2302, the width and height thresholds are each 1, so filtering can be performed only on the prediction pixels at the edge positions smaller than the threshold among the pixel positions. In the case where the current block is 8x8, as in 2303, filtering can be performed only on the edge sub-blocks, and reference pixel filtering can be performed only on the prediction pixels at positions smaller than the threshold among them.In Fig. 23, O represents a pixel position where prediction pixel correction is performed, and X and SKIP represent positions where prediction pixel correction is not performed.

[0502]

[0503] The following describes the second prediction block filtering.

[0504] The second prediction block filtering can correct the predicted pixel values ​​using the upper reference pixels and the left reference pixels of the current coding block. First, the predicted pixel values ​​can be corrected using the upper reference pixels. The predicted pixel for which correction is currently to be performed is multiplied by a first weight coefficient, the upper reference pixel is multiplied by a second weight coefficient, and then a weighted sum is performed to obtain the corrected predicted pixel values. At this time, the first weight coefficient may be a weight coefficient determined by a method preset in the encoding device and the decoding device. The second weight coefficient may be a value obtained by subtracting the first weight coefficient from 1. For each predicted pixel, a weighted sum with the upper reference pixel can be performed to obtain the corrected pixel values. Next, the predicted pixel values ​​can be corrected using the left reference pixels. The predicted pixel for which correction is currently to be performed is multiplied by the first weight coefficient, the left reference pixel is multiplied by the second weight coefficient, and then a weighted sum is performed to obtain the corrected predicted pixel values. At this time, the first weight coefficient may be a weight coefficient determined by a method preset in the encoding device and the decoding device. The second weight coefficient may be a value obtained by subtracting the first weight coefficient from 1. For each predicted pixel, a weighted sum may be performed with the left reference pixel to obtain a corrected pixel value.

[0505] At this time, the second prediction block filtering may perform correction on all pixels in the prediction block or may perform correction on only some pixels. The number of prediction pixels on which correction is performed may be determined by a method preset in the encoding device and the decoding device. For example, the number of prediction pixels on which correction is performed may vary depending on the horizontal or vertical size of the block. When the vertical length is 4, correction can be performed only on the prediction pixel line existing at the top within the prediction block. When the horizontal length is 4, correction can be performed only on the prediction pixel line existing at the left within the prediction block. That is, correction can be performed only on the prediction pixel indicated by O in 2301 of FIG. 23. If the horizontal or vertical length of the block is 4, correction can be performed on only 1 line, 2 lines if it is 8, 4 lines if it is 16, 8 lines if it is 32, and 16 lines if it is 64.

[0506] Additionally, the value of the weight coefficient may vary depending on the distance between the predicted pixel and the upper or left reference pixel. Alternatively, the same weight coefficient value may be used regardless of the predicted pixel location. For example, assuming that correction is performed using the upper reference pixel in 2301 of Figure 23, the distance between pixels c4 and r1 is greater than the distance between pixels c0 and r1. In this case, the value of the weight coefficient applied to c4 may be greater than the value of the weight coefficient applied to c0.

[0507] Additionally, the second prediction block filtering can be performed using the top and left reference pixels, or only the top or left reference pixels, depending on the availability of reference pixels. For example, if the left side of the current coding block is adjacent to the image boundary and a reconstructed block within the same second block exists above it, there may be no reference pixels on the left side of the current coding block, but there may be reference pixels on the top side. In this case, the second prediction block filtering can be performed using only the top reference pixels.

[0508]

[0509] The following describes another method for filtering the second prediction block.

[0510] If a directional prediction mode is selected as the prediction mode within the current coding block, filtering can be performed using reference pixels in the opposite direction. For example, if the prediction mode information within the current screen is in the lower left or upper right direction, reference pixels may also exist in the opposite direction to the prediction mode direction. After generating a prediction block using the selected directional prediction mode information, a second prediction block can be generated using the prediction mode information in the opposite direction of the selected directional prediction mode. A final prediction block can be generated by weighting the first and second prediction blocks. To generate the final prediction block, an operation of multiplying a weight coefficient for each prediction pixel position can be performed. At this time, the weight coefficient value can vary depending on the position of the current prediction pixel. For example, if the position of the current coefficient is close to a reference pixel existing in the original prediction mode direction, the weight coefficient value of the prediction pixel of the first prediction block can be increased. Conversely, if the position of the current prediction pixel is close to a reference pixel in the opposite direction, the weight coefficient value of the prediction pixel of the second prediction block can be increased.

[0511] For convenience of explanation, it is stated that the first prediction block filtering is performed first, and then the second prediction block filtering is performed. However, the order may be reversed. The second prediction block filtering is performed on the prediction block generated based on the prediction mode information within the current screen, and then the first prediction block filtering is performed to generate the final prediction block.

[0512] 1901 of FIG. 19 is a case where reference pixel determination (S1901), reference pixel filtering (S1902), prediction block generation (S1903), and prediction block filtering (S1904) are all performed, and 1901 to 1903 may be cases where some processes are omitted. The prediction block generation process may be determined by a method preset in the encoding device and the decoding device. Alternatively, information may be transmitted from the encoding device to the decoding device to select and perform among S1901 to 1904.

[0513]

[0514] Figures 24 and 25 are flowcharts showing the process of encoding / decoding prediction mode information and reference pixel line information in on-screen prediction.

[0515] A prediction mode can be encoded for the current coding block (S2401). If inter-screen prediction is used as the prediction mode of the current coding block (S2402), existing inter-screen prediction information is encoded (S2403). If intra-screen prediction is used as the prediction mode of the current coding block (S2402), it can be checked whether the current coding block is adjacent to the boundary of a second block (S2406). If the left or top of the current coding block is adjacent to the boundary of the second block, reference pixel line information may not be encoded, and only the reference pixel line adjacent to the coding block may be used. If the current coding block is not adjacent to the boundary of the second block, reference pixel information may be encoded (S2404). Thereafter, the intra-screen prediction information may be encoded to indicate which one of the directional modes and the non-directional modes is selected as the prediction mode of the current coding block. Then, the process may be terminated.

[0516] The prediction mode for the current coding block can be decoded (S2501). If inter-screen prediction is used as the prediction mode of the current coding block (S2502), existing inter-screen prediction information is encoded (S2503). If intra-screen prediction is used as the prediction mode of the current coding block (S2502), it can be confirmed whether the current coding block is adjacent to the boundary of a second block (S2506). If the left or top of the current coding block is adjacent to the boundary of the second block, reference pixel line information may not be decoded, and only the reference pixel line adjacent to the coding block may be used. If the current coding block is not adjacent to the boundary of the second block, reference pixel information may be decoded (S2504). Thereafter, the intra-screen prediction information may be decoded (S2505) to confirm which one of the directional modes and the non-directional modes is selected as the prediction mode of the current coding block. Then, the process may be terminated.

[0517] The following describes the process of encoding / decoding prediction information, considering the case where the upper part of the current coding block is adjacent to the boundary of the second block.

[0518]

[0519] Figures 26 and 27 are flowcharts showing the process of encoding / decoding prediction mode information and reference pixel line information in on-screen prediction.

[0520] A prediction mode can be encoded for the current coding block (S2601). If inter-screen prediction is used as the prediction mode of the current coding block (S2602), existing inter-screen prediction information is encoded (S2603), and if intra-screen prediction is used as the prediction mode of the current coding block (S2602), reference pixel information is encoded (S2604) to inform the reference pixel line used in the current coding block. It can be confirmed whether the top of the current coding block is adjacent to the boundary of the second block (S2606). If the top of the current coding block is adjacent to the boundary of the second block, the top reference pixel line uses only the reference pixel line adjacent to the coding block, and the left reference pixel line can use the transmitted reference pixel line (S2607). If the current coding block is not adjacent to the boundary of the second block, the top reference pixel line and the left reference pixel line can use the transmitted reference pixel line as is (S2608). Afterwards, the prediction information within the screen can be encoded (S2605) to indicate which one of the directional modes and the non-directional mode has been selected as the prediction mode of the current coding block. Then, the process can be terminated.

[0521] The prediction mode for the current coding block can be decoded (S2701). If inter-screen prediction is used as the prediction mode of the current coding block (S2702), existing inter-screen prediction information is decoded (S2703). If intra-screen prediction is used as the prediction mode of the current coding block (S2702), reference pixel information is decoded (S2704) to confirm the reference pixel line used in the current coding block. It can be confirmed whether the top of the current coding block is adjacent to the boundary of the second block (S2706). If the top of the current coding block is adjacent to the boundary of the second block, the top reference pixel line uses only the reference pixel line adjacent to the coding block, and the left reference pixel line can use the transmitted reference pixel line (S2707). If the current coding block is not adjacent to the boundary of the second block, the top reference pixel line and the left reference pixel line can use the transmitted reference pixel line as is (S2708). Afterwards, the prediction information within the screen can be decoded (S2705) to determine which one of the directional modes and the non-directional mode is selected as the prediction mode of the current coding block. Then, the process can be terminated.

[0522] The following describes the process of encoding / decoding reference pixel line information when the upper part of the current coding block is adjacent to the boundary of the second block and considering prediction information within the screen.

[0523]

[0524] Figures 28 and 29 are flowcharts showing the process of encoding / decoding prediction mode information and reference pixel line information in on-screen prediction.

[0525] A prediction mode for the current coding block can be encoded (S2801). If inter-screen prediction is used as the prediction mode of the current coding block (S2802), existing inter-screen prediction information is encoded (S2803), and if intra-screen prediction is used as the prediction mode of the current coding block (S2802), the intra-screen prediction information is encoded (S2804) so ​​that one mode determined among directional prediction modes and non-directional prediction modes can be notified as the prediction mode of the current coding block. If the transmitted prediction mode is a non-directional prediction mode (S2805), the process can be terminated, and if it is a directional prediction mode (S2805), the process can be moved to the next step. Reference pixel information can be encoded (S2806) so as to indicate a reference pixel line used in the current coding block. It can be confirmed whether the top of the current coding block is adjacent to the boundary of the second block (S2807). If the top of the current coding block is adjacent to the boundary of the second block, the top reference pixel line may only use the reference pixel lines adjacent to the coding block, and the left reference pixel line may use the transmitted reference pixel line (S2808). If the current coding block is not adjacent to the boundary of the second block, the top reference pixel line and the left reference pixel line may use the transmitted reference pixel line as is (S2809). Then, the process may be terminated.

[0526] The prediction mode for the current coding block can be decoded (S2901). If inter-screen prediction is used as the prediction mode of the current coding block (S2902), existing inter-screen prediction information is decoded (S2903). If intra-screen prediction is used as the prediction mode of the current coding block (S2902), the intra-screen prediction information is decoded (S2904) to confirm one mode determined among directional prediction modes and non-directional prediction modes as the prediction mode of the current coding block. If the transmitted prediction mode is a non-directional prediction mode (S2905), the process is terminated, and if it is a directional prediction mode (S2905), the process can be moved to the next step. Reference pixel information can be decoded (S2906) to confirm the reference pixel line used in the current coding block. It can be confirmed whether the top of the current coding block is adjacent to the boundary of the second block (S2907). If the top of the current coding block is adjacent to the boundary of the second block, the top reference pixel line may only use the reference pixel lines adjacent to the coding block, and the left reference pixel line may use the transmitted reference pixel line (S2908). If the current coding block is not adjacent to the boundary of the second block, the top reference pixel line and the left reference pixel line may use the transmitted reference pixel line as is (S2909). Then, the process may be terminated.

[0527]

[0528] While the exemplary methods of this disclosure are presented as a series of operations for clarity of description, this is not intended to limit the order in which the steps are performed, and individual steps may be performed simultaneously or in different orders, if desired. To implement a method according to this disclosure, additional steps may be included in addition to the steps illustrated, some steps may be excluded and the remaining steps included, or some steps may be excluded and additional steps included.

[0529] The various embodiments of the present disclosure are not intended to list all possible combinations but rather to illustrate representative aspects of the present disclosure, and the matters described in the various embodiments may be applied independently or in combinations of two or more.

[0530] Additionally, various embodiments of the present disclosure may be implemented by hardware, firmware, software, or a combination thereof. In the case of hardware implementation, the embodiments may be implemented by one or more ASICs (Application Specific Integrated Circuits), DSPs (Digital Signal Processors), DSPDs (Digital Signal Processing Devices), PLDs (Programmable Logic Devices), FPGAs (Field Programmable Gate Arrays), general processors, controllers, microcontrollers, microprocessors, etc.

[0531] The scope of the present disclosure may include software or machine-executable instructions (e.g., operating systems, applications, firmware, programs, etc.) that cause operations according to the methods of various embodiments to be executed on a device or a computer, and a non-transitory computer-readable medium having such software or instructions stored thereon and executable on the device or computer.

[0532] The present disclosure relates to an improved intra-screen prediction method that improves the encoding and decoding efficiency of existing intra-screen prediction methods, and the technology of the present disclosure can be utilized in industrial fields related to video compression, codecs, and real-time streaming technologies that utilize intra-screen prediction methods.

Claims

1. A step of determining a reference pixel of the current coding block based on reference pixel information of the current coding block; A step of generating a prediction block of a current coding block based on the above reference pixel and the prediction mode of the current coding block, An image decoding method, characterized in that the determination of the above reference pixel is determined based on all or part of the pixels included in the reference pixel line of the current coding block.

2. In paragraph 1, An image decoding method, characterized in that the number of reference pixel lines of the current coding block is plural.

3. In paragraph 1, An image decoding method, characterized in that the reference pixel line of the current coding block includes at least one of a reference pixel line adjacent to the current coding block or a reference pixel line not adjacent to the current coding block.

4. In paragraph 1, The step of determining the above reference pixel is: A step of determining a reference area of ​​the current coding block; a step of determining a reference pixel line within the above reference area; and An image decoding method, characterized in that it comprises a step of determining all or part of the pixels of the reference pixel line as reference pixels of the current coding block.

5. In paragraph 1, An image decoding method, characterized in that the generation of the above prediction block is performed using a corrected reference pixel obtained by correcting the pixel value by performing reference pixel filtering on the reference pixel.

6. In paragraph 5, An image decoding method, characterized in that the above reference pixel filtering is performed using surrounding reference pixels of the target reference pixel.

7. In paragraph 6, An image decoding method, characterized in that the number of the above surrounding reference pixels is 8.

8. In paragraph 1, An image decoding method, characterized in that the above reference pixel line includes at least one of a vertical reference pixel line, a horizontal reference pixel line, and a diagonal reference pixel line.

9. In paragraph 8, An image decoding method, characterized in that the horizontal reference pixel line is composed of pixels between the pixel pointed to by the upper left pixel of the current coding block and the pixel pointed to by the upper right pixel.

10. In paragraph 8, An image decoding method, characterized in that the vertical reference pixel line is composed of pixels between the pixel pointed to by the upper left pixel of the current coding block and the pixel pointed to by the lower left pixel.

11. In paragraph 8, An image decoding method, characterized in that when both the horizontal reference pixel line and the vertical reference pixel line are used, the lengths of the horizontal reference pixel line and the vertical reference pixel line are different.

12. In paragraph 11, The reference pixels used in the above horizontal reference pixel line are some pixels included in the above horizontal reference pixel line, An image decoding method, characterized in that the reference pixels used in the vertical reference pixel line are all pixels included in the vertical reference pixel line.

13. In paragraph 1, An image decoding method, characterized in that the reference pixel of the upper left pixel of the current coding block includes a first pixel of a first reference pixel line and a second pixel of a second reference pixel line.

14. A step of determining a reference pixel of the current coding block; A step of generating a prediction block of a current coding block based on the above reference pixel and the prediction mode of the current coding block, The determination of the above reference pixel is determined based on all or part of the pixels included in the reference pixel line of the current coding block, A video encoding method, characterized in that reference pixel information of the current coding block is determined based on the reference pixel.

15. In a non-transitory computer-readable recording medium storing a bitstream generated by a video encoding method, The above image encoding method is, A step of determining a reference pixel of the current coding block; A step of generating a prediction block of a current coding block based on the above reference pixel and the prediction mode of the current coding block, The determination of the above reference pixel is determined based on all or part of the pixels included in the reference pixel line of the current coding block, A non-transitory computer-readable recording medium, characterized in that reference pixel information of the current coding block is determined based on the reference pixel.

Images