Image encoding / decoding method and apparatus, and recording medium storing bit stream

Abstract

Provided are an image encoding / decoding method, an image encoding / decoding apparatus, a recording medium storing a bit stream, and a transmission method. The image decoding method may comprise the steps of: generating a first reference sample based on a first reference region adjacent to a current block; generating a second reference sample based on a second reference region adjacent to the current block; and performing intra prediction on the current block based on the first reference sample and the second reference sample, where the first reference region may be a reconstruction region located at the left, upper, and upper left of the current block, and the second reference region may be a prediction region located at the right, lower, and lower right of the current block.

Description

Technical Field

[0001] The present invention relates to an image encoding / decoding method and apparatus and a recording medium for storing a bit stream. More particularly, the present invention relates to an image encoding / decoding method and apparatus using bidirectional intra prediction and a recording medium for storing a bit stream. Background Art

[0002] Recently, in various application fields, the demand for high-resolution, high-quality images (such as ultra-high-definition (UHD) images) has been increasing. As image data becomes higher in resolution and quality, the amount of data increases relatively compared to existing image data. Therefore, when image data is transmitted using a medium such as existing wired and wireless broadband lines or stored using existing storage media, transmission and storage costs increase. In order to solve these problems that arise as image data becomes higher in resolution and quality, high-efficiency image encoding / decoding technology is needed for images with higher resolution and quality.

[0003] The existing intra prediction method is unidirectional intra prediction, which generates a prediction block by copying the value of a first reference sample determined according to the directionality of the intra prediction mode. Here, the first reference sample can be determined from the reconstructed left reference sample, the upper reference sample, or the upper left reference sample around the current block. Such unidirectional intra prediction has the following problem: the prediction accuracy decreases as the sample moves away from the first reference sample (such as a sample present in the lower right area of ​​the current block). Summary of the invention

[0004] Technical issues

[0005] An object of the present invention is to provide an image encoding / decoding method and apparatus with improved encoding / decoding efficiency.

[0006] Another object of the present invention is to provide a recording medium storing a bit stream generated by the image decoding method or apparatus according to the present invention.

[0007] Another object of the present invention is to provide a bidirectional intra prediction method and a second reference sample generation method for solving the problem of unidirectional intra prediction.

[0008] Technical Solution

[0009] The image decoding method according to an embodiment of the present invention may include generating a chroma mode list of a current chroma block, deriving a chroma intra prediction mode of the current chroma block based on the chroma mode list, and generating a prediction block of the current chroma block based on the chroma intra prediction mode. The chroma mode list may include at least one of a default mode, a derived chroma mode, or a direct mode.

[0010] The image decoding method according to an embodiment of the present invention may include: generating a first reference sample based on a first reference area adjacent to a current block; generating a second reference sample based on a second reference area adjacent to the current block; and performing intra-frame prediction based on the current block based on the first reference sample and the second reference sample. The first reference area may be a reconstruction area located to the left, above, and above-left of the current block, and the second reference area may be a prediction area located to the right, below, and below-right of the current block.

[0011] In the image decoding method, the second reference region may be predicted based on neighboring samples of a matching block searched by performing template matching.

[0012] In the image decoding method, the template matching may include searching for a reference template most similar to a current template around a current block in a reconstructed search area of ​​a current picture and determining a matching block based on the searched reference template.

[0013] In the image decoding method, the second reference region may be predicted based on neighboring samples of a matching block in a current picture indicated by motion information of the current block.

[0014] In the image decoding method, the motion information may be a block vector.

[0015] In the image decoding method, the second reference region may be predicted based on a neural network model that receives as input the reconstructed samples of the first reference region.

[0016] In the image decoding method, the first reference sample and the second reference sample may be located in opposite directions with respect to the current block.

[0017] In the image decoding method, the step of performing the intra prediction may include performing the intra prediction by applying a first weight and a second weight to the first reference sample and the second reference sample, respectively.

[0018] In the image decoding method, the first weight and the second weight may be determined based on a ratio of a distance between a current prediction target sample and a first reference sample and a distance between the current prediction target sample and a second reference sample.

[0019] In the image decoding method, when the intra prediction mode of the current block is a non-directional mode, intra prediction may be performed using only a first reference sample.

[0020] The image encoding method according to an embodiment of the present invention may include: generating a first reference sample based on a first reference area adjacent to a current block; generating a second reference sample based on a second reference area adjacent to the current block; performing intra-frame prediction based on the current block based on the first reference sample and the second reference sample. The first reference area may be a reconstruction area located to the left, above, and above-left of the current block, and the second reference area may be a prediction area located to the right, below, and below-right of the current block.

[0021] A non-transitory computer-readable recording medium according to an embodiment of the present invention may store a bitstream generated by an image encoding method, wherein the image encoding method includes: generating a first reference sample based on a first reference area adjacent to a current block; generating a second reference sample based on a second reference area adjacent to the current block; performing intra-frame prediction based on the current block based on the first reference sample and the second reference sample. The first reference area may be a reconstruction area located to the left, above, and upper left of the current block, and the second reference area may be a prediction area located to the right, below, and lower right of the current block.

[0022] A method for transmitting a bitstream generated by an image encoding method according to an embodiment of the present invention may include transmitting the bitstream. The image encoding method may include: generating a first reference sample based on a first reference area adjacent to a current block; generating a second reference sample based on a second reference area adjacent to the current block; and performing intra-frame prediction based on the current block based on the first reference sample and the second reference sample. The first reference area may be a reconstruction area located to the left, above, and upper left of the current block, and the second reference area may be a prediction area located to the right, below, and lower right of the current block.

[0023] The features briefly summarized above with respect to the present disclosure are merely exemplary aspects of the following detailed description of the present disclosure and do not limit the scope of the present disclosure.

[0024] Beneficial Effects

[0025] According to the present invention, it is possible to provide an image encoding / decoding method and apparatus with improved encoding / decoding efficiency.

[0026] In addition, according to the present invention, a bidirectional intra prediction method and a second reference sample generating method can be provided.

[0027] In addition, according to the present invention, the encoding efficiency in intra-frame prediction can be improved.

[0028] It should be understood by those skilled in the art that the effects that can be achieved by the present disclosure are not limited to the effects specifically described above, and other advantages of the present disclosure will be more clearly understood from the detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1is a block diagram showing the configuration of an encoding device according to an embodiment of the present invention.

[0030] Figure 2 is a block diagram showing the configuration of a decoding device according to an embodiment of the present invention.

[0031] Figure 3 is a diagram schematically showing a video encoding system to which the present invention is applicable.

[0032] Figure 4 is a flowchart of a bidirectional intra prediction method according to an embodiment of the present invention.

[0033] Figure 5 is a diagram for explaining a bidirectional intra prediction method according to an embodiment of the present invention.

[0034] Figure 6 is a flowchart of a method for generating a second reference sample based on template matching according to an embodiment of the present invention.

[0035] Figure 7 is a diagram for explaining an intra prediction method based on template matching according to an embodiment of the present invention.

[0036] Figure 8 is a diagram for explaining a method of generating a second reference sample based on template matching according to an embodiment of the present invention.

[0037] Fig. 9 is a flowchart of a method for generating a second reference sample based on motion information according to an embodiment of the present invention.

[0038] Fig.10 is a diagram for explaining a method of generating a second reference sample based on motion information according to an embodiment of the present invention.

[0039] Fig.11 is a diagram for explaining a method of generating a second reference sample based on a neural network according to an embodiment of the present invention.

[0040] Fig.12 is a flowchart illustrating an image decoding method according to an embodiment of the present invention.

[0041] Fig.13 A content streaming system applicable according to an embodiment of the present invention is exemplarily shown. DETAILED DESCRIPTION

[0042] The present invention may have various modifications and embodiments, and specific embodiments are shown in the drawings and described in detail in the detailed description. However, this is not intended to limit the present invention to specific embodiments, but should be understood to include all modifications, equivalents or substitutes included in the spirit and scope of the present invention. In various aspects, similar reference numerals in the drawings indicate the same or similar functions. The shapes and sizes of the elements in the drawings can be provided by way of example for a clearer description. The detailed description of the exemplary embodiments described below refers to the drawings, which show specific embodiments by way of example. These embodiments are described in sufficient detail to enable those skilled in the art to practice these embodiments. It should be understood that various embodiments are different from each other, but not necessarily mutually exclusive. For example, the specific shapes, structures and characteristics described herein can be implemented in other embodiments without departing from the spirit and scope of the present invention with respect to an embodiment. It should also be understood that the position or arrangement of each component in each disclosed embodiment can be changed without departing from the spirit and scope of the embodiment. Therefore, the specific embodiments set forth below are not intended to be restrictive, and the scope of the exemplary embodiments is limited only by the full range of equivalents to which the attached claims and these claims are entitled (if properly described).

[0043] In the present invention, the terms first, second, etc. may be used to describe various components, but these components should not be limited by these terms. These terms are only used for the purpose of distinguishing one component from another component. For example, without departing from the scope of the present invention, a first component may be referred to as a second component, and similarly, a second component may also be referred to as a first component. The terms and / or include a combination of multiple related descriptive items or any item among multiple related descriptive items.

[0044] Each component in the embodiment of the present invention is described independently to indicate different characteristic functions, which does not mean that each component is formed as a separate hardware or software configuration unit. That is, for ease of explanation, each component is listed and included as a separate component, and at least two components can be combined to form a single component, or a component can be divided into multiple components to perform functions, and an embodiment in which the components are integrated and an embodiment in which each component is divided are also included in the scope of the present invention, as long as they do not deviate from the essence of the present invention.

[0045] The terms used in the present invention are only used to describe specific embodiments and are not intended to limit the present invention. Unless the context clearly indicates otherwise, singular expressions include plural expressions. In addition, some components of the present invention are not necessary components for performing the basic functions of the present invention, and may be optional components that are only used to improve performance. The present invention can be implemented by only including necessary components for realizing the essence of the present invention and not including components that are only used to improve performance, and the structure that only includes necessary components except optional components that are only used to improve performance is also included in the scope of the present invention.

[0046] In one embodiment, the term "at least one" may refer to one of a number greater than or equal to 1, such as 1, 2, 3, and 4. In one embodiment, the term "plurality" may refer to one of a number greater than or equal to 2, such as 2, 3, and 4.

[0047] Hereinafter, the embodiments of the present invention will be described in detail with reference to the accompanying drawings. In describing the embodiments of the present specification, if it is determined that a detailed description of a related known configuration or function may obscure the subject of the present specification, the detailed description will be omitted, and the same reference numerals will be used for the same components in the accompanying drawings, and repeated description of the same components will be omitted.

[0048] Terminology Description

[0049] Hereinafter, "image" may refer to one picture constituting a video, and may also refer to the video itself. For example, "encoding and / or decoding of an image" may mean "encoding and / or decoding of a video", and may also mean "encoding and / or decoding of one of the images constituting a video".

[0050] Hereinafter, "motion picture" and "video" may be used with the same meaning and may be used interchangeably. In addition, the target image may be an encoding target image as an encoding target and / or a decoding target image as a decoding target. In addition, the target image may be an input image input to an encoding device, and may be an input image input to a decoding device. Here, the target image may have the same meaning as the current image.

[0051] Hereinafter, an encoder and an image encoding device may be used with the same meaning and may be used interchangeably.

[0052] Hereinafter, a decoder and an image decoding device may be used with the same meaning and may be used interchangeably.

[0053] Hereinafter, “image”, “picture”, “frame” and “screen” may be used with the same meaning and may be used interchangeably.

[0054] Hereinafter, a "target block" may be an encoding target block as an encoding target and / or a decoding target block as a decoding target. In addition, a target block may be a current block as a target of current encoding and / or decoding. For example, "target block" and "current block" may be used with the same meaning and may be used interchangeably.

[0055] Hereinafter, "block" and "unit" may be used with the same meaning and may be used interchangeably. In addition, "unit" may mean a block including a luma component block and a chroma component block corresponding thereto so as to distinguish it from a block. For example, a coding tree unit (CTU) may consist of one luma component (Y) coding tree block (CTB) and two chroma component (Cb, Cr) coding tree blocks associated therewith.

[0056] Hereinafter, "sample", "picture element" and "pixel" may be used in the same meaning and may be used interchangeably. Herein, a sample may refer to a basic unit constituting a block.

[0057] Hereinafter, “inter” and “inter-screen” may be used with the same meaning and may be used interchangeably.

[0058] Hereinafter, “within” and “within the screen” may be used with the same meaning and may be used interchangeably.

[0059] Figure 1 is a block diagram showing the configuration of an encoding device according to an embodiment of the present invention.

[0060] The encoding device 100 may be an encoder, a video encoding device, or an image encoding device. A video may include one or more images. The encoding device 100 may sequentially encode one or more images.

[0061] refer to Figure 1 , the encoding device 100 may include an image segmentation unit 110, an intra-frame prediction unit 120, a motion prediction unit 121, a motion compensation unit 122, a switch 115, a subtractor 125, a transform unit 130, a quantization unit 140, an entropy encoding unit 150, a dequantization unit 160, an inverse transform unit 170, an adder 117, a filter unit 180 and a reference picture buffer 190.

[0062] In addition, the encoding apparatus 100 may generate a bitstream including information encoded by encoding the input image and output the generated bitstream. The generated bitstream may be stored in a computer-readable recording medium or may be streamed through a wired / wireless transmission medium.

[0063] The image segmentation unit 110 can segment the input image into various forms to improve the efficiency of video encoding / decoding. That is, the input video consists of multiple pictures, and one picture can be hierarchically segmented and processed for compression efficiency, parallel processing, etc. For example, one picture can be segmented into one or more tiles or slices, and then segmented into multiple CTUs (coding tree units) again. Alternatively, one picture can be first segmented into multiple sub-pictures defined as groups of rectangular slices, and each sub-picture can be segmented into tiles / slices. Here, sub-images can be used to support the functions of partially independent encoding / decoding and transmitting images. Since multiple sub-pictures can be reconstructed separately, it has the advantage of being easy to edit in applications that configure multi-channel inputs into one picture. In addition, tiles can be hierarchically divided to generate bricks. Here, bricks can be used as basic units for parallel processing within a picture. In addition, one CTU can be recursively segmented into a quadtree (QT), and the segmented terminal node can be defined as a CU (coding unit). CU can be segmented into PU (prediction unit) as a prediction unit and TU (transform unit) as a transform unit to perform prediction and segmentation. At the same time, the CU can be used as a prediction unit and / or a transform unit itself. Here, for flexible partitioning, each CTU can be recursively partitioned into a multi-type tree (MTT) and a quadtree (QT). The partitioning of the CTU into multi-type trees can start from the end node of the QT, and the MTT can be composed of a binary tree (BT) and a ternary tree (TT). For example, the MTT structure can be classified into a vertical binary split mode (SPLIT_BT_VER), a horizontal binary split mode (SPLIT_BT_HOR), a vertical ternary split mode (SPLIT_TT_VER), and a horizontal ternary split mode (SPLIT_TT_HOR). In addition, the minimum block size (MinQTSize) of the quadtree of the luminance block during partitioning can be set to 16×16, the maximum block size (MaxBtSize) of the binary tree can be set to 128×128, and the maximum block size (MaxTtSize) of the ternary tree can be set to 64×64. In addition, the minimum block size (MinBtSize) of the binary tree and the minimum block size (MinTtSize) of the ternary tree can be specified as 4×4, and the maximum depth (MaxMttDepth) of the multi-type tree can be specified as 4. In addition, in order to improve the coding efficiency of the I slice, a dual tree of the CTU partition structure that uses the luminance component and the chrominance component differently can be applied. On the other hand, in P and B slices, the luminance and chrominance CTBs (coding tree blocks) within the CTU can be partitioned into a single tree that shares the coding tree structure.

[0064] The encoding device 100 may perform encoding on the input image in intra mode and / or inter mode. Optionally, the encoding device 100 may perform encoding on the input image in a third mode other than intra mode and inter mode (e.g., IBC mode, palette mode, etc.). However, if the third mode has functional characteristics similar to those of the intra mode or the inter mode, the third mode may be classified as the intra mode or the inter mode for ease of explanation. In the present invention, the third mode is separately classified and described only when a specific description of the third mode is required.

[0065] When the intra mode is used as the prediction mode, the switch 115 may be switched to intra, and when the inter mode is used as the prediction mode, the switch 115 may be switched to inter. Here, the intra mode may represent the intra prediction mode, and the inter mode may represent the inter prediction mode. The encoding device 100 may generate a prediction block for an input block of an input image. In addition, the encoding device 100 may encode the residual block using the residual of the input block and the prediction block after generating the prediction block. The input image may be referred to as the current image of the current encoding target. The input block may be referred to as the current block as the current encoding target or the encoding target block.

[0066] When the prediction mode is intra mode, the intra prediction unit 120 may use samples of blocks that have been encoded / decoded around the current block as reference samples. The intra prediction unit 120 may perform spatial prediction on the current block using the reference samples, or generate prediction samples of the input block by spatial prediction. Here, intra prediction may refer to intra prediction.

[0067] As the intra prediction method, a non-directional prediction mode and a directional prediction mode (eg, 65 directions) such as a DC mode and a planar mode may be applied. Here, the intra prediction method may be denoted as an intra prediction mode or an intra prediction mode.

[0068] When the prediction mode is the inter mode, the motion prediction unit 111 may retrieve the area that best matches the input block from the reference image in the motion prediction process, and derive the motion vector by using the retrieved area. In this case, the search area may be used as the area. The reference image may be stored in the reference picture buffer 190. Here, when encoding / decoding for the reference image is performed, it may be stored in the reference picture buffer 190.

[0069] The motion compensation unit 112 may generate a prediction block of the current block by performing motion compensation using a motion vector. Here, inter prediction may refer to inter prediction or motion compensation.

[0070] When the value of the motion vector is not an integer, the motion prediction unit 111 and the motion compensation unit 112 may generate a prediction block by applying an interpolation filter to a partial area of ​​the reference picture. In order to perform inter-frame prediction or motion compensation, it may be determined whether the motion prediction and motion compensation mode of the prediction unit included in the coding unit is one of the skip mode, the merge mode, the advanced motion vector prediction (AMVP) mode, and the coding unit-based intra-block copy (IBC) mode, and the inter-frame prediction or motion compensation may be performed according to each mode.

[0071] In addition, based on the above-mentioned inter-frame prediction method, the AFFINE mode based on sub-PU prediction, the SbTMVP (sub-block-based temporal motion vector prediction) mode, the MMVD (merged with MVD) mode based on PU prediction, and the GPM (geometric partitioning mode) mode can be applied. In addition, in order to improve the performance of each mode, HMVP (history-based MVP), PAMVP (paired average MVP), CIIP (combined intra / inter prediction), AMVR (adaptive motion vector resolution), BDOF (bidirectional optical flow), BCW (bidirectional prediction with CU weights), LIC (local illumination compensation), TM (template matching), OBMC (overlapping block motion compensation), etc. can be applied.

[0072] Among them, the AFFINE mode is a technology used in both AMVP and MERGE modes, and also has high coding efficiency. In the existing video coding standard, because MC (motion compensation) is performed by considering only the parallel movement of the block, its disadvantage is that it cannot properly compensate for the actual motion, such as zooming in / out and rotation. To supplement this, a four-parameter affine motion model of two control point motion vectors (CPMV) and a six-parameter affine motion model using three control point motion vectors can be used and applied to inter-frame prediction. Here, CPMV is a vector representing an affine motion model of one of the upper left, upper right, and lower left of the current block. The AFFINE mode is divided into AMVP or MERGE mode for CPMV encoding. At the same time, considering the computational complexity of video coding, affine motion compensation can be performed in units of 4×4 blocks without performing pixel-by-pixel affine motion compensation. That is, when viewed in units of 4×4 blocks, it is the same as the existing motion compensation, but from the perspective of the entire PU, it can be regarded as affine motion compensation.

[0073] The subtractor 113 may generate a residual block by using the difference between the input block and the prediction block. The residual block may be referred to as a residual signal. The residual signal may represent the difference between the original signal and the prediction signal. Alternatively, the residual signal may be a signal generated by transforming or quantizing, or transforming and quantizing the difference between the original signal and the prediction signal. The residual block may be a residual signal of a block unit.

[0074] The transform unit 130 may generate a transform coefficient by performing a transform on the residual block, and output the generated transform coefficient. Here, the transform coefficient may be a coefficient value generated by performing a transform on the residual block. When the transform skip mode is applied, the transform unit 130 may skip the transform of the residual block.

[0075] The quantization level may be generated by applying quantization to a transform coefficient or a residual signal. Hereinafter, the quantization level may also be referred to as a transform coefficient in an embodiment.

[0076] For example, a 4×4 luminance residual block generated by intra prediction is transformed using a basic vector based on DST (discrete sine transform), and the remaining residual block can be transformed using a basic vector based on DCT (discrete cosine transform). In addition, the transform block is divided into a quadtree shape for one block using RQT (Residual Quad Tree) technology, and after transforming and quantizing each transform block divided by RQT, a coded block flag (cbf) can be transmitted when all coefficients become 0 to increase coding efficiency.

[0077] As another alternative, a multiple transform selection (MTS) technique that selectively uses multiple transform bases to perform transforms may be applied. That is, instead of partitioning the CU into TUs by RQT, a function similar to TU partitioning may be performed by a sub-block transform (SBT) technique. Specifically, SBT is only applied to inter-prediction blocks, and unlike RQT, the current block can be partitioned into 1 / 2 or 1 / 4 sizes in the vertical or horizontal direction, and then the transform may be performed on only one of the blocks. For example, if it is partitioned vertically, the transform may be performed on the leftmost or rightmost block, and if it is partitioned horizontally, the transform may be performed on the topmost or bottommost block.

[0078] In addition, LFNST (Low Frequency Non-Separable Transform) can be applied. LFNST is a secondary transform technique that additionally transforms the residual signal transformed to the frequency domain by DCT or DST. LFNST additionally transforms the 4×4 or 8×8 low-frequency region in the upper left, so that the residual coefficients can be concentrated in the upper left.

[0079] The quantization unit 140 may generate a quantization level by quantizing the transform coefficient or the residual signal according to a quantization parameter (QP) and output the generated quantization level. Here, the quantization unit 140 may quantize the transform coefficient by using a quantization matrix.

[0080] For example, a quantizer may be used that uses a QP value of 0 to 51. Alternatively, if the image size is large and high encoding efficiency is required, a QP of 0 to 63 may be used. In addition, a DQ (Dependent Quantization) method using two quantizers instead of one quantizer may be applied. DQ performs quantization using two quantizers (e.g., Q0 and Q1), but even without using signaling information about a specific quantizer, the quantizer to be used for the next transform coefficient may be selected based on the current state through a state transition model.

[0081] The entropy encoding unit 150 may generate a bitstream by performing entropy encoding on the value calculated by the quantization unit 140 or on the encoding parameter value calculated when performing encoding according to probability distribution, and output the bitstream. The entropy encoding unit 150 may perform entropy encoding of information about samples of an image and information for decoding the image. For example, the information for decoding the image may include a syntax element.

[0082] When entropy coding is applied, symbols are represented so that a smaller number of bits are allocated to symbols with a high probability of occurrence and a larger number of bits are allocated to symbols with a low probability of occurrence, and therefore, the size of the bitstream for the symbol to be encoded can be reduced. The entropy coding unit 150 may use a coding method such as exponential Golomb, context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), etc. for entropy coding. For example, the entropy coding unit 150 may perform entropy coding by using a variable length coding / code (VLC) table. In addition, the entropy coding unit 150 may derive a binarization method of a target symbol and a probability model of a target symbol / bin, and perform arithmetic coding by using the derived binarization method and context model.

[0083] In this regard, when CABAC is applied, in order to reduce the size of the probability table stored in the decoding device, the table probability update method can be changed to a table update method using a simple equation and applied. In addition, two different probability models can be used to obtain more accurate symbol probability values.

[0084] In order to encode the transformation coefficient level (quantization level), the entropy encoding unit 150 may change the two-dimensional block form coefficient into a one-dimensional vector form through a transformation coefficient scanning method.

[0085] The encoding parameters may include information (flags, indexes, etc.) encoded in the encoding device 100 and notified to the decoding device 200 by signals (such as syntax elements) and information derived in the encoding or decoding process, and may represent information required when encoding or decoding an image.

[0086] Here, signaling a flag or an index may mean that the corresponding flag or index is entropy encoded in an encoder and included in a bitstream, and may mean that the corresponding flag or index is entropy decoded from the bitstream in a decoder.

[0087] The encoded current image may be used as a reference image of another image to be processed later. Therefore, the encoding apparatus 100 may reconstruct or decode the encoded current image again and store the reconstructed or decoded image in the reference picture buffer 190 as a reference image.

[0088] The quantization level may be dequantized in the dequantization unit 160, or may be inversely transformed in the inverse transform unit 170. The dequantized and / or inversely transformed coefficient may be added to the prediction block by the adder 117. Here, the dequantized and / or inversely transformed coefficient may mean a coefficient on which at least one of dequantization and inverse transformation is performed, and may mean a reconstructed residual block. The dequantization unit 160 and the inverse transform unit 170 may be performed as inverse processes of the quantization unit 140 and the transform unit 130.

[0089] The reconstructed block may pass through the filter unit 180. The filter unit 180 may apply a deblocking filter, a sample adaptive offset (SAO), an adaptive in-loop filter (ALF), a bilateral filter (BIF), a luminance mapping with chroma scaling (LMCS), etc. to the reconstructed sample, the reconstructed block, or the reconstructed image using all or some filtering techniques. The filter unit 180 may be referred to as an in-loop filter. In this case, the in-loop filter is also used as a name excluding LMCS.

[0090] The deblocking filter can remove block distortion generated in the boundary between blocks. In order to determine whether to apply the deblocking filter, it can be determined whether to apply the deblocking filter to the current block based on samples included in a number of rows or columns included in the block. When the deblocking filter is applied to the block, different filters can be applied according to the required deblocking filter strength.

[0091] In order to compensate for the encoding error using the sample adaptive offset, an appropriate offset value may be added to the sample value. The sample adaptive offset may correct the offset of the deblocked image from the original image by sample units. A method of dividing the samples included in the image into a predetermined number of regions, determining the regions to which the offset is applied, and applying the offset to the determined regions, or a method of applying the offset in consideration of edge information about each sample may be used.

[0092] The bilateral filter (BIF) can also correct the deviation from the original image on a sample-by-sample basis for the image on which deblocking has been performed.

[0093] The adaptive loop filter may perform filtering based on the comparison result of the reconstructed image and the original image. The samples included in the image may be divided into predetermined groups, the filter to be applied to each group may be determined, and differential filtering may be performed for each group. Information on whether to apply the ALF may be signaled by a coding unit (CU), and the form and coefficient of the adaptive loop filter to be applied to each block may be changed.

[0094] In LMCS (luminance mapping with chrominance scaling), luminance mapping (LM) means remapping luminance values ​​by a piecewise linear model, and chrominance scaling (CS) means a technique for scaling the residual values ​​of chrominance components according to the average luminance value of a prediction signal. Specifically, LMCS can be used as an HDR correction technique that reflects the characteristics of an HDR (High Dynamic Range) image.

[0095] The reconstructed block or the reconstructed image that has passed through the filter unit 180 may be stored in the reference picture buffer 190. The reconstructed block that has passed through the filter unit 180 may be a part of the reference image. That is, the reference image is a reconstructed image composed of the reconstructed block that has passed through the filter unit 180. The stored reference image may be used later in inter-frame prediction or motion compensation.

[0096] Figure 2 is a block diagram showing the configuration of a decoding device according to an embodiment of the present invention.

[0097] The decoding device 200 may be a decoder, a video decoding device, or an image decoding device.

[0098] refer to Figure 2 , the decoding device 200 may include an entropy decoding unit 210, a dequantization unit 220, an inverse transform unit 230, an intra-frame prediction unit 240, a motion compensation unit 250, an adder 201, a switch 203, a filter unit 260 and a reference picture buffer 270.

[0099] The decoding device 200 may receive a bit stream output from the encoding device 100. The decoding device 200 may receive a bit stream stored in a computer-readable recording medium, or may receive a bit stream streamed through a wired / wireless transmission medium. The decoding device 200 may decode the bit stream in an intra-frame mode or an inter-frame mode. In addition, the decoding device 200 may generate a reconstructed image or a decoded image generated by decoding, and output the reconstructed image or the decoded image.

[0100] When the prediction mode for decoding is the intra mode, the switch 203 may be switched to intra. Alternatively, when the prediction mode for decoding is the inter mode, the switch 203 may be switched to inter.

[0101] The decoding device 200 can obtain a reconstructed residual block by decoding the input bit stream and generate a prediction block. When the reconstructed residual block and the prediction block are obtained, the decoding device 200 can generate a reconstructed block that becomes a decoding target by adding the reconstructed residual block and the prediction block. The decoding target block can be referred to as a current block.

[0102] The entropy decoding unit 210 may generate symbols by entropy decoding the bit stream according to the probability distribution. The generated symbols may include symbols in the form of quantization levels. Here, the entropy decoding method may be an inverse process of the above entropy encoding method.

[0103] The entropy decoding unit 210 may change the coefficients of the one-dimensional vector shape into coefficients of the two-dimensional block shape through a transform coefficient scanning method to decode the transform coefficient levels (quantization levels).

[0104] The quantization level may be dequantized in the dequantization unit 220 or inversely transformed in the inverse transform unit 230. The quantization level may be a result of dequantization and / or inverse transformation and may be generated as a reconstructed residual block. Here, the dequantization unit 220 may apply a quantization matrix to the quantization level. The dequantization unit 220 and the inverse transform unit 230 applied to the decoding device may apply the same technology as the dequantization unit 160 and the inverse transform unit 170 applied to the above-mentioned encoding device.

[0105] When the intra mode is used, the intra prediction unit 240 can generate a prediction block by performing spatial prediction on the current block using sample values ​​of blocks decoded around the decoding target block. The intra prediction unit 240 applied to the decoding device can apply the same technology as the intra prediction unit 120 applied to the above encoding device.

[0106] When the inter-frame mode is used, the motion compensation unit 250 may generate a prediction block by performing motion compensation using a motion vector and a reference image stored in the reference picture buffer 270 on the current block. When the value of the motion vector is not an integer value, the motion compensation unit 250 may generate a prediction block by applying an interpolation filter to a partial area within the reference image. In order to perform motion compensation, it may be determined based on the coding unit whether the motion compensation method of the prediction unit included in the corresponding coding unit is a skip mode, a merge mode, an AMVP mode, or a current picture reference mode, and motion compensation may be performed according to each mode. The motion compensation unit 250 applied to the decoding device may apply the same technology as the motion compensation unit 122 applied to the above-mentioned encoding device.

[0107] The adder 201 can generate a reconstructed block by adding the reconstructed residual block and the prediction block. The filter unit 260 can apply at least one of an inverse LMCS, a deblocking filter, a sample adaptive offset, and an adaptive in-loop filter to the reconstructed block or the reconstructed image. The filter unit 260 applied to the decoding device can apply the same filtering technology as the filtering technology of the filter unit 180 applied to the above-mentioned encoding device.

[0108] The filter unit 260 may output a reconstructed image. The reconstructed block or the reconstructed image may be stored in the reference picture buffer 270 and used for inter-frame prediction. The reconstructed block that has passed through the filter unit 260 may be a part of the reference image. That is, the reference image may be a reconstructed image composed of the reconstructed blocks that have passed through the filter unit 260. The stored reference image may be used later in inter-frame prediction or motion compensation.

[0109] Figure 3 is a diagram schematically showing a video encoding system to which the present invention is applicable.

[0110] The video encoding system according to the embodiment may include an encoding device 10 and a decoding device 20. The encoding device 10 may transmit the encoded video and / or image information or data to the decoding device 20 in the form of a file or a stream through a digital storage medium or a network.

[0111] The encoding device 10 according to the embodiment may include a video source generating unit 11, an encoding unit 12, and a transmitting unit 13. The decoding device 20 according to the embodiment may include a receiving unit 21, a decoding unit 22, and a rendering unit 23. The encoding unit 12 may be referred to as a video / image encoding unit, and the decoding unit 22 may be referred to as a video / image decoding unit. The transmitting unit 13 may be included in the encoding unit 12. The receiving unit 21 may be included in the decoding unit 22. The rendering unit 23 may include a display unit, and the display unit may be configured as a separate device or an external component.

[0112] The video source generation unit 11 can obtain the video / image by capturing, synthesizing or generating the video / image process. The video source generation unit 11 may include a video / image capture device and / or a video / image generation device. The video / image capture device may include (for example) one or more cameras, a video / image archive containing previously captured videos / images, etc. The video / image generation device may include, for example, a computer, a tablet computer, a smart phone, etc., and may (electronically) generate the video / image. For example, a virtual video / image may be generated by a computer, etc., in which case the video / image capture process may be replaced by a process of generating relevant data.

[0113] The encoding unit 12 may encode the input video / image. The encoding unit 12 may perform a series of processes for compression and encoding efficiency, such as prediction, transformation, and quantization. The encoding unit 12 may output encoded data (encoded video / image information) in the form of a bit stream. The detailed configuration of the encoding unit 12 may also be the same as described above. Figure 1 The encoding device 100 is configured in the same manner.

[0114] The sending unit 13 may send the encoded video / image information or data output in the form of a bit stream to the receiving unit 21 of the decoding device 20 in the form of a file or stream through a digital storage medium or a network. The digital storage medium may include various storage media, such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The sending unit 13 may include an element for generating a media file in a predetermined file format, and may include an element for sending through a broadcast / communication network. The receiving unit 21 may extract / receive a bit stream from a storage medium or a network and send it to the decoding unit 22.

[0115] The decoding unit 22 can decode the video / image by performing a series of processes corresponding to the operation of the encoding unit 12, such as dequantization, inverse transformation, and prediction. The detailed configuration of the decoding unit 22 can also be the same as that of the above-mentioned Figure 2 The decoding device 200 is configured in the same manner.

[0116] The rendering unit 23 may render the decoded video / image. The rendered video / image may be displayed by the display unit.

[0117] In the following, reference Figures 4 to 12 , a bidirectional intra prediction method and a second reference sample generating method according to an embodiment of the present invention will be described in detail.

[0118] In the present invention, the first reference sample may refer to a sample located at the left, top, or upper left of the reconstruction of the current block as a reference sample used in the unidirectional intra prediction method and the bidirectional intra prediction method. In addition, the second reference sample may refer to a sample located at the right, bottom, or lower right of the current block as a reference sample used in the bidirectional intra prediction method. In addition, an area located at the left, top, or upper left of the current block from which the first reference sample is derived may be referred to as a first reference area, and an area located at the right, bottom, and lower right of the current block from which the second reference sample is derived may be referred to as a second reference area.

[0119] Figure 4 is a flowchart of a bidirectional intra prediction method according to an embodiment of the present invention.

[0120] Reference Figure 4, the encoder / decoder may generate a first reference sample based on the intra prediction mode of the current block (S410). Specifically, the first reference sample may be determined based on the intra prediction mode of the current block in samples located to the left, above, and above-left of the reconstruction of the current block (i.e., the first reference area).

[0121] In addition, the encoder / decoder may generate a second reference sample based on the intra prediction mode of the current block (S420). Specifically, the second reference sample may be determined based on the intra prediction mode of the current block in samples located to the right, below, and right below the current block (i.e., the second reference area). Here, the direction of the intra prediction mode used to generate the second reference sample may be opposite to the direction of the intra prediction mode of the current block. For example, when the direction of the intra prediction mode of the current block is +45 degrees, the direction used to generate the second reference sample may be +225 degrees.

[0122] Meanwhile, the right, lower, and lower-right regions of the current block are unreconstructed regions, and their sample values ​​can be predicted and used to use them as second reference samples. Figures 6 to 11 Describe its detailed description.

[0123] In addition, the encoder / decoder may perform bidirectional intra prediction based on the first reference sample and the second reference sample (S430). Specifically, the encoder / decoder may perform bidirectional intra prediction by weighting the first reference sample and the second reference sample. The following equation 1 describes the weighting of the first weight W1 and the second reference sample by respectively adding the first weight W2 to the weighted sum of the first reference sample and the second reference sample. _Cur and the second weight W2 _Cur Applied to the first reference sample Ref1 _Cur and the second reference sample Ref2 _Cur To generate the final prediction sample Cur _pred method.

[0124] [Equation 1]

[0125] Cur_ Pred =W1 _cur ×Ref1 _cur +W2 _cur ×Ref2 _cur

[0126] Among them, the first weight W1 _cur and the second weight W2 _cur The sum of can be 1.

[0127] The weight used in the bidirectional intra prediction can be determined by the ratio of the distance between the current sample and each reference sample. That is, the closer the distance between the current sample and the reference sample, the larger the weight can be set, and the farther the distance between the current sample and the reference sample, the smaller the weight can be set.

[0128] In addition, if the directionality of the intra prediction mode of the current block is a vertical directionality mode (ie, if the upper reference sample is used), the directionality may be determined based on the y-axis distance (ie, y-coordinate difference) between the current sample and each reference sample. Conversely, if the directionality of the intra prediction mode of the current block is a horizontal directionality mode (ie, if the left reference sample is used), the directionality may be determined based on the x-axis distance (ie, x-coordinate difference) between the current sample and each reference sample.

[0129] Meanwhile, the weights used in the bidirectional intra prediction may be predetermined (eg, W1 _cur =0.5, W2 _cur =0.5) or adaptively determined by weight information. For example, the weight information may be implicitly derived from neighboring blocks or explicitly signaled via a bitstream.

[0130] Meanwhile, if the intra prediction mode of the current block is a non-directional mode (eg, DC mode or planar mode), the above-mentioned bidirectional intra prediction mode may not be performed. In this case, a unidirectional intra prediction mode may be performed.

[0131] Figure 5 is a diagram for explaining a bidirectional intra prediction method according to an embodiment of the present invention.

[0132] exist Figure 5 In the example, (A1 to A16), (L1 to L16), (B1 to B8), and AL may represent an upper reference sample, a left reference sample, a lower reference sample, and an upper left reference sample, respectively. In addition, Cur represents a sample to be predicted in the current block, and Ref1 _Cur and Ref2 _Cur Represent the first reference sample and the second reference sample respectively. _Cur and W2 _Cur Represent the weight of the first reference sample and the weight of the second reference sample respectively. Figure 5 In the example, BL can represent L9, and AR can represent A9.

[0133] In execution Figure 5 In the case of the existing unidirectional intra prediction in the _Cur The value of generates the predicted sample of the current sample Cur. At this time, there is a difference between the current sample Cur and the first reference sample Ref1 _Cur The large distance between them may reduce the prediction accuracy.

[0134] Figure 5 The bidirectional intra prediction shown in is used to solve the problem of unidirectional intra prediction. Figure 5, the first reference sample Ref1 determined by considering the directionality of the intra prediction mode may be used _Cur and a second reference sample Ref2 corresponding to the first reference sample _Cur To generate a prediction sample. Specifically, as in the above equation 1, the prediction sample of the current sample Cur can be based on the first reference sample Ref1 _Cur and the second reference sample Ref2 _Cur The weighted sum of is generated. At this time, if Figure 5 As shown, the weights of the two reference samples can be determined by the ratio of the distance between the current sample Cur and each reference sample.

[0135] Figure 6 is a flowchart of a method for generating a second reference sample based on template matching according to an embodiment of the present invention.

[0136] refer to Figure 6 , the encoder / decoder may search for a reference template in a search area based on the current template (S610). Specifically, the reference template may be searched in a reconstructed search area in the current picture based on a current template that is adjacent to the current block and has a predefined shape. This may be defined as template matching.

[0137] In addition, the encoder / decoder may generate a second reference sample based on the searched reference template (S620). Specifically, the unreconstructed second reference region of the current block may be predicted based on the right sample, the lower sample, and the lower right sample of the matching block of the searched reference template, and the second reference sample for bidirectional intra prediction may be generated based on the intra prediction mode of the current block in the predicted second reference region.

[0138] Regarding template matching, Figure 7 1 is a diagram for explaining an intra prediction method based on template matching according to an embodiment of the present invention. Here, the intra prediction method based on template matching (Intra Template Matching Prediction, Intra TMP) may refer to a method of searching for an optimal prediction block in a reconstructed area of ​​a current picture using template matching and copying it to generate a prediction block of the current block.

[0139] refer to Figure 7 , the adjacent ┌ area (i.e., the left, upper, and upper-left areas) of the current block 710 may be defined as the current template 720. In addition, a reference template 730 that is most similar to the current template 720 may be searched within a predefined search range R1, R2, R3, and R4 of the reconstruction area of ​​the current picture 700. In addition, a prediction block of the current block 710 may be determined based on a corresponding matching block 740 of the determined reference template 730.

[0140] Figure 7 The predefined search ranges R1, R2, R3, and R4 in may be defined to include a current CTU (coding tree unit), an upper left CTU, an upper CTU, and a left CTU of a current block.

[0141] In addition, in the predefined search range, the reference template may be searched based on a predefined search order. For example, the reference template may be searched in a zigzag order (ie, in the order of R1, R4, R3, and R2).

[0142] Meanwhile, information about the search range and the size and shape of the current template may be determined in the encoder and transmitted to the decoder.

[0143] Figure 8 is a diagram for explaining a method of generating a second reference sample based on template matching according to an embodiment of the present invention.

[0144] refer to Figure 8 , the adjacent ┌ area (i.e., the left, upper and upper-left areas) of the current block 810 may be defined as the current template 820. In addition, a reference template 830 that is most similar to the current template 820 may be searched within a predefined search range R1, R2, R3, and R4 of the reconstructed area of ​​the current picture 800. In addition, a non-reconstructed second reference area 860 of the current block 810 may be predicted based on a reference area 850 including right samples, lower samples, and lower-right samples of a corresponding matching block 840 of the determined reference template 830. In addition, a second reference sample for bidirectional intra prediction may be generated based on the intra prediction mode of the current block in the predicted second reference area 860.

[0145] Figure 8 The predefined search ranges R1, R2, R3, and R4 in may be defined as a current CTU (coding tree unit), an upper left CTU, an upper CTU, and a left CTU including the current block, respectively.

[0146] In addition, in the predefined search range, the reference template may be searched based on a predefined search order. For example, the reference template may be searched in a zigzag order (ie, the order of R1, R4, R3, and R2).

[0147] Meanwhile, information about the search range and the size and shape of the current template may be determined in the encoder and transmitted to the decoder.

[0148] exist Figure 8 The method for generating the second reference sample based on template matching proposed in can generate the second reference sample by performing the same template matching process as that of the encoder in the decoder without signaling (sending / parsing) the syntax related to the generation of the second reference sample.

[0149] Fig. 9 is a flowchart of a method for generating a second reference sample based on motion information according to an embodiment of the present invention.

[0150] refer to Fig. 9 , the encoder / decoder may search for a matching block in a search area based on motion information of the current block (S910). Here, the motion information may be a block vector indicating a matching block in the current picture.

[0151] In addition, the encoder / decoder can generate a second reference sample based on the searched matching block (S920). Specifically, the unreconstructed second reference area of ​​the current block can be predicted based on the right sample, the lower sample and the lower right sample of the searched matching block, and the second reference sample for bidirectional intra prediction can be generated based on the intra prediction mode of the current block in the predicted second reference area.

[0152] Fig.10 is a diagram for explaining a method of generating a second reference sample based on motion information according to an embodiment of the present invention.

[0153] See also Fig.10 , a matching block 1020 may be derived within a predefined search range R1, R2, R3, and R4 of a reconstruction area of ​​the current picture 1000 based on the motion information of the current block 1010. In addition, a non-reconstructed second reference area 1040 of the current block 1010 may be predicted based on a reference area 1030 including a right sample, a lower sample, and a lower-right sample of the matching block 1020. In addition, a second reference area (1040) for bidirectional intra prediction may be generated based on an intra prediction mode of the current block 1010 in the predicted second reference area.

[0154] Although in Fig.10 The matching block is described as being derived within a predefined search range, but the matching block may be derived based on motion information in a reconstructed region within the current picture 1000 .

[0155] At the same time, motion information for generating the second reference sample may be determined in the encoder and sent to the decoder.

[0156] Fig.11 is a diagram for explaining a method of generating a second reference sample based on a neural network according to an embodiment of the present invention.

[0157] Reference Fig.11, the encoder / decoder can predict the second reference region 1130 of the current block 1100 by inputting the reconstructed samples 1110 around the current block 1100 to the neural network processing unit 1120. In addition, the encoder / decoder can generate a second reference sample for bidirectional intra prediction based on the intra prediction mode of the current block 1100 in the predicted second reference region 1130.

[0158] At the same time, the size of the reconstructed samples used as input to the neural network processing unit around the current block may be determined based on the signaling information. For example, the size of the reconstructed samples used as input to the neural network processing unit may be (L1×2h)+(2w×L2)+(L1×L2). At this time, the sizes of L1 and L2 may be determined in the encoder and sent to the decoder. As another example, the size of the reconstructed samples used as input to the neural network processing unit may be (L1×h)+(w×L2)+(L1×L2).

[0159] Meanwhile, the size of the reconstructed samples surrounding the current block as input to the neural network processing unit may be a predetermined fixed size.

[0160] exist Fig.11 In the embodiment of the present invention, the neural network processing unit 1120 can be implemented by a neural network model. Here, the artificial neural network model can represent a deep neural network including one or more neural layers. In addition, the neural network model can include all or some of the convolutional layers, fully connected layers, and pooling layers. The neural network model can be implemented in the form of one type of neural layer, or can be implemented in the form of a combination of different types of layers.

[0161] Meanwhile, initial internal parameters of the neural network model used in the neural network processing unit 1120 are pre-learned but may be additionally learned during the encoding / decoding process.

[0162] Already referenced Figures 6 to 11 A second reference sample generation method for bidirectional intra prediction according to various embodiments of the present invention is described.

[0163] In bidirectional intra prediction, the second reference sample generation method may be adaptively selected. Information indicating the second reference sample generation method may be determined in the encoder and transmitted to the decoder.

[0164] At the same time, use Figures 6 to 11The second reference area predicted by various methods in the second reference sample generation method can be used for unidirectional intra prediction. As the reference area for unidirectional intra prediction, not only the reconstructed sample area on the left, top and upper left of the current block (i.e., the first reference area), but also the predicted sample area on the right, bottom and lower right side of the current block (i.e., the second reference area) can be used. In this case, the directional intra prediction mode of the unidirectional intra prediction may include all 360-degree omnidirectional modes. In this way, by realizing 360-degree omnidirectional intra prediction via prediction of the second reference area, the problem of reduced prediction accuracy of the current sample far away from the first reference sample, which is a problem of unidirectional intra prediction using only the existing first reference area, can be solved.

[0165] Fig.12 is a flowchart illustrating an image decoding method according to an embodiment of the present invention. Fig.12 The image decoding method can be executed by an image decoding device.

[0166] The image decoding device may generate a first reference sample based on a first reference area adjacent to the current block (S1210). Specifically, the image decoding device may generate a first reference sample from the first reference area according to the direction of the intra prediction mode of the current block. Here, the first reference area may be a reconstruction area located to the left, above, and left-upper of the current block.

[0167] In addition, the image decoding device may generate a second reference sample based on a second reference area adjacent to the current block (S1220). Specifically, the image decoding device may generate a second reference sample from the second reference area by considering the direction of the intra prediction mode of the current block. Here, the second reference area may be a predicted area located to the right, bottom, and bottom-right of the current block.

[0168] Meanwhile, according to an embodiment of the present invention, the second reference area may be predicted based on adjacent samples of the matching block searched by performing template matching. Here, template matching may mean searching for a reference template most similar to the current template around the current block in the reconstructed search area of ​​the current picture and determining the matching block based on the searched reference template. Figures 6 to 8 A method for generating a second reference sample based on template matching is described in detail.

[0169] Meanwhile, according to an embodiment of the present invention, the second reference region may be predicted based on adjacent samples of a matching block in the current picture indicated by motion information of the current block. Here, the motion information may be a block vector. Figures 9 and 10 The method of generating the second reference sample based on motion information has been described in detail.

[0170] Meanwhile, according to an embodiment of the present invention, the second reference region can be predicted based on a neural network model that receives the reconstructed sample of the first reference region as input. Fig.11 The second reference sample generation method based on neural network is described in detail.

[0171] Meanwhile, according to an embodiment of the present invention, the first reference sample and the second reference sample may be located in opposite directions with respect to the current block.

[0172] In addition, the image decoding apparatus may perform intra prediction on the current block based on the first reference sample and the second reference sample (S1230). The image decoding apparatus may perform intra prediction to generate a predicted block of the current block.

[0173] Specifically, the image decoding device can perform intra-frame prediction by applying the first weight and the second weight to the first reference sample and the second reference sample, respectively. Here, the first weight and the second weight can be determined based on the ratio of the distance between the current prediction target sample and the first reference sample to the distance between the current prediction target sample and the second reference sample.

[0174] Meanwhile, according to an embodiment of the present invention, when the intra prediction mode of the current block is a non-directional mode, intra prediction may be performed using only the first reference sample.

[0175] At the same time, Fig.12 The steps described in the above may be performed in the same manner as in the image encoding method. In addition, the bit stream may be included in Fig.12 The image encoding method of the steps described in the above is used to generate the bit stream. The bit stream can be stored in a non-transitory computer-readable recording medium and can also be transmitted (or streamed).

[0176] Fig.13 A content streaming system applicable according to an embodiment of the present invention is exemplarily shown.

[0177] like Fig.13 As shown, the content streaming system to which the embodiment of the present invention is applied may mainly include an encoding server, a streaming server, a network server, a media storage, a user device and a multimedia input device.

[0178] The encoding server compresses the content received from the multimedia input device (such as a smart phone, camera, CCTV, etc.) into digital data to generate a bit stream and transmits it to the streaming server. As another example, if the multimedia input device (e.g., a smart phone, camera, CCTV, etc.) directly generates the bit stream, the encoding server can be omitted.

[0179] A bitstream may be generated by the image encoding method and / or the image encoding apparatus to which the embodiments of the present invention are applied, and in the process of transmitting or receiving the bitstream, the streaming server may temporarily store the bitstream.

[0180] The streaming server transmits multimedia data to the user device via the network server based on the user request, and the network server can act as an intermediary to inform the user of any available services. When the user requests the desired service from the network server, the network server sends it to the streaming server, and the streaming server can send the multimedia data to the user. At this time, the content streaming system may include a separate control server, in which case the control server can control the command / response between the devices within the content streaming system.

[0181] The streaming server can receive content from a media storage and / or an encoding server. For example, when receiving content from an encoding server, the content can be received in real time. In this case, in order to provide a smooth streaming service, the streaming server can store the bitstream for a specific period of time.

[0182] Examples of user devices may include mobile phones, smart phones, laptop computers, digital broadcast terminals, personal digital assistants (PDAs), portable multimedia players (PMPs), navigation devices, tablet PCs, tablet PCs, ultrabooks, wearable devices (e.g., smart watches, smart glasses, HMDs), digital TVs, desktop computers, digital signage, etc.

[0183] Each server in the above content streaming system may operate as a distributed server, in which case data received from each server may be distributed and processed.

[0184] The above embodiments may be performed in the encoding device and the decoding device in the same or corresponding manner. In addition, at least one embodiment of the above embodiments or a combination of at least one of the above embodiments may be used to encode / decode an image.

[0185] In the encoding device and the decoding device, the order of applying the above embodiments may be different. Alternatively, in the encoding device and the decoding device, the order of applying the above embodiments may be the same.

[0186] The above embodiments may be performed for each of the luminance and chrominance signals. Alternatively, the above embodiments for the luminance and chrominance signals may be performed identically.

[0187] In the above embodiments, the method is described based on a flow chart having a series of steps or units, but the present invention is not limited to the order of the steps, but some steps can be performed simultaneously with other steps or in a different order. In addition, it should be understood by those skilled in the art that the steps in the flow chart are not mutually exclusive, and other steps can be added to the flow chart or some steps can be deleted from the flow chart without affecting the scope of the present invention.

[0188] The embodiments may be implemented in the form of program instructions executable by various computer components and recorded in a computer-readable recording medium. The computer-readable recording medium may include independent program instructions, data files, data structures, etc. or a combination of program instructions, data files, data structures, etc. The program instructions recorded in the computer-readable recording medium may be specially designed and constructed for the present invention, or may be known to those of ordinary skill in the field of computer software technology.

[0189] The bit stream generated by the encoding method according to the above-described embodiment may be stored in a non-transitory computer-readable recording medium. In addition, the bit stream stored in the non-transitory computer-readable recording medium may be decoded by the decoding method according to the above-described embodiment.

[0190] Examples of computer-readable recording media include magnetic recording media such as hard disks, floppy disks, and magnetic tapes; optical data storage media such as CD-ROMs or DVD-ROMs; magnetic optimal media such as floppy disks; and hardware devices such as read-only memory (ROM), random access memory (RAM), flash memory, etc., which are specially constructed to store and implement program instructions. Examples of program instructions include not only machine language codes formatted by encoders, but also high-level language codes that can be implemented by computers using interpreters. Hardware devices can be configured to be operated by one or more software modules, and vice versa, to perform processing according to the present invention.

[0191] Although the present invention has been described with respect to specific items such as detailed elements and limited embodiments and drawings, these items are only provided to help a more comprehensive understanding of the present invention, and the present invention is not limited to the above embodiments. It will be appreciated by those skilled in the art that various modifications and changes may be made from the above description.

[0192] Therefore, the spirit of the present invention should not be limited to the above-described embodiments, and the full scope of the appended claims and their equivalents will fall within the scope and spirit of the present invention.

[0193] Industrial Applicability

[0194] The present invention can be used for an apparatus for encoding / decoding an image and a recording medium for storing a bit stream.

Claims

1. An image decoding method, comprising: generating a first reference sample based on a first reference area adjacent to the current block; generating a second reference sample based on a second reference area adjacent to the current block; as well as performing intra prediction on the current block based on the first reference sample and the second reference sample, The first reference area is a reconstruction area located to the left, above, and above-left of the current block, and The second reference area is a prediction area located to the right, below and right below the current block.

2. The image decoding method according to claim 1, wherein: The second reference region is predicted based on neighboring samples of a matching block searched by performing template matching.

3. The image decoding method according to claim 2, wherein: The template matching includes searching for a reference template most similar to a current template around the current block in a reconstructed search area of ​​the current picture, and determining a matching block based on the searched reference template.

4. The image decoding method according to claim 1, wherein: The second reference region is predicted based on neighboring samples of a matching block indicated by motion information of the current block in a current picture.

5. The image decoding method according to claim 4, wherein: The motion information is a block vector.

6. The image decoding method according to claim 1, wherein: The second reference region is predicted based on a neural network model that receives as input the reconstructed samples of the first reference region.

7. The image decoding method according to claim 1, wherein: The first reference sample and the second reference sample are located in opposite directions relative to the current block.

8. The image decoding method according to claim 1, wherein: Performing the intra prediction includes performing the intra prediction by applying a first weight and a second weight to the first reference sample and the second reference sample, respectively.

9. The image decoding method according to claim 8, wherein: The first weight and the second weight are determined based on a ratio of a distance between a current prediction target sample and the first reference sample to a distance between the current prediction target sample and the second reference sample.

10. The image decoding method according to claim 1, wherein: When the intra prediction mode of the current block is a non-directional mode, the intra prediction is performed using only the first reference samples.

11. An image encoding method, comprising: generating a first reference sample based on a first reference area adjacent to the current block; generating a second reference sample based on a second reference area adjacent to the current block; as well as performing intra prediction on the current block based on the first reference sample and the second reference sample, The first reference area is a reconstruction area located to the left, above, and above-left of the current block, and The second reference area is a prediction area located to the right, below and right below the current block.

12. A non-transitory computer-readable recording medium for storing a bit stream generated by an image encoding method, the image encoding method comprising: generating a first reference sample based on a first reference area adjacent to the current block; generating a second reference sample based on a second reference area adjacent to the current block; as well as performing intra prediction on the current block based on the first reference sample and the second reference sample, The first reference area is a reconstruction area located to the left, above, and above-left of the current block, and The second reference area is a prediction area located to the right, below and right below the current block.

13. A method for transmitting a bit stream generated by an image encoding method, the method comprising: transmitting the bit stream, Wherein, the image encoding method comprises: generating a first reference sample based on a first reference area adjacent to the current block; generating a second reference sample based on a second reference area adjacent to the current block; and performing intra prediction on the current block based on the first reference sample and the second reference sample, The first reference area is a reconstruction area located to the left, above, and above-left of the current block, and The second reference area is a prediction area located to the right, below and right below the current block.

Images