Image encoding / decoding method and apparatus, and recording medium storing bitstream

Abstract

An image decoding / encoding method and apparatus according to the present disclosure may: construct a reference sample for linear model intra prediction based on reconstructed neighboring samples of a current block; deriving a linear model parameter based on the reference sample; generating a prediction sample of the current block based on the linear model parameters; and decoding / encoding the current block based on the prediction sample of the current block.

Description

Technical Field

[0001] The present disclosure relates to an image encoding / decoding method and apparatus and a recording medium having a bit stream stored therein. Background Art

[0002] Recently, demands for high-resolution and high-quality images such as HD (High Definition) images and UHD (Ultra High Definition) images have been increasing in various application fields, and therefore, efficient image compression technology is being discussed.

[0003] There are various technologies, such as inter-frame prediction technology that uses video compression technology to predict pixel values ​​included in the current picture from pictures before or after the current picture, intra-frame prediction technology that predicts pixel values ​​included in the current picture by using pixel information in the current picture, entropy coding technology that assigns short symbols to values ​​with high occurrence frequency and assigns long symbols to values ​​with low occurrence frequency, etc., and these image compression technologies can be used to effectively compress image data and transmit or store it. Summary of the invention

[0004] Technical issues

[0005] The present disclosure is directed to providing a method and apparatus for deriving a linear model based on neighboring reconstructed samples and performing intra prediction using the derived linear model.

[0006] The present disclosure is directed to providing methods and apparatus for constructing reference samples for deriving linear models.

[0007] The present disclosure is directed to providing a method and apparatus for efficiently constructing a reference sample taking into account the shape of a block.

[0008] The present disclosure is directed to providing methods and apparatus for designing various linear models to improve prediction accuracy.

[0009] Technical Solution

[0010] The image decoding method and device according to the present disclosure construct reference samples for linear model intra prediction based on reconstructed adjacent samples of the current block, derive linear model parameters based on the reference samples, and generate prediction samples of the current block based on the linear model parameters.

[0011] In the image decoding method and apparatus according to the present disclosure, the linear model parameter may include at least one of a first linear model parameter related to a variance in a horizontal direction or a second linear model parameter related to a variance in a vertical direction.

[0012] In the image decoding method and device according to the present disclosure, the reference sample may include at least one of the following items: a reference sample included in a left sample line adjacent to the current block, a reference sample included in an upper sample line adjacent to the current block, or an upper-left reference sample adjacent to the current block.

[0013] The image decoding method and apparatus according to the present disclosure may derive first and second linear model parameters based on a reference sample, and apply a weight to at least one of the first or second linear model parameters.

[0014] In the image decoding method and device according to the present disclosure, the weight may be determined based on predefined encoding information.

[0015] In the image decoding method and apparatus according to the present disclosure, the predefined encoding information may include at least one of a width, a height, a size, a shape, or a ratio of a width to a height of the current block.

[0016] The image decoding method and apparatus according to the present disclosure may set a reference sample included in one of a left sample line or an upper sample line to a predetermined value.

[0017] In the image decoding method and apparatus according to the present disclosure, the predetermined value may be one of an average value, a maximum value, a minimum value, or a median value of reference samples included in one of the left sample line or the upper sample line.

[0018] In the image decoding method and apparatus according to the present disclosure, the predetermined value may be determined as a value of a first reference sample or a last reference sample based on an upper left reference sample among reference samples included in one of a left sample line or an upper sample line.

[0019] According to the image decoding method and apparatus of the present disclosure, a reference line offset indicating one of a plurality of reference sample lines of a current block can be obtained.

[0020] In the image decoding method and apparatus according to the present disclosure, the reference sample may be constructed using reference samples included in a reference sample line indicated by a reference line offset.

[0021] The image encoding method and apparatus according to the present disclosure construct reference samples for linear model intra prediction based on reconstructed adjacent samples of the current block, derive linear model parameters based on the reference samples, and generate prediction samples of the current block based on the linear model parameters.

[0022] In the image encoding method and apparatus according to the present disclosure, the linear model parameter may include at least one of a first linear model parameter related to a variance in a horizontal direction or a second linear model parameter related to a variance in a vertical direction.

[0023] In the image encoding method and device according to the present disclosure, the reference sample may include at least one of the following items: a reference sample included in a left sample line adjacent to the current block, a reference sample included in an upper sample line adjacent to the current block, or an upper-left reference sample adjacent to the current block.

[0024] The image encoding method and apparatus according to the present disclosure may derive a first linear model parameter and a second linear model parameter based on a reference sample, and apply a weight to at least one of the first linear model parameter or the second linear model parameter.

[0025] In the image encoding method and apparatus according to the present disclosure, the weight may be determined based on predefined encoding information.

[0026] In the image encoding method and apparatus according to the present disclosure, the predefined encoding information may include at least one of a width, a height, a size, a shape, or a ratio of a width to a height of the current block.

[0027] The image encoding method and apparatus according to the present disclosure may set a reference sample included in one of a left sample line or an upper sample line to a predetermined value.

[0028] In the image encoding method and apparatus according to the present disclosure, the predetermined value may be one of an average value, a maximum value, a minimum value, or a median value of reference samples included in one of the left sample line or the upper sample line.

[0029] In the image encoding method and apparatus according to the present disclosure, the predetermined value may be determined as a value of a first reference sample or a last reference sample based on an upper left reference sample among reference samples included in one of a left sample line or an upper sample line.

[0030] According to the image encoding method and apparatus of the present disclosure, a reference line offset indicating one of a plurality of reference sample lines of a current block can be obtained.

[0031] In the image encoding method and apparatus according to the present disclosure, the reference sample may be constructed using reference samples included in a reference sample line indicated by a reference line offset.

[0032] A computer-readable digital storage medium is provided, in which encoded video / image information is stored for enabling an image decoding device according to the present disclosure to perform an image decoding method.

[0033] A computer-readable digital storage medium is provided, in which video / image information generated according to the image encoding method of the present disclosure is stored.

[0034] A method and apparatus for transmitting video / image information generated based on an image encoding method according to the present disclosure are provided.

[0035] Technical Effects

[0036] According to the present disclosure, a linear model is derived based on adjacent reconstructed samples and intra-frame prediction is performed using the derived linear model, so more diverse prediction methods can be considered in the prediction, the prediction accuracy can be improved, and the compression performance can be improved.

[0037] According to the present disclosure, by effectively constructing reference samples for linear model derivation, prediction accuracy can be improved and compression performance can be improved.

[0038] According to the present disclosure, by effectively constructing reference samples in consideration of the shape of a block, prediction accuracy may be improved and compression performance may be improved.

[0039] According to the present disclosure, the accuracy of prediction can be improved by providing design methods and devices for various linear models. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 A video / image compilation system according to the present disclosure is shown.

[0041] Figure 2 A rough block diagram of an encoding device to which embodiments of the present disclosure can be applied and to perform encoding of a video / image signal is shown.

[0042] Figure 3 A rough block diagram of a decoding device to which embodiments of the present disclosure can be applied and which performs decoding of a video / image signal is shown.

[0043] Figure 4 It is a diagram showing an intra prediction method performed by a decoding device (300) according to an embodiment of the present disclosure.

[0044] Figure 5 is a diagram illustrating a schematic configuration of an intra-frame predictor (331) for performing an intra-frame prediction method according to the present disclosure.

[0045] Figure 6 It is a diagram illustrating an inter-frame prediction method performed by an encoding device (200) according to an embodiment of the present disclosure.

[0046] Figure 7 is a diagram illustrating a schematic configuration of an intra-frame predictor (222) for performing an intra-frame prediction method according to the present disclosure.

[0047] Figure 8 is a diagram illustrating an example of a content streaming system to which an embodiment disclosed in the present disclosure can be applied. DETAILED DESCRIPTION

[0048] Because the present disclosure can make various changes and has several embodiments, specific embodiments will be illustrated in the drawings and described in detail in the detailed description. However, it is not intended to limit the present disclosure to specific embodiments, and it should be understood to include all changes, equivalents and substitutes included in the spirit and technical scope of the present disclosure. While describing each of the accompanying drawings, similar reference numerals are used for similar components.

[0049] Terms such as first, second, etc. can be used to describe various components, but components should not be limited by these terms. These terms are only used to distinguish one component from other components. For example, without departing from the scope of the present disclosure, a first component can be referred to as a second component, and similarly, a second component can also be referred to as a first component. Terms and / or combinations of any one or more related statement items in a plurality of related statement items are included.

[0050] When a component is referred to as being "connected" or "linked" to another component, it should be understood that it can be directly connected or linked to another component, but another component may also exist in between. On the other hand, when a component is referred to as being "directly connected" or "directly linked" to another component, it should be understood that another component does not exist in between.

[0051] The terms used in this application are only used to describe specific embodiments and are not intended to limit the present disclosure. Unless the context clearly indicates otherwise, singular expressions include plural expressions. In this application, it should be understood that terms such as "including" or "having" are intended to designate the existence of features, numbers, steps, operations, components, parts or combinations thereof described in the specification, but do not exclude the possibility of the existence or addition of one or more other features, numbers, steps, operations, components, parts or combinations thereof in advance.

[0052] The present disclosure relates to video / image coding. For example, the methods / implementations disclosed herein may be applied to methods disclosed in the general video coding (VVC) standard. In addition, the methods / implementations disclosed herein may be applied to methods disclosed in the basic video coding (EVC) standard, the AOMedia video 1 (AV1) standard, the second generation audio video coding standard (AVS2), or the next generation video / image coding standard (e.g., H.267 or H.268, etc.).

[0053] This specification proposes various embodiments of video / image coding, and unless otherwise stated, these embodiments may be performed in combination with each other.

[0054] Here, video may refer to a collection of a series of images over time. A picture generally refers to a unit representing an image within a specific time period, and a slice / tile is a unit that forms a part of a picture in coding. A slice / tile may include at least one coding tree unit (CTU). A picture may be composed of at least one slice / tile. A tile is a rectangular area consisting of a plurality of CTUs within a specific tile column and a specific tile row of a picture. A tile column is a rectangular area of ​​a CTU having the same height as the picture and a width assigned by the syntax requirements of the picture parameter set. A tile row is a rectangular area of ​​a CTU having a height assigned by the picture parameter set and the same width as the width of the picture. The CTU within a tile may be arranged continuously according to a CTU raster scan, and the tiles within a picture may be arranged continuously according to a raster scan of the tile. A slice may include an integer number of complete tiles or an integer number of continuous complete CTU rows within a tile of a picture that may be exclusively included in a single NAL unit. At the same time, a picture may be divided into at least two sub-pictures. A sub-picture may be a rectangular area of ​​at least one slice within a picture.

[0055] Pixel, pixel or picture element may refer to the smallest unit constituting a picture (or image). In addition, "sample" may be used as a term corresponding to a pixel. A sample may generally represent a pixel or a pixel value, and may represent only a pixel / pixel value of a luminance component, or only a pixel / pixel value of a chrominance component.

[0056] A unit may represent a basic unit of image processing. A unit may include at least one of a specific region of a picture and information associated with the corresponding region. A unit may include a luminance block and two chrominance (e.g., cb, cr) blocks. In some cases, a unit may be used interchangeably with terms such as a block or region. In general, an MxN block may include a set (or array) of transform coefficients or samples (or sample arrays) consisting of M columns and N rows.

[0057] Here, "A or B" may refer to "only A", "only B", or "both A and B". In other words, herein, "A or B" may be interpreted as "A and / or B". For example, herein, "A, B or C" may refer to "only A", "only B", "only C", or "any combination of A, B, and C".

[0058] As used herein, a slash ( / ) or a comma may mean "and / or". For example, "A / B" may mean "A and / or B". Thus, "A / B" may mean "only A", "only B", or "both A and B". For example, "A, B, C" may mean "A, B, or C".

[0059] Here, "at least one of A and B" may refer to "only A", "only B", or "both A and B". In addition, herein, expressions such as "at least one of A or B" or "at least one of A and / or B" can be interpreted in the same manner as "at least one of A and B".

[0060] In addition, herein, “at least one of A, B, and C” may refer to “only A”, “only B”, “only C”, or “any combination of A, B, and C”. In addition, “at least one of A, B, or C” or “at least one of A, B and / or C” may refer to “at least one of A, B, and C”.

[0061] In addition, the brackets used herein may refer to "for example". Specifically, when the indication is "prediction (intra-frame prediction)", "intra-frame prediction" may be proposed as an example of "prediction". In other words, the "prediction" here is not limited to "intra-frame prediction", and "intra-frame prediction" may be proposed as an example of "prediction". In addition, even when the indication is "prediction (ie, intra-frame prediction)", "intra-frame prediction" may be proposed as an example of "prediction".

[0062] Here, technical features described individually in one drawing may be implemented individually or simultaneously.

[0063] Figure 1 A video / image compilation system according to the present disclosure is shown.

[0064] Reference Figure 1 , a video / image coding system may include a first device (source device) and a second device (receiving device).

[0065] The source device may send the encoded video / image information or data to the receiving device in the form of a file or stream transmission through a digital storage medium or a network. The source device may include a video source, an encoding device, and a sending unit. The receiving device may include a receiving unit, a decoding device, and a renderer. The encoding device may be referred to as a video / image encoding device, and the decoding device may be referred to as a video / image decoding device. The transmitter may be included in the encoding device. The receiver may be included in the decoding device. The renderer may include a display unit, and the display unit may be composed of a separate device or an external component.

[0066] The video source may acquire the video / image by a process of capturing, synthesizing or generating the video / image. The video source may include a device for capturing the video / image and a device for generating the video / image. The device for capturing the video / image may include at least one camera, a video / image archive including previously captured videos / images, etc. The device for generating the video / image may include a computer, a tablet computer, a smart phone, etc., and may (electronically) generate the video / image. For example, a virtual video / image may be generated by a computer, etc., and in this case, the process of capturing the video / image may be replaced by a process of generating the relevant data.

[0067] The encoding device can encode the input video / image. The encoding device can perform a series of processes such as prediction, transformation, quantization, etc. for compression and coding efficiency. The encoded data (encoded video / image information) can be output in the form of a bit stream.

[0068] The sending unit may send the encoded video / image information or data output in the form of a bit stream to the receiving unit of the receiving device in the form of a file or stream transmission through a digital storage medium or a network. The digital storage medium may include various storage media, such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The sending unit may include an element for generating a media file in a predetermined file format and may include an element for transmitting through a broadcast / communication network. The receiving unit may receive / extract a bit stream and send it to a decoding device.

[0069] The decoding device may decode the video / image by performing a series of processes corresponding to the operations of the encoding device, such as inverse quantization, inverse transformation, prediction, etc.

[0070] The renderer may render the decoded video / image. The rendered video / image may be displayed through a display unit.

[0071] Figure 2 A rough block diagram of an encoding device to which embodiments of the present disclosure can be applied and to perform encoding of a video / image signal is shown.

[0072] Reference Figure 2, the encoding device 200 may be composed of an image segmenter 210, a predictor 220, a residual processor 230, an entropy encoder 240, an adder 250, a filter 260, and a memory 270. The predictor 220 may include an inter-frame predictor 221 and an intra-frame predictor 222. The residual processor 230 may include a transformer 232, a quantizer 233, an inverse quantizer 234, and an inverse transformer 235. The residual processor 230 may further include a subtractor 231. The adder 250 may be referred to as a reconstructor or a reconstructed block generator. According to an embodiment, the above-mentioned image segmenter 210, the predictor 220, the residual processor 230, the entropy encoder 240, the adder 250, and the filter 260 may be configured by at least one hardware component (e.g., an encoder chipset or processor). In addition, the memory 270 may include a decoded picture buffer (DPB) and may be configured by a digital storage medium. The hardware component may further include a memory 270 as an internal / external component.

[0073] The image divider 210 may divide the input image (or picture, frame) input to the encoding device 200 into at least one processing unit. As an example, the processing unit may be referred to as a coding unit (CU). In this case, the coding unit may be recursively divided from a coding tree unit (CTU) or a maximum coding unit (LCU) according to a quadtree binary tree ternary tree (QTBTTT) structure.

[0074] For example, one coding unit may be segmented into a plurality of coding units having a deeper depth based on a quadtree structure, a binary tree structure, and / or a ternary structure. In this case, for example, the quadtree structure may be applied first, and the binary tree structure and / or the ternary structure may be applied later. Alternatively, the binary tree structure may be applied before the quadtree structure. The coding process according to this specification may be performed based on a final coding unit that is no longer segmented. In this case, based on coding efficiency according to image characteristics, etc., the maximum coding unit may be directly used as the final coding unit, or if necessary, the coding unit may be recursively segmented into coding units of a deeper depth, and the coding unit with the best size may be used as the final coding unit. Here, the coding process may include processes such as prediction, transformation, and reconstruction described later.

[0075] As another example, the processing unit may further include a prediction unit (PU) or a transform unit (TU). In this case, the prediction unit and the transform unit may be divided or partitioned from the above-mentioned final coding unit, respectively. The prediction unit may be a unit of sample prediction, and the transform unit may be a unit for deriving a transform coefficient and / or a unit for deriving a residual signal from the transform coefficient.

[0076] In some cases, a unit may be used interchangeably with terms such as a block or region. In general, an MxN block may represent a set of transform coefficients or samples consisting of M columns and N rows. A sample may generally represent a pixel or a pixel value, and may represent only a pixel / pixel value of a luma component, or only a pixel / pixel value of a chroma component. A sample may be used as a term to make a picture (or image) correspond to a pixel or a picture element.

[0077] The encoding device 200 may subtract the prediction signal (prediction block, prediction sample array) output from the inter predictor 221 or the intra predictor 222 from the input image signal (original block, original sample array) to generate a residual signal (residual signal, residual sample array), and the generated residual signal is sent to the transformer 232. In this case, the unit that subtracts the prediction signal (prediction block, prediction sample array) from the input image signal (original block, original sample array) within the encoding device 200 may be referred to as a subtractor 231.

[0078] The predictor 220 may perform prediction on a block to be processed (hereinafter referred to as a current block) and generate a predicted block including a prediction sample for the current block. The predictor 220 may determine whether intra prediction or inter prediction is applied in units of the current block or CU. The predictor 220 may generate various information about the prediction, such as prediction mode information, and send it to the entropy encoder 240, as described later in the description of each prediction mode. The information about the prediction may be encoded in the entropy encoder 240 and output in the form of a bitstream.

[0079] The intra-frame predictor 222 can predict the current block by referring to the samples in the current picture. Depending on the prediction mode, the referenced sample can be located near the current block or can be located at a specific distance away from the current block. In intra-frame prediction, the prediction mode may include at least one non-directional mode and multiple directional modes. The non-directional mode may include at least one of the DC mode or the plane mode. Depending on the detail level of the prediction direction, the directional mode may include 33 directional modes or 65 directional modes. However, this is only an example, and more or less directional modes may be used depending on the configuration. The intra-frame predictor 222 may determine the prediction mode applied to the current block by using the prediction mode applied to the neighboring blocks.

[0080] The inter-frame predictor 221 may derive a prediction block for the current block based on a reference block (reference sample array) specified by a motion vector on a reference picture. In this case, in order to reduce the amount of motion information sent in the inter-frame prediction mode, the motion information may be predicted in units of blocks, sub-blocks, or samples based on the correlation of the motion information between the neighboring blocks and the current block. The motion information may include a motion vector and a reference picture index. The motion information may further include inter-frame prediction direction information (L0 prediction, L1 prediction, Bi prediction, etc.). For inter-frame prediction, the neighboring blocks may include spatial neighboring blocks present in the current picture and temporal neighboring blocks present in the reference picture. The reference picture including the reference block and the reference picture including the temporal neighboring block may be the same or different. The temporal neighboring block may be referred to as a collocated reference block, a collocated CU (colCU), etc., and the reference picture including the temporal neighboring block may be referred to as a collocated picture (colPic). For example, the inter-frame predictor 221 may configure a motion information candidate list based on the neighboring blocks, and generate information indicating which candidate is used to derive the motion vector and / or reference picture index of the current block. Inter prediction may be performed based on various prediction modes, and for example, for skip mode and merge mode, the inter predictor 221 may use motion information of neighboring blocks as motion information of the current block. For skip mode, unlike merge mode, a residual signal may not be sent. For motion vector prediction (MVP) mode, the motion vector of the neighboring block is used as a motion vector predictor, and a motion vector difference is signaled to indicate the motion vector of the current block.

[0081] The predictor 220 may generate a prediction signal based on various prediction methods described later. For example, the predictor may not only apply intra prediction or inter prediction to predict a block, but may also apply intra prediction and inter prediction at the same time. It may be referred to as a combined inter and intra prediction (CIIP) mode. In addition, the predictor may be based on an intra block copy (IBC) prediction mode or may be based on a palette mode for prediction of a block. The IBC prediction mode or the palette mode may be used for content image / video coding of games, such as screen content coding (SCC), etc. IBC basically performs prediction within the current picture, but it may be performed similarly to inter prediction because it derives a reference block within the current picture. In other words, IBC may use at least one of the inter prediction techniques described herein. The palette mode may be considered an example of intra coding or intra prediction. When the palette mode is applied, the sample values ​​within the picture may be signaled based on information about the palette table and the palette index. The prediction signal generated by the predictor 220 may be used to generate a reconstructed signal or a residual signal.

[0082] The transformer 232 may generate transform coefficients by applying a transform technique to the residual signal. For example, the transform technique may include at least one of a discrete cosine transform (DCT), a discrete sine transform (DST), a Karhunen-Loeve transform (KLT), a graph-based transform (GBT), or a conditional nonlinear transform (CNT). Here, GBT refers to a transform obtained from a graph when the relationship information between pixels is expressed as a graph. CNT refers to a transform obtained based on generating a prediction signal using all previously reconstructed pixels. In addition, the transform process may be applied to square pixel blocks of the same size or may be applied to non-square blocks of variable size.

[0083] The quantizer 233 may quantize the transform coefficients and send them to the entropy encoder 240, and the entropy encoder 240 may encode the quantized signal (information about the quantized transform coefficients) and output it as a bitstream. The information about the quantized transform coefficients may be referred to as residual information. The quantizer 233 may rearrange the quantized transform coefficients in the form of blocks into a one-dimensional vector form based on the coefficient scanning order, and may generate information about the quantized transform coefficients based on the quantized transform coefficients in the form of the one-dimensional vector.

[0084] The entropy encoder 240 may perform various encoding methods such as Exponential Golomb, Context Adaptive Variable Length Coding (CAVLC), Context Adaptive Binary Arithmetic Coding (CABAC), etc. The entropy encoder 240 may encode information necessary for video / video image reconstruction (e.g., values ​​of syntax elements, etc.) in addition to transform coefficients quantized together or individually.

[0085] The encoded information (e.g., encoded video / image information) can be transmitted or stored in units of network abstraction layer (NAL) units in the form of a bitstream. The video / image information may further include information about various parameter sets such as an adaptive parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video / image information may further include general constraint information. Here, information and / or syntax elements sent from the encoding device / signaled to the decoding device may be included in the video / image information. The video / image information may be encoded and included in the bitstream through the above-mentioned encoding process. The bitstream may be transmitted through a network or may be stored in a digital storage medium. Here, the network may include a broadcast network and / or a communication network, etc., and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. A transmission unit (not shown) for transmission and / or a storage unit (not shown) for storing a signal output from the entropy encoder 240 may be configured as an internal / external element of the encoding device 200, or the transmission unit may also be included in the entropy encoder 240.

[0086] The quantized transform coefficients output from the quantizer 233 can be used to generate a prediction signal. For example, the residual signal (residual block or residual sample) can be reconstructed by applying dequantization and inverse transform to the quantized transform coefficients through the inverse quantizer 234 and the inverse transformer 235. The adder 250 can add the reconstructed residual signal to the prediction signal output from the inter-frame predictor 221 or the intra-frame predictor 222 to generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array). When there is no residual of the block to be processed, such as when the skip mode is applied, the prediction block can be used as a reconstructed block. The adder 250 can be referred to as a reconstructor or a reconstructed block generator. The generated reconstructed signal can be used for intra-frame prediction of the next block to be processed in the current picture, and can also be used for inter-frame prediction of the next picture through filtering described later. At the same time, luminance mapping with chroma scaling (LMCS) can be applied in the picture encoding and / or reconstruction process.

[0087] The filter 260 can improve the subjective / objective image quality by applying filtering to the reconstructed signal. For example, the filter 260 can generate a modified reconstructed picture by applying various filtering methods to the reconstructed picture, and the modified reconstructed picture can be stored in the memory 270, specifically in the DPB of the memory 270. Various filtering methods may include deblocking filtering, sample adaptive offset, adaptive loop filter, bilateral filter, etc. The filter 260 can generate various information about filtering and send it to the entropy encoder 240. The information about filtering can be encoded in the entropy encoder 240 and output in the form of a bit stream.

[0088] The modified reconstructed picture transmitted to the memory 270 may be used as a reference picture in the inter predictor 221. When inter prediction is applied therethrough, the encoding apparatus can avoid prediction mismatch in the encoding apparatus 200 and the decoding apparatus, and can also improve encoding efficiency.

[0089] The DPB of the memory 270 may store the modified reconstructed picture to be used as a reference picture in the inter-frame predictor 221. The memory 270 may store the motion information of the block from which the motion information in the current picture is derived (or encoded) and / or the motion information of the block in the pre-reconstructed picture. The stored motion information may be sent to the inter-frame predictor 221 to be used as the motion information of the spatial neighboring block or the motion information of the temporal neighboring block. The memory 270 may store the reconstructed samples of the reconstructed block in the current picture and send them to the intra-frame predictor 222.

[0090] Figure 3 A rough block diagram of a decoding device to which embodiments of the present disclosure can be applied and which performs decoding of a video / image signal is shown.

[0091] Reference Figure 3 , the decoding device 300 may be configured by including an entropy decoder 310, a residual processor 320, a predictor 330, an adder 340, a filter 350, and a memory 360. The predictor 330 may include an inter-frame predictor 332 and an intra-frame predictor 331. The residual processor 320 may include an inverse quantizer 321 and an inverse transformer 322.

[0092] According to an embodiment, the above entropy decoder 310, the residual processor 320, the predictor 330, the adder 340 and the filter 350 may be configured by one hardware component (e.g., a decoder chipset or a processor). In addition, the memory 360 may include a decoded picture buffer (DPB) and may be configured by a digital storage medium. The hardware component may further include the memory 360 as an internal / external component.

[0093] When a bit stream including video / image information is input, the decoding device 300 may respond to the Figure 2 The image is reconstructed by a process of processing video / image information in an encoding device. For example, the decoding device 300 can derive a unit / block based on relevant information of block segmentation obtained from a bitstream. The decoding device 300 can perform decoding by using a processing unit applied in the encoding device. Therefore, the processing unit of decoding can be a coding unit, and the coding unit can be divided from a coding tree unit or a maximum coding unit according to a quadtree structure, a binary tree structure and / or a ternary tree structure. At least one transform unit can be derived from the coding unit. And, the reconstructed image signal decoded and output by the decoding device 300 can be played by a playback device.

[0094] The decoding device 300 may receive the bit stream from Figure 2 The signal output by the encoding device of the video image processing apparatus 310, and the received signal can be decoded by the entropy decoder 310. For example, the entropy decoder 310 can parse the bitstream to derive information (e.g., video / image information) necessary for image reconstruction (or picture reconstruction). The video / image information may further include information about various parameter sets such as an adaptive parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video / image information may further include general constraint information. The decoding device may further decode the picture based on the information about the parameter set and / or the general constraint information. The information sent / received by the signal and / or the syntax elements described later in this document may be decoded and obtained from the bitstream by a decoding process. For example, the entropy decoder 310 may decode the information in the bitstream based on a coding method such as exponential Golomb coding, CAVLC, CABAC, etc., and output the values ​​of the syntax elements necessary for image reconstruction and the quantized values ​​of the transform coefficients of the residual. In more detail, the CABAC entropy decoding method can receive a bin corresponding to each syntax element from a bitstream, determine a context model by using information of syntax elements to be decoded, information of decoded neighboring blocks and blocks to be decoded, or information of symbols / bins decoded in the previous step, perform arithmetic decoding on the bins by predicting the probability of occurrence of the bins according to the determined context model, and generate symbols corresponding to the values ​​of each syntax element. In this case, after determining the context model, the CABAC entropy decoding method can update the context model by using information about the decoded symbols / bins of the context model for the next symbol / bin. Among the information decoded in the entropy decoder 310, information about prediction is provided to the predictor (inter-frame predictor 332 and intra-frame predictor 331), and the residual value, i.e., the quantized transform coefficient and related parameter information, which is entropy decoded in the entropy decoder 310, can be input to the residual processor 320. The residual processor 320 can derive a residual signal (residual block, residual sample, residual sample array). In addition, information about filtering among the information decoded in the entropy decoder 310 can be provided to the filter 350. Meanwhile, a receiving unit (not shown) that receives a signal output from the encoding device may be further configured as an internal / external element of the decoding device 300 or the receiving unit may be a component of the entropy decoder 310 .

[0095] In addition, the decoding device according to this specification may be referred to as a video / image / picture decoding device, and the decoding device may be divided into an information decoder (video / image / picture information decoder) and a sample decoder (video / image / picture sample decoder). The information decoder may include an entropy decoder 310, and the sample decoder may include at least one of an inverse quantizer 321, an inverse transformer 322, an adder 340, a filter 350, a memory 360, an inter-frame predictor 332, and an intra-frame predictor 331.

[0096] The inverse quantizer 321 may inverse quantize the quantized transform coefficient and output the transform coefficient. The inverse quantizer 321 may rearrange the quantized transform coefficient into a two-dimensional block form. In this case, the rearrangement may be performed based on the coefficient scanning order performed in the encoding device. The inverse quantizer 321 may perform inverse quantization on the quantized transform coefficient by using a quantization parameter (e.g., quantization step size information) and obtain the transform coefficient.

[0097] The inverse transformer 322 inversely transforms the transform coefficients to obtain a residual signal (residual block, residual sample array).

[0098] The predictor 320 may perform prediction on the current block and generate a prediction block including prediction samples for the current block. The predictor 320 may determine whether to apply intra prediction or inter prediction to the current block based on the information on prediction output from the entropy decoder 310, and determine a specific intra / inter prediction mode.

[0099] The predictor 320 can generate a prediction signal based on various prediction methods described later. For example, the predictor 320 can not only apply intra prediction or inter prediction to predict a block, but also apply intra prediction and inter prediction at the same time. It can be called a combined inter and intra prediction (CIIP) mode. In addition, the predictor can be based on an intra block copy (IBC) prediction mode or can be based on a palette mode for prediction of a block. The IBC prediction mode or the palette mode can be used for content image / video coding of games, such as screen content coding (SCC), etc. IBC basically performs prediction within the current picture, but it can be performed similarly to inter prediction because it derives a reference block within the current picture. In other words, IBC can use at least one of the inter prediction techniques described herein. The palette mode can be considered as an example of intra coding or intra prediction. When the palette mode is applied, information about the palette table and the palette index can be included in the video / image information and sent with a signal.

[0100] The intra-frame predictor 331 can predict the current block by referring to samples within the current picture. Depending on the prediction mode, the referenced sample can be located near the current block or can be located at a specific distance away from the current block. In intra-frame prediction, the prediction mode can include at least one non-directional mode and multiple directional modes. The intra-frame predictor 331 can determine the prediction mode applied to the current block by using the prediction mode applied to the neighboring block.

[0101] The inter-frame predictor 332 may derive a prediction block for the current block based on a reference block (reference sample array) specified by a motion vector on a reference picture. In this case, in order to reduce the amount of motion information sent in the inter-frame prediction mode, the motion information may be predicted in units of blocks, sub-blocks, or samples based on the correlation of the motion information between the neighboring blocks and the current block. The motion information may include a motion vector and a reference picture index. The motion information may further include inter-frame prediction direction information (L0 prediction, L1 prediction, Bi prediction, etc.). For inter-frame prediction, the neighboring blocks may include spatial neighboring blocks present in the current picture and temporal neighboring blocks present in the reference picture. For example, the inter-frame predictor 332 may configure a motion information candidate list based on the neighboring blocks, and derive a motion vector and / or a reference picture index for the current block based on the received candidate selection information. Inter-frame prediction may be performed based on various prediction modes, and information about the prediction may include information indicating an inter-frame prediction mode for the current block.

[0102] The adder 340 may add the obtained residual signal to the prediction signal (prediction block, prediction sample array) output from the predictor (including the inter-frame predictor 332 and / or the intra-frame predictor 331) to generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array). When there is no residual of the block to be processed, such as when the skip mode is applied, the prediction block may be used as the reconstructed block.

[0103] The adder 340 may be referred to as a reconstructor or a reconstructed block generator. The generated reconstructed signal may be used for intra prediction of the next block to be processed in the current picture, may be output by filtering described later, or may be used for inter prediction of the next picture. At the same time, luminance mapping with chroma scaling (LMCS) may be applied during picture decoding.

[0104] The filter 350 can improve the subjective / objective image quality by applying filtering to the reconstructed signal. For example, the filter 350 can generate a modified reconstructed picture by applying various filtering methods to the reconstructed picture, and send the modified reconstructed picture to the memory 360, specifically the DPB of the memory 360. The various filtering methods may include deblocking filtering, sample adaptive offset, adaptive loop filter, bilateral filter, etc.

[0105] The (modified) reconstructed picture stored in the DPB of the memory 360 can be used as a reference picture in the inter-frame predictor 332. The memory 360 can be derived from the motion information in its current picture (or decoded) The motion information of the block and / or the motion information of the block in the pre-reconstructed picture. The stored motion information can be sent to the inter-frame predictor 332 to be used as the motion information of the spatial neighboring block or the motion information of the temporal neighboring block. The memory 360 can store the reconstructed samples of the reconstructed block in the current picture and send them to the intra-frame predictor 331.

[0106] Here, the embodiments described in the filter 260, the inter-frame predictor 221, and the intra-frame predictor 222 of the encoding device 200 may also be equally or correspondingly applied to the filter 350, the inter-frame predictor 332, and the intra-frame predictor 331 of the decoding device 300, respectively.

[0107] Figure 4 is a diagram illustrating an intra prediction method performed by the decoding device 300 according to an embodiment of the present disclosure.

[0108] An embodiment of the present disclosure provides a method for deriving a linear model based on neighboring reconstructed samples of a current block and performing intra-frame prediction using the derived linear model. When describing the embodiments of the present disclosure, the intra-frame prediction method using a linear model is referred to as linear model prediction, but is of course not limited to this name. In the present disclosure, linear model prediction may also be referred to as linear model intra-frame prediction, linear prediction, linear intra-frame prediction, model prediction, model intra-frame prediction, linear model-based prediction, linear model-based intra-frame prediction, position-based linear prediction, position-based linear model prediction, position-based linear model intra-frame prediction, etc.

[0109] Reference Figure 4 , the decoding device may construct reference samples for linear model intra prediction based on neighboring reconstructed samples of the current block (S400), and derive linear model parameters (or variables) based on the constructed reference samples (S410). In other words, the decoding device may construct reference samples for linear model intra prediction based on neighboring reconstructed samples of the current block, and derive (or determine) the linear model based on the constructed reference samples. The linear model for linear model intra prediction may be derived based on the linear model parameters derived in operation S410. Hereinafter, various embodiments of the intra prediction method based on the linear model will be described in detail.

[0110] According to an embodiment of the present disclosure, a linear model can be derived based on neighboring reconstructed samples of the current block. Specifically, linear model parameters can be derived using the values ​​and / or coordinate information (or position information) of the neighboring reconstructed samples, and the linear model can be determined based on the derived linear model parameters. In the present disclosure, a linear model may also be referred to as a linear formula or a linear equation. Assuming that the current block is N×N, the following equation 1 can be used to generate (or calculate and derive) a predicted sample at position (x, y). In other words, a linear model such as the following equation 1 can be defined, and a linear model such as the following equation 1 can be used to generate a predicted sample at position (x, y).

[0111] [Equation 1]

[0112] Pred (x,y) =ax+by+c

[0113] In the above equation 1, a, b and c represent linear model parameters (or variables). In the present disclosure, c is a constant, which may be referred to as a linear model parameter or an offset. The linear model may be determined based on the linear model parameters. In addition, a and b may be referred to as horizontal (horizontal component) parameters and vertical (vertical component) parameters, respectively. Because the linear model is determined based on the linear model parameters, the linear model parameters may be a linear model, or may be referred to simply as a linear model. In order to derive the parameters a, b and c of the above equation 1, the values ​​and coordinate information (or position information) of the samples that have been reconstructed around the current block may be used.

[0114] Specifically, an equation for minimizing the error between the predicted sample and the reconstructed sample may be derived by using an equation expressing the correlation between the reconstructed pixel value and the pixel coordinates through the following Equation 2. As described above, the term "sample" may be used to refer to a pixel in the present disclosure.

[0115] [Equation 2]

[0116]

[0117] In the above equation 2, L refers to the number of neighboring reconstructed samples. As an embodiment, L=4N+1, and as in the following embodiments, L may be defined as various other numbers. In this case, as shown in the above equation 1, since Pred=ax+by+c and the minimum value of the square error is 0, the following equation 3 may be derived by performing partial differentiation on a, b, and c with respect to the above equation 2.

[0118] [Equation 3]

[0119]

[0120] The above equation 3 can be organized into a matrix of a, b, c, as shown in the following equation 4.

[0121] [Equation 4]

[0122]

[0123] When the matrix According to Cramer's law, The solution of can be expressed as the following Equation 5.

[0124] [Equation 5]

[0125]

[0126] In equation 5 above, Matrix A can be calculated using only coordinate information. In addition, matrix B 0 , B 1 and B 2 It can be calculated using coordinate information and reconstructed sample values ​​of neighboring blocks.

[0127] As an example, assuming that 1) the current block is a 4×4 block, 2) the value of the upper left reference sample adjacent to the upper left sample (or corner) of the current block is 533, 3) the reference sample values ​​included in the upper reference sample line adjacent to the right side of the upper left reference sample are {514, 487, 456, 470, 535, 533, 527, 525} in order from the left, and 4) the reference sample values ​​included in the left reference sample line adjacent to the bottom of the upper left reference sample are {540, 528, 504, 489, 489, 489, 489, 489} in order from the top. In this case, the linear model parameters a, b, and c can be derived as shown in Equation 6 below.

[0128] [Equation 6]

[0129]

[0130] Thereafter, the linear model according to the above equation 1 may be determined based on the calculated parameter values ​​a, b, and c. As an example, the values ​​of the linear model parameters calculated based on the reconstructed reference samples according to an embodiment of the present disclosure may be values ​​approximated by methods such as scaling, shift operations, fixed-point operations, and integer operations. Based on the derived linear model, the predicted samples may be calculated based on the positions of the samples in the current block. As an example, the predicted samples (or predicted blocks) of the 4×4 current block may be determined as shown in Table 1 below.

[0131] [Table 1]

[0132]

[0133]

[0134] Referring to Table 1, the prediction sample of the current sample can be calculated based on the linear model to be derived and the position of the current sample in the current block. In other words, the prediction sample can be calculated sample by sample (ie, pixel by pixel) based on the derived linear model and the sample position.

[0135] In the above embodiment, a linear model is defined based on the values ​​and coordinate information of the surrounding reconstructed samples, and a parameter for the linear model is obtained according to the defined linear model to generate a prediction block. In this embodiment, a method for generating a prediction block by using linear model parameters derived by transformation such as weights is proposed.

[0136] Referring back to equation 1 above, parameter a is a parameter for the x-axis, and b is a parameter for the y-axis. In the present disclosure, a may be referred to as a horizontal (component) parameter and b may be referred to as a vertical (component) parameter. Alternatively, in the present disclosure, a may be a linear model parameter representing the amount of change in the horizontal direction, and b may be a linear model parameter representing the amount of change in the vertical direction.

[0137] According to an embodiment of the present disclosure, any of the methods previously described in Equations 1 to 6 may be used to derive a linear model according to Equation 1, and a predetermined weight (or variable) may be applied to the derived linear model parameters. In one example, the derived linear model parameters of Equation 1 may be weighted as shown in Equation 7 below. In other words, a final prediction sample may be derived (or generated) based on a model such as Equation 7.

[0138] [Equation 7]

[0139] WeightPred (x,y) =W 0 ax+W 1 by+c

[0140] Referring to Equation 7, the weight W 0 and W 1 Can have value 0 <W 0 ≤1 and 0 <W 1 ≤1. Here, W 0 and W 1 May be determined to be equal to each other or may be defined as different values. In Equation 7, the weights applied to the linear model parameters may be determined based on the predefined encoding information. In this case, the predefined encoding information may be at least one of the width, height, size, shape, and width-to-height ratio of the current block.

[0141] In one embodiment, if the current block is non-rectangular, i.e., the current block has a width different from the height, there may be a difference between the adjacent reconstructed sample data reflected in the horizontal parameter and the adjacent reconstructed sample data reflected in the vertical parameter in the linear model according to Equation 1. Therefore, by determining and applying weights that take into account the difference, various prediction samples can be used for prediction to improve the accuracy of the prediction.

[0142] In addition, in one embodiment, it can be determined that W 0 =1 / 2, W 1 = 1. Alternatively, W can be determined 0 =1,W 1 =1 / 2. For example, according to the shape of the current block, W can be determined 0 =1 / 2, W 1 =1, or we can determine W 0 =1,W 1 =1 / 2. Alternatively, W can be determined 0 =1 / width, W 1 =1. Here, width represents the width of the current block. Alternatively, it can be determined as W 0 =1,W 1 =1 / height. Here, the height represents the height of the current block. In one example, the prediction block generated using the linear model of Equation 7 may be determined as the final prediction block.

[0143] Alternatively, in another example, a final prediction block (or final prediction sample) may be generated by applying a weighted sum to a prediction block generated based on a model of Equation 1 and a prediction block generated based on a model of Equation 7. In the present disclosure, two prediction blocks generated using each model may be referred to as initial prediction blocks or temporary prediction blocks.

[0144] In one example, a commitment weight between the encoder and the decoder may be used as a mixing ratio (ie, weight). Alternatively, a weight signaled from the encoder to the decoder may be used. Alternatively, the weight may be variably determined based on a prediction mode of an adjacent block (e.g., intra / inter prediction information, whether a linear model is used, or whether a planar mode is used).

[0145] Alternatively, in one example, the prediction sample may be generated using Equation 7, and the generated prediction block may be mixed (or weighted summed) at a specific ratio with the prediction block generated by Equation 1 to derive the final prediction sample. For example, the final prediction block may be calculated in the form of FinalPred=(3*WeightPred+Pred) / 4, where Pred is the prediction block generated by Equation 1.

[0146] Alternatively, the prediction block generated using Equation 7 can be derived as a final prediction sample by mixing it with the prediction block generated in the planar mode at a specific ratio. For example, the final prediction block can have the form of FinalPred=(3*WeightPred+Planar) / 4, where Planar represents the prediction block generated in the planar mode.

[0147] According to one embodiment of the present disclosure, a method for deriving parameters by partially utilizing or transforming reconstructed pixels around a current block is provided. In the linear model of equation 1 for constructing a prediction block, parameters a and b may refer to the amount of change along the x-axis (or the amount of change in the horizontal direction) and the amount of change along the y-axis (or the amount of change in the vertical direction), respectively. In this embodiment, it is proposed to use the derived adjacent reference samples for parameter calculation in order to minimize the amount of change along one axis (or in one direction) or to emphasize one axis relative to another axis.

[0148] As an embodiment, at least one of the left reference sample or the upper reference sample can be set to a specific value for the export, and the linear model parameters can be exported. This allows limiting the amount of change in a specific direction when exporting the linear model.

[0149] For example, the upper reference sample may be used, but only the derived specific sample may be used for the left reference sample. In this way, when the linear model is derived, the amount of change along the x-axis may be limited. In this case, the specific sample may be the Nth sample of the left reference sample. For example, the left reference samples may all be set to the value of the first reference sample or the last reference sample of the left reference sample. The order of the reference samples may be determined by the order of being adjacent to the upper left reference sample of the current block.

[0150] Alternatively, the specific sample may be the average value of the left reference samples. Alternatively, the specific sample may be the minimum value or the maximum value of the left reference samples. Alternatively, the specific sample may be the median value of the left reference samples. Alternatively, the specific sample may be the Nth sample of the upper reference samples. Alternatively, the specific sample may be the average value of the upper reference samples. Alternatively, the specific sample may be the minimum value or the maximum value of the upper reference samples. Alternatively, the specific sample may be the median value of the upper reference samples.

[0151] When using the specific samples derived by the aforementioned method, the upper reference sample can be used as the reconstruction value, while the reference sample smoothing can be applied to the left reference sample. The reference sample smoothing can be performed by applying a predefined n-tap filter. Here, n can be defined as 2, 3 or 5. Alternatively, when using the specific samples derived by the aforementioned method, the reference sample smoothing can be applied to the upper reference sample, while the reference sample smoothing may not be applied to the left reference sample.

[0152] For example, the left reference sample can be used as is, and only the derived specific sample can be used for the upper reference sample. By deriving the linear model in this way, the amount of change along the y-axis can be limited. In this case, the specific sample can be the Nth sample of the upper reference sample. For example, all upper reference samples can be set to the first reference sample value or the last reference sample value of the upper reference sample. The order of the reference samples can be determined in the order of being adjacent to the upper left reference sample of the current block.

[0153] Alternatively, the specific sample may be the average value of the upper reference sample. The specific sample may also be the minimum value or maximum value of the upper reference sample. In another case, the specific sample may be the median value of the upper reference sample. Alternatively, the specific sample may be the Nth sample of the left reference sample. The specific sample may also be the average value, minimum value, maximum value or median value of the left reference sample.

[0154] When using the specific samples derived by the aforementioned method, the upper reference sample can be used as the reconstruction value, while the reference sample smoothing can be applied to the left reference sample. The reference sample smoothing can be performed by applying a predefined n-tap filter. Alternatively, when using the specific samples derived by the aforementioned method, the reference sample smoothing can be applied to the upper reference sample, while the reference sample smoothing may not be applied to the left reference sample.

[0155] According to the aforementioned method, the left reference sample or the upper reference sample can be set to a specific value for the export, and the linear model parameters can be derived based on this.

[0156] In addition, according to an embodiment of the present disclosure, when deriving a linear model, a method is proposed that not only utilizes (or refers to) the reference sample line closest to the current block among the reconstructed area around the current block, but also utilizes (or refers to) a wider area (or reference sample line). In other words, multiple reference lines including neighboring samples that are not adjacent to the current block can be used for linear model derivation. Since the embodiment uses coordinate information and pixel values, the same method can be applied even when multiple reference lines are used.

[0157] For example, in the embodiments described with reference to Equations 1 to 6, the focus is on deriving linear model parameters using the first reference sample line closest to the current block. However, if the coordinate information and sample values ​​are valid, the parameters can be derived by using the values ​​of reference samples located at a relatively far distance. In other words, when the line offset of adjacent reconstructed reference samples is expressed as an offset, reference samples with an offset > 1 can be used for parameter derivation. In the present disclosure, the line offset of a reference sample may also be referred to as a reference sample line index or a reference line index. The line offset may be obtained (or derived) by explicitly signaling from the encoder to the decoder or by a predefined agreement between the encoder and the decoder. When the upper left coordinate of the current block is defined as (0,0), the reference sample for parameter derivation may be defined as shown in Equation 8 below.

[0158] [Equation 8]

[0159] Ref Above =Recx ,y , where 0≤x<2*width+offset above -1 and -offset Above ≤y≤-1

[0160] Ref Left =Rec x,y Among them, -offset Left ≤x≤-1 and 0≤y≤2*height+offset left -1

[0161] Referring to Equation 8, the linear model parameters can be derived by using the reference samples included in the reference sample line determined by the offset. As an embodiment, the offset may be applied differently to the upper reference sample and the left reference sample, or may be applied with the same value. Alternatively, the parameters may be derived by using only a portion of the reference samples defined above. When defining the range of the reference samples proposed in this embodiment, Offset (Left,Above) can be an integer greater than 0. For example, in the embodiments described with reference to Equations 1 to 6, by replacing offset Left =1 and offset Above =1 to define a reference sample. As another example, a reference sample whose length corresponds to the block size may be used. The equation for this may be defined as shown in Equation 9 below.

[0162] [Equation 9]

[0163] Ref Above =Rec x,y , where 0≤x<width and -offset Above ≤y≤-1

[0164] Ref Left =Rec x,y Among them, -offset Left ≤x≤-1 and 0≤y<height

[0165] Refer to Equation 9, Offset (Left,Above) It may be an integer greater than 0. In addition, as another example, for odd offsets, only reference samples whose x coordinates (i.e., horizontal coordinates) of the left reference sample are odd or whose y coordinates (i.e., vertical coordinates) of the upper reference sample are odd may be used. For even offsets, only reference samples whose x coordinates of the left reference sample are even or whose y coordinates of the upper reference sample are even may be used. Alternatively, the above examples may be combined and used.

[0166] The decoding device may generate a prediction sample for the current block based on the linear model parameters (S420). In other words, the decoding device may perform intra prediction for the current block based on the derived linear model parameters (or linear model). The prediction sample may be generated on a per-sample basis according to the position of the current sample within the current block based on the derived linear model.

[0167] Figure 5 A schematic configuration of the intra predictor 331 performing the intra prediction method according to the present disclosure is illustrated.

[0168] Reference Figure 5 , the intra predictor 331 may include a reference sample configurator 500 , a linear model parameter deriver 510 , and a prediction sample generator 520 .

[0169] Specifically, the reference sample configurator 500 may configure reference samples for linear model intra prediction based on reconstructed neighboring samples of the current block. The linear model parameter deriver 510 may derive linear model parameters (or variables) based on the configured reference samples. In other words, the reference sample configurator 500 may configure reference samples for linear model intra prediction based on reconstructed neighboring samples of the current block, and the linear model parameter deriver 510 may derive (or determine) a linear model based on the configured reference samples. A linear model for linear model intra prediction may be derived according to the linear model parameters derived by the linear model parameter deriver 510. Previously referred to Figure 4 Various embodiments of the linear model-based intra prediction method are described, and redundant descriptions will be omitted.

[0170] As described above, the linear model parameter deriver 510 can derive a linear model based on the reconstructed samples around the current block. Specifically, the linear model parameter deriver 510 can derive linear model parameters using the values ​​and / or coordinate information (or position information) of the adjacent reconstructed samples, and can determine the linear model based on the derived linear model parameters. At this time, a linear model as described in Equation 1 can be defined, and the linear model described in Equation 1 can be used to generate a prediction sample at a (x, y) position.

[0171] As described above, the linear model parameter deriver 510 can modify the derived linear model parameters, such as by applying weights, to generate a prediction block. The linear model parameter deriver 510 can derive a linear model according to Equation 1 by the method described with reference to Equations 1 to 6, and apply predetermined weights (or variables) to the derived linear model parameters. For example, the weights described in Equation 7 can be applied to the derived linear model parameters of Equation 1. Redundant descriptions are omitted.

[0172] The linear model according to Equation 1 can change its linear model parameter values ​​depending on which reference samples among the neighboring reference samples of the current block are used. Therefore, as described above, the linear model parameter deriver 510 can derive parameters by only utilizing or modifying some of the already reconstructed pixels around the current block. The linear model parameter deriver 510 can set at least one of the left reference sample or the upper reference sample to a specific value for deriving and derive the linear model parameters.

[0173] For example, the upper reference sample can be used as is, and only the derived specific sample can be used for the left reference sample. By deriving the linear model in this way, the amount of change along the x-axis can be limited. At this time, the specific sample can be the Nth sample of the left reference sample. For example, all left reference samples can be set to the first reference sample value or the last reference sample value of the left reference sample. The order of the reference samples can be determined in the order of being adjacent to the upper left reference sample of the current block.

[0174] Alternatively, the specific sample may be the average value of the left reference sample. Alternatively, the specific sample may also be the minimum value or maximum value of the left reference sample. Alternatively, the specific sample may be the median value of the left reference sample. Alternatively, the specific sample may also be the Nth sample of the upper reference sample. Alternatively, the specific sample may be the average value, minimum value, maximum value or median value of the upper reference sample.

[0175] When using the specific samples derived by the aforementioned method, the upper reference sample can be used as the reconstruction value, while the reference sample smoothing can be applied to the left reference sample. The reference sample smoothing can be performed by applying a predefined n-tap filter. Alternatively, when using the specific samples derived by the aforementioned method, the reference sample smoothing can be applied to the upper reference sample, while the reference sample smoothing may not be applied to the left reference sample.

[0176] In addition, as described above, multiple reference lines including neighboring samples that are not adjacent to the current block can be used for linear model derivation. Figure 4 The described embodiment uses coordinate information and pixel values, so the same method can be applied even when multiple reference lines are used.

[0177] The prediction sample generator 520 may generate a prediction sample for the current block based on the linear model parameters. In other words, the prediction sample generator 520 may perform intra prediction for the current block based on the derived linear model parameters (or linear model). The prediction sample may be generated on a per-sample basis according to the position of the current sample within the current block based on the derived linear model.

[0178] Figure 6 An example of an inter prediction method performed by the encoding apparatus 200 according to an embodiment of the present disclosure is illustrated.

[0179] In one embodiment of the present disclosure, a linear model intra prediction method performed by an encoding device is described. The encoding device may derive a linear model based on reconstructed samples around a current block and use the derived linear model to perform intra prediction. Figure 4 and Figure 5 The embodiments described with respect to linear model intra prediction can be applied in substantially the same manner, and redundant descriptions are omitted.

[0180] Reference Figure 6 , the encoding device may configure reference samples for linear model intra prediction based on the reconstructed neighboring samples of the current block (S600), and derive linear model parameters (or variables) based on the configured reference samples (S610). In other words, the encoding device may configure reference samples for linear model intra prediction based on the reconstructed neighboring samples of the current block and derive (or determine) a linear model based on the configured reference samples. The linear model for linear model intra prediction may be derived according to the linear model parameters derived in step S610.

[0181] The encoding device may derive a linear model based on the reconstructed samples around the current block. Specifically, the encoding device may derive linear model parameters using the values ​​and / or coordinate information (or position information) of the adjacent reconstructed samples, and may determine the linear model based on the derived linear model parameters. At this point, a linear model such as the linear model described in Equation 1 may be defined, and the linear model described in Equation 1 may be used to generate a predicted sample at the (x, y) position.

[0182] As described above, the encoding device may generate a prediction block by modifying the derived linear model parameters using weights, etc. The encoding device may derive a linear model according to Equation 1 by the method described with reference to Equations 1 to 6, and apply a predetermined weight (or variable) to the derived linear model parameters. For example, a weight such as the weight described in Equation 7 may be applied to the derived linear model parameters of Equation 1. Redundant descriptions are omitted.

[0183] The linear model according to Equation 1 can change its linear model parameter value depending on which reference samples among the neighboring reference samples of the current block are used. Therefore, as described above, the encoding device can derive parameters by only utilizing or modifying some of the already reconstructed pixels around the current block. The encoding device can set at least one of the left reference sample or the upper reference sample to a specific value for deriving and derive the linear model parameter.

[0184] For example, the upper reference sample can be used as is, and only the derived specific sample can be used for the left reference sample. By deriving the linear model in this way, the amount of change along the x-axis can be limited. At this time, the specific sample can be the Nth sample of the left reference sample. For example, all left reference samples can be set to the first reference sample value or the last reference sample value of the left reference sample. The order of the reference samples can be determined in the order of being adjacent to the upper left reference sample of the current block.

[0185] Alternatively, the specific sample may be the average value of the left reference sample. Alternatively, the specific sample may also be the minimum value or maximum value of the left reference sample. Alternatively, the specific sample may be the median value of the left reference sample. Alternatively, the specific sample may also be the Nth sample of the upper reference sample. Alternatively, the specific sample may be the average value, minimum value, maximum value or median value of the upper reference sample.

[0186] When using the specific samples derived by the aforementioned method, the upper reference sample can be used as the reconstruction value, while reference sample smoothing can be applied to the left reference sample. Reference sample smoothing can be performed by applying a predefined n-tap filter. Alternatively, when using the specific samples derived by the aforementioned method, reference sample smoothing can be applied to the upper reference sample, while reference sample smoothing may not be applied to the left reference sample.

[0187] In addition, as described above, multiple reference lines including neighboring samples that are not adjacent to the current block can be used for linear model derivation. Figure 4 The described embodiment uses coordinate information and pixel values, so the same method can be applied even when multiple reference lines are used.

[0188] The encoding device may generate a prediction sample for the current block based on the linear model parameters (S620). In other words, the encoding device may perform intra prediction for the current block based on the derived linear model parameters (or linear model). The prediction sample may be generated on a per-sample basis according to the position of the current sample within the current block based on the derived linear model.

[0189] Figure 7 A schematic configuration of the intra predictor 222 that performs the intra prediction method according to the present disclosure is illustrated.

[0190] Reference Figure 7 , the intra predictor 222 may include a reference sample configurator 700 , a linear model parameter deriver 710 , and a prediction sample generator 720 .

[0191] Specifically, the reference sample configurator 700 may configure reference samples for linear model intra prediction based on reconstructed neighboring samples of the current block. The linear model parameter deriver 710 may derive linear model parameters (or variables) based on the configured reference samples. In other words, the reference sample configurator 700 may configure reference samples for linear model intra prediction based on reconstructed neighboring samples of the current block, and the linear model parameter deriver 710 may derive (or determine) a linear model based on the configured reference samples. A linear model for linear model intra prediction may be derived according to the linear model parameters derived by the linear model parameter deriver 710. Previously referred to Figure 4 Various embodiments of the linear model-based intra prediction method are described, and redundant descriptions will be omitted.

[0192] As described above, the linear model parameter deriver 710 can derive a linear model based on the reconstructed samples around the current block. Specifically, the linear model parameter deriver 710 can derive linear model parameters using the values ​​and / or coordinate information (or position information) of the adjacent reconstructed samples, and can determine the linear model based on the derived linear model parameters. At this time, a linear model such as the linear model described in Equation 1 can be defined, and the linear model described in Equation 1 can be used to generate a prediction sample at the (x, y) position.

[0193] As described above, the linear model parameter deriver 710 can generate a prediction block by modifying the derived linear model parameters using weights, etc. The linear model parameter deriver 710 can derive a linear model according to Equation 1 by the method described with reference to Equations 1 to 6, and apply predetermined weights (or variables) to the derived linear model parameters. For example, weights such as those described in Equation 7 can be applied to the derived linear model parameters of Equation 1. Redundant descriptions are omitted.

[0194] The linear model according to Equation 1 can change its linear model parameter values ​​depending on which reference samples among the neighboring reference samples of the current block are used. Therefore, as described above, the linear model parameter deriver 710 can derive parameters by only utilizing or modifying some of the already reconstructed pixels around the current block. The linear model parameter deriver 710 can set at least one of the left reference sample or the upper reference sample to derive a specific value and derive the linear model parameters.

[0195] For example, the upper reference sample can be used as is, and only the derived specific sample can be used for the left reference sample. By deriving the linear model in this way, the amount of change along the x-axis can be limited. At this time, the specific sample can be the Nth sample of the left reference sample. For example, all left reference samples can be set to the first reference sample value or the last reference sample value of the left reference sample. The order of the reference samples can be determined in the order of being adjacent to the upper left reference sample of the current block.

[0196] Alternatively, the specific sample may be the average value of the left reference sample. Alternatively, the specific sample may also be the minimum value or maximum value of the left reference sample. Alternatively, the specific sample may be the median value of the left reference sample. Alternatively, the specific sample may also be the Nth sample of the upper reference sample. Alternatively, the specific sample may be the average value, minimum value, maximum value or median value of the upper reference sample.

[0197] When using the specific samples derived by the aforementioned method, the upper reference sample can be used as the reconstruction value, while reference sample smoothing can be applied to the left reference sample. Reference sample smoothing can be performed by applying a predefined n-tap filter. Alternatively, when using the specific samples derived by the aforementioned method, reference sample smoothing can be applied to the upper reference sample, while reference sample smoothing may not be applied to the left reference sample.

[0198] In addition, as described above, multiple reference lines including neighboring samples that are not adjacent to the current block can be used for linear model derivation. Figure 4 The described embodiment uses coordinate information and pixel values, so the same method can be applied even when multiple reference lines are used.

[0199] The prediction sample generator 720 may generate a prediction sample for the current block based on the linear model parameters. In other words, the prediction sample generator 720 may perform intra prediction for the current block based on the derived linear model parameters (or linear model). The prediction sample may be generated on a per-sample basis according to the position of the current sample within the current block based on the derived linear model.

[0200] In the above embodiments, the method is described as a series of steps or boxes based on the flowchart, but the corresponding embodiments are not limited to the order of the steps, and some steps may occur simultaneously or in a different order than other steps described above. In addition, those skilled in the art will understand that the steps shown in the flowchart are not exclusive, and other steps may be included or one or more steps in the flowchart may be deleted without affecting the scope of the embodiments of the present disclosure.

[0201] The above-mentioned method according to the embodiment of the present disclosure can be implemented in the form of software, and the encoding device and / or decoding device according to the present disclosure can be included in a device that performs image processing, such as a TV, a computer, a smart phone, a set-top box, a display device, etc.

[0202] In the present disclosure, when the embodiment is implemented as software, the above method can be implemented as a module (process, function, etc.) that performs the above functions. The module can be stored in a memory and can be executed by a processor. The memory can be located inside or outside the processor and can be connected to the processor by various well-known means. The processor may include an application-specific integrated circuit (ASIC), another chipset, a logic circuit, and / or a data processing device. The memory may include a read-only memory (ROM), a random access memory (RAM), a flash memory, a memory card, a storage medium, and / or other storage devices. In other words, the embodiments described herein can be implemented on a processor, a microprocessor, a controller, or a chip to perform. For example, the functional unit shown in each of the accompanying drawings can be implemented on a computer, a processor, a microprocessor, a controller, or a chip to perform. In this case, information (e.g., information about instructions) or an algorithm for implementation can be stored in a digital storage medium.

[0203] In addition, the decoding device and encoding device of the embodiment of the present disclosure can be included in multimedia broadcast sending and receiving devices, mobile communication terminals, home theater video devices, digital theater video devices, surveillance cameras, video conversation devices, real-time communication devices such as video communication, mobile streaming devices, storage media, cameras, devices for providing video on demand (VoD) services, over-the-top video (OTT) devices, devices for providing Internet streaming services, three-dimensional (3D) video devices, virtual reality (VR) devices, augmented reality (AR) devices, videophone video devices, transportation terminals (e.g., vehicle (including autonomous driving vehicle) terminals, aircraft terminals, ship terminals, etc.) and medical video devices, etc., and can be used to process video signals or data signals. For example, over-the-top video (OTT) devices can include game consoles, Blu-ray players, networked TVs, home theater systems, smart phones, tablet computers, digital video recorders (DVRs), etc.

[0204] In addition, the processing method of the embodiment of the present disclosure can be generated in the form of a program executed by a computer and can be stored in a computer-readable recording medium. Multimedia data with a data structure according to an embodiment of the present disclosure can also be stored in a computer-readable recording medium. Computer-readable recording media include all types of storage devices and distributed storage devices that store computer-readable data. Computer-readable recording media may include, for example, Blu-ray discs (BD), universal serial buses (USB), ROMs, PROMs, EPROMs, EEPROMs, RAMs, CD-ROMs, magnetic tapes, floppy disks, and optical media storage devices. In addition, computer-readable recording media include media implemented in the form of carrier waves (e.g., transmitted via the Internet). In addition, the bit stream generated by the encoding method may be stored in a computer-readable recording medium or may be sent via a wired or wireless communication network.

[0205] In addition, the embodiments of the present disclosure may be implemented by a computer program product through a program code, and the program code may be executed on a computer by the embodiments of the present disclosure. The program code may be stored on a computer-readable carrier.

[0206] Figure 8 is an example of a content streaming system to which embodiments of the present disclosure can be applied.

[0207] Reference Figure 8 The content streaming transmission system to which the embodiments of the present disclosure are applied may mainly include an encoding server, a streaming transmission server, a web server, a media storage, a user device, and a multimedia input device.

[0208] The encoding server generates a bitstream by compressing content input from a multimedia input device such as a smartphone, a camera, a camcorder, etc. into digital data and transmits it to the streaming server. As another example, when a multimedia input device such as a smartphone, a camera, a camcorder, etc. directly generates a bitstream, the encoding server may be omitted.

[0209] A bitstream may be generated by applying the encoding method or the bitstream generating method of the embodiment of the present disclosure, and the streaming server may temporarily store the bitstream in the process of transmitting or receiving the bitstream.

[0210] The streaming server sends multimedia data to the user device through the web server based on the user's request, and the web server serves as a medium to inform the user what services are available. When the user requests the required service from the web server, the web server delivers it to the streaming server, and the streaming server sends the multimedia data to the user. In this case, the content streaming system may include a separate control server, and in this case, the control server controls the command / response between each device in the content streaming system.

[0211] The streaming server may receive content from a media storage and / or encoding server. For example, when receiving content from an encoding server, the content may be received in real time. In this case, in order to provide a smooth streaming service, the streaming server may store the bitstream for a certain period of time.

[0212] Examples of user devices may include mobile phones, smart phones, laptop computers, digital broadcast terminals, personal digital assistants (PDAs), portable multimedia players (PMPs), navigation, tablet PCs, tablet computers, ultrabooks, wearable devices (e.g., smart watches, smart glasses, head-mounted displays (HMDs), digital televisions, desktops, digital signage, etc.).

[0213] Each server in the content streaming system may be operated as a distributed server, and in this case, data received from each server may be distributed and processed.

[0214] The claims set forth herein can be combined in various ways. For example, the technical features of the method claims of the present disclosure can be combined and implemented as a device, and the technical features of the device claims of the present disclosure can be combined and implemented as a method. In addition, the technical features of the method claims of the present disclosure and the technical features of the device claims can be combined and implemented as a device, and the technical features of the method claims of the present disclosure and the technical features of the device claims can be combined and implemented as a method.

Claims

1. A method for decoding an image, the method comprising the following steps: Configure reference samples for linear model intra prediction based on reconstructed neighboring samples of the current block; deriving linear model parameters based on the reference sample, the linear model parameters comprising at least one of a first linear model parameter related to a variance in a horizontal direction or a second linear model parameter related to a variance in a vertical direction; as well as Generate prediction samples of the current block based on the linear model parameters.

2. The image decoding method according to claim 1, wherein: The reference sample includes at least one of the following items: a reference sample included in a left sample line adjacent to the current block, a reference sample included in an upper sample line adjacent to the current block, or an upper-left reference sample adjacent to the current block.

3. The image decoding method according to claim 1, wherein: The step of deriving the linear model parameters comprises the following steps: deriving the first linear model parameter and the second linear model parameter based on the reference sample; and A weight is applied to at least one of the first linear model parameters or the second linear model parameters.

4. The image decoding method according to claim 3, wherein: The weights are determined based on predefined coding information.

5. The image decoding method according to claim 4, wherein: The predefined encoding information includes at least one of a width, a height, a size, a shape, or a ratio of the width to the height of the current block.

6. The image decoding method according to claim 2, wherein: The step of configuring the reference sample comprises the following steps: The reference sample included in one of the left sample line or the upper sample line is set to a predetermined value.

7. The image decoding method according to claim 6, wherein: The predetermined value is one of an average value, a maximum value, a minimum value, or a median value of the reference samples included in one of the left sample line or the upper sample line.

8. The image decoding method according to claim 6, wherein: The predetermined value is determined as a value of a first reference sample or a last reference sample based on the upper left reference sample among the reference samples included in one of the left sample line or the upper sample line.

9. The image decoding method according to claim 1, further comprising the following steps: obtaining a reference line offset indicating one of a plurality of reference sample lines of the current block, The reference sample is constructed using reference samples included in a reference sample line indicated by the reference line offset.

10. A method for encoding an image, comprising the following steps: Configure reference samples for linear model intra prediction based on reconstructed neighboring samples of the current block; deriving linear model parameters based on the reference sample, the linear model parameters comprising at least one of a first linear model parameter related to a variance in a horizontal direction or a second linear model parameter related to a variance in a vertical direction; as well as Generate prediction samples of the current block based on the linear model parameters. 11 . A computer-readable storage medium configured to store a bit stream generated by the image encoding method according to claim 10 .

12. A method for transmitting data of image information, the method comprising the following steps: Configure reference samples for linear model intra prediction based on reconstructed neighboring samples of the current block; deriving linear model parameters based on the reference sample, the linear model parameters comprising at least one of a first linear model parameter related to a variance in a horizontal direction or a second linear model parameter related to a variance in a vertical direction; Generate a prediction sample of the current block based on the linear model parameters; generating a bitstream by encoding the current block based on the prediction samples; as well as Data including the bit stream is transmitted.

Images