Image encoding / decoding method and apparatus and recording medium storing bitstream

Abstract

The image decoding method and apparatus according to the present disclosure can derive one or more most dominant intra prediction modes (MDMs) by applying a prescribed mask kernel to a peripheral region of a current block, generate a prediction block of the current block based on the one or more MDMs, and reconstruct the current block based on the prediction block of the current block.

Description

Technical Field

[0001] This disclosure relates to an image encoding / decoding method and apparatus, as well as a recording medium for storing bit streams. Background Technology

[0002] Recently, the demand for high-resolution and high-quality images, such as HD (high-definition) and UHD (ultra-high-definition) images, has been increasing in various application areas, and therefore, efficient image compression technologies are being discussed.

[0003] Various techniques exist, such as inter-frame prediction techniques that use video compression technology to predict pixel values ​​included in the current frame from frames before or after the current frame, intra-frame prediction techniques that use pixel information in the current frame to predict pixel values ​​included in the current frame, and entropy coding techniques that assign short codes to values ​​that occur frequently and long codes to values ​​that occur infrequently. These image compression techniques can be used to effectively compress image data and send or store it. Summary of the Invention

[0004] Technical issues

[0005] This disclosure provides methods and apparatus for deriving MDM.

[0006] This disclosure provides methods and apparatus for configuring adjacent regions to derive MDM.

[0007] This disclosure provides a mask kernel for deriving MDM.

[0008] Technical solution

[0009] The image decoding method and apparatus according to this disclosure can derive one or more primary intra-prediction modes (MDMs) by applying a predetermined mask kernel to the adjacent regions of the current block, generate a prediction block of the current block based on one or more MDMs, and reconstruct the current block based on the prediction block.

[0010] In the image decoding method and apparatus according to this disclosure, any one of the coefficients of the mask kernel is 1, 3, 47 or 48, and another coefficient of the mask kernel is 2, 10, 162 or 160.

[0011] In the image decoding method and apparatus according to this disclosure, the mask kernel can be any one of a plurality of predefined mask kernels. Here, the plurality of mask kernels may include at least one of a first mask kernel having coefficients 1 and 2, a second mask kernel having coefficients 3 and 10, a third mask kernel having coefficients 47 and 162, or a fourth mask kernel having coefficients 48 and 160.

[0012] In the image decoding method and apparatus according to the present disclosure, the mask kernel can be determined as any one of a plurality of mask kernels based on at least one of the size or shape of the current block.

[0013] In the image decoding method and apparatus according to the present disclosure, the adjacent region may be at least one of a first adjacent region including the upper adjacent region of the current block, a second adjacent region including the left adjacent region of the current block, or a third adjacent region including the upper adjacent region and the left adjacent region of the current block.

[0014] In the image decoding method and apparatus according to the present disclosure, based on at least one of the size or shape of the current block, the adjacent region can be determined as any one of the first adjacent region to the third adjacent region.

[0015] In the image decoding method and apparatus according to this disclosure, the step of deriving one or more MDMs may include: deriving sample gradients at target points where a mask kernel is applied in adjacent regions, determining intra-prediction modes to be mapped to the sample gradients, and calculating intensity values ​​of the determined intra-prediction modes based on the sample gradients. The sample gradients may include at least one of sample gradients in the horizontal direction or sample gradients in the vertical direction.

[0016] In the image decoding method and apparatus according to the present disclosure, a predetermined weight can be applied to the intensity value based on the position of the sample row to which the target point belongs in the adjacent region.

[0017] In the image decoding method and apparatus according to the present disclosure, the intensity value can be calculated based on at least one of the absolute value of the sample gradient in the horizontal direction or the absolute value of the sample gradient in the vertical direction.

[0018] In the image decoding method and apparatus according to this disclosure, the intra-prediction mode to be mapped to the sample gradient can be determined from among intra-prediction modes that can be used to derive the MDM.

[0019] In the image decoding method and apparatus according to this disclosure, the available intra-frame prediction mode can be a mode with an even number or an odd number among N predefined intra-frame prediction modes.

[0020] In the image decoding method and apparatus according to this disclosure, the available intra-frame prediction modes can be M predefined intra-frame prediction modes for each frame, slice, or coding tree unit.

[0021] The image coding method and apparatus according to this disclosure can derive one or more dominant intra-prediction modes (MDMs) by applying a predetermined mask kernel to the adjacent regions of the current block, generate a prediction block of the current block based on one or more MDMs, generate a residual block of the current block based on the prediction block of the current block, derive the transform coefficients of the current block based on the residual block, and encode the residual information related to the transform coefficients.

[0022] A computer-readable digital storage medium is provided for storing encoded video / image information, which enables a decoding device according to the present disclosure to perform an image decoding method.

[0023] A computer-readable digital storage medium is provided for storing video / image information generated based on the image encoding method of this disclosure.

[0024] A method and apparatus are provided for transmitting video / image information generated according to the image encoding method of this disclosure.

[0025] Beneficial effects

[0026] According to this disclosure, by defining various types of mask kernels and selectively using the mask kernels, the directionality in adjacent regions can be predicted more accurately, and the computational complexity can be reduced.

[0027] According to this disclosure, by adaptively setting the range of adjacent regions for MDM derivation, the computational complexity can be reduced while maintaining the accuracy of MDM.

[0028] According to this disclosure, the computational complexity required for strength value operations can be greatly reduced while maintaining the verification of MDM.

[0029] According to this disclosure, by adjusting the range of intra-prediction modes that can be used for MDM derivation, the computational complexity of MDM derivation can be greatly reduced. Attached Figure Description

[0030] Figure 1 A video / image encoding system according to this disclosure is shown.

[0031] Figure 2 A schematic block diagram of an encoding apparatus to which embodiments of the present disclosure are applicable and which performs encoding of video / image signals is shown.

[0032] Figure 3 A schematic block diagram of a decoding apparatus to which embodiments of the present disclosure are applicable and which performs decoding of video / image signals is shown.

[0033] Figure 4 An example is shown of a decoding method performed by a decoding device 300 as an embodiment of the present disclosure.

[0034] Figure 5 An intra-frame prediction mode used in a video compression standard as an embodiment of this disclosure is illustrated.

[0035] Figures 6 to 11 This relates to a method for deriving an MDM as an embodiment of the present disclosure.

[0036] Figure 12 An illustrative configuration of a decoding device 300 performing a decoding method according to the present disclosure is shown.

[0037] Figure 13 An encoding method performed by the encoding device 200 as an embodiment of the present disclosure is illustrated.

[0038] Figure 14 An illustrative configuration of an encoding device 200 performing an encoding method according to the present disclosure is shown.

[0039] Figure 15 An example of a content streaming system to which embodiments of the present disclosure can be applied is shown. Detailed Implementation

[0040] Because this disclosure can be modified in various ways and has multiple embodiments, specific embodiments will be shown in the accompanying drawings and described in detail in the specific embodiments. However, this disclosure is not intended to be limited to the specific embodiments, but should be understood to include all changes, equivalents, and substitutions included within the spirit and scope of this disclosure. Similar reference numerals are used for similar components in the description of the various figures.

[0041] Terms such as "first," "second," etc., may be used to describe various components, but components should not be limited by these terms. Terms are used only to distinguish one component from others. For example, without departing from the scope of this disclosure, a first component may be referred to as a second component, and similarly, a second component may be referred to as a first component. Terms include any combination of one or more of the associated terms.

[0042] When a component is described as "connected" or "linked" to another component, it should be understood that it can be directly connected or linked to the other component, but the other component can exist in between. On the other hand, when a component is described as "directly connected" or "directly linked" to another component, it should be understood that there is no other component in between.

[0043] The terminology used in this application is for describing particular embodiments only and is not intended to limit this disclosure. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this application, it should be understood that terms such as “comprising” or “having” are intended to specify the presence of the features, quantities, steps, operations, components, portions, or combinations thereof described in this specification, but do not preclude the possibility of the presence or addition of one or more other features, quantities, steps, operations, components, portions, or combinations thereof.

[0044] This disclosure relates to video / image coding. For example, the methods / implementations disclosed herein can be applied to methods disclosed in the Multifunctional Video Coding (VVC) standard. Additionally, the methods / implementations disclosed herein can be applied to methods disclosed in the Basic Video Coding (EVC) standard, the AOMedia Video 1 (AV1) standard, the Audio Video Coding 2 (AVN2) standard, or next-generation video / image coding standards (e.g., H.267 or H.268, etc.).

[0045] This specification sets forth various implementations of video / image encoding, and unless otherwise specified, these implementations may be combined with each other.

[0046] In this article, video can refer to a collection of images over time. A frame typically refers to a unit representing an image within a specific time period, and a slice / tile is a unit that forms part of a frame during encoding. A slice / tile can include at least one Code Tree Unit (CTU). A frame can consist of at least one slice / tile. A tile is a rectangular area composed of multiple CTUs within a specific tile column and a specific tile row of a frame. A tile column is a rectangular area of ​​CTUs with a height equal to the height of the frame and a width specified by the syntax requirements of the frame parameter set. A tile row is a rectangular area of ​​CTUs with a height specified by the frame parameter set and a width equal to the width of the frame. CTUs within a tile can be arranged continuously according to CTU raster scans, and tiles within a frame can be arranged continuously according to tile raster scans. A slice can include an integer number of complete tiles of a frame that can be exclusively included in a single NAL unit, or an integer number of consecutive complete CTU rows within a tile. Furthermore, a frame can be divided into at least two sub-frames. A sub-frame can be a rectangular area of ​​at least one slice within a frame.

[0047] A pixel, or pelin, can represent the smallest unit that makes up a frame (or image). Additionally, "sample" can be used as the term corresponding to a pixel. A sample can typically represent a pixel or a pixel value, and can represent only the pixel / pixel value of the luminance component or only the pixel / pixel value of the chrominance component.

[0048] A unit can represent the basic unit of image processing. A unit may include a specific region of the image and at least one of the information associated with that region. A unit may include a luminance block and two chrominance (e.g., cb, cr) blocks. In some cases, the term "unit" may be used interchangeably with terms such as "block" or "region". In general, an M×N block may include a set (or array) of transform coefficients or samples (or sample arrays) consisting of M columns and N rows.

[0049] In this document, “A or B” can mean “A only”, “B only”, or “both A and B”. In other words, “A or B” can be interpreted as “A and / or B”. For example, “A, B or C” can mean “A only”, “B only”, “C only”, or “any combination of A, B and C”.

[0050] The forward slash ( / ) or comma used in this article can indicate "and / or". For example, "A / B" can mean "A and / or B". Therefore, "A / B" can mean "A only", "B only", or "both A and B". For example, "A, B, C" can mean "A, B, or C".

[0051] In this document, "at least one of A and B" can mean "only A", "only B" or "both A and B". Furthermore, in this document, expressions such as "at least one of A or B" or "at least one of A and / or B" can be interpreted in the same way as "at least one of A and B".

[0052] Additionally, in this document, "at least one of A, B, and C" can mean "A only", "B only", "C only" or "any combination of A, B, and C". Furthermore, "at least one of A, B, or C" or "at least one of A, B, and / or C" can mean "at least one of A, B, and C".

[0053] Additionally, the parentheses used in this document can indicate "for example". Specifically, when the indication is "prediction (intra-frame prediction)", "intra-frame prediction" can be cited as an example of "prediction". In other words, "prediction" in this document is not limited to "intra-frame prediction", and "intra-frame prediction" can be cited as an example of "prediction". Furthermore, even when the indication is "prediction (i.e., intra-frame prediction)", "intra-frame prediction" can be cited as an example of "prediction".

[0054] In this article, the technical features described individually in a single diagram can be implemented individually or simultaneously.

[0055] Figure 1 A video / image encoding system according to this disclosure is shown.

[0056] Reference Figure 1A video / image encoding system may include a first device (source device) and a second device (receiving device).

[0057] A source device can transmit encoded video / image information or data to a receiving device in the form of a file or stream via a digital storage medium or network. The source device may include a video source, an encoding device, and a transmitting unit. The receiving device may include a receiving unit, a decoding device, and a renderer. The encoding device may be referred to as a video / image encoding device, and the decoding device may be referred to as a video / image decoding device. The transmitter may be included in the encoding device. The receiver may be included in the decoding device. The renderer may include a display unit, and the display unit may consist of a separate device or external components.

[0058] A video source can acquire video / images through processes that capture, synthesize, or generate video / images. A video source may include means for capturing video / images and means for generating video / images. Means for capturing video / images may include at least one camera, a video / image archive containing previously captured video / images, etc. Means for generating video / images may include a computer, tablet computer, smartphone, etc., and can generate video / images (electronically). For example, virtual video / images can be generated by a computer, etc., and in this case, the process of capturing video / images can be replaced by a process of generating related data.

[0059] Encoding devices can encode input video / images. They can perform a series of processes such as prediction, transformation, and quantization for compression and encoding efficiency. The encoded data (encoded video / image information) can be output as a bitstream.

[0060] The transmitting unit can send encoded video / image information or data, output in bitstream form, to the receiving unit of the receiving device in the form of a file or stream via a digital storage medium or network. The digital storage medium can include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The transmitting unit can include elements for generating media files according to a predetermined file format, and may include elements for transmission via a broadcast / communication network. The receiving unit can receive / extract the bitstream and send it to a decoding device.

[0061] Decoding devices can decode video / images by performing a series of processes, such as dequantization, inverse transform, and prediction, that correspond to the operations of encoding devices.

[0062] The renderer can render decoded video / images. The rendered video / images can be displayed through a display unit.

[0063] Figure 2A schematic block diagram of an encoding apparatus that can be applied to embodiments of the present disclosure and perform encoding of video / image signals is shown.

[0064] Reference Figure 2 The encoding device 200 may consist of an image segmenter 210, a predictor 220, a residual processor 230, an entropy encoder 240, an adder 250, a filter 260, and a memory 270. The predictor 220 may include an inter-frame predictor 221 and an intra-frame predictor 222. The residual processor 230 may include a transformer 232, a quantizer 233, a dequantizer 234, and an inverse transformer 235. The residual processor 230 may also include a subtractor 231. The adder 250 may be referred to as a reconstructor or a reconstruction block generator. According to embodiments, the image segmenter 210, predictor 220, residual processor 230, entropy encoder 240, adder 250, and filter 260 may be configured by at least one hardware component (e.g., an encoder chipset or processor). Additionally, the memory 270 may include a decoded picture buffer (DPB) and may be configured by a digital storage medium. The hardware component may also include the memory 270 as an internal / external component.

[0065] Image segmenter 210 can segment an input image (or picture or frame) input to encoding device 200 into at least one processing unit. As an example, a processing unit can be referred to as a coding unit (CU). In this case, the coding unit can be recursively segmented from coding tree unit (CTU) or maximum coding unit (LCU) according to a quadtree-binary-tritree (QTBTTT) structure.

[0066] For example, a coding unit can be segmented into multiple deeper coding units based on a quadtree, binary tree, and / or ternary tree structure. In this case, for example, a quadtree structure can be applied first, followed by a binary tree and / or ternary tree structure. Alternatively, a binary tree structure can be applied before the quadtree structure. The coding process according to this specification can be performed based on the final coding unit that is no longer segmented. In this case, based on image characteristics, coding efficiency, etc., the largest coding unit can be directly used as the final coding unit, or, if necessary, the coding unit can be recursively segmented into deeper coding units, and the coding unit with the optimal size can be used as the final coding unit. Here, the coding process can include processes such as prediction, transformation, and reconstruction, as described later.

[0067] As another example, the processing unit may also include a prediction unit (PU) or a transform unit (TU). In this case, the prediction unit and the transform unit can be divided or segmented from the aforementioned final encoding unit, respectively. The prediction unit may be a unit for predicting samples, and the transform unit may be a unit for deriving transform coefficients and / or a unit for deriving residual signals from transform coefficients.

[0068] In some cases, a unit can be used interchangeably with terms such as block or region. Generally, an M×N block can represent a set of transform coefficients or samples consisting of M columns and N rows. Samples can typically represent pixels or pixel values, and can represent only the pixel / pixel value of the luminance component, or only the pixel / pixel value of the chrominance component. Samples can be used as a term to form a frame (or image) corresponding to a pixel or cell.

[0069] Encoding device 200 can subtract the prediction signal (prediction block, prediction sample array) output from inter-frame predictor 221 or intra-frame predictor 222 from the input image signal (original block, original sample array) to generate a residual signal (residual signal, residual sample array), and the generated residual signal is sent to converter 232. In this case, the unit in encoding device 200 that subtracts the prediction signal (prediction block, prediction sample array) from the input image signal (original block, original sample array) can be called subtractor 231.

[0070] Predictor 220 can perform prediction on the block to be processed (hereinafter referred to as the current block) and generate a prediction block that includes prediction samples of the current block. Predictor 220 can determine whether to apply intra-frame prediction or inter-frame prediction on a per-block or per-unit basis. Predictor 220 can generate various information about the prediction (e.g., prediction mode information) and send it to entropy encoder 240, as described later in the description of the various prediction modes. The information about the prediction can be encoded in entropy encoder 240 and output as a bitstream.

[0071] Intra-frame predictor 222 can predict the current block by referencing samples within the current frame. Depending on the prediction mode, the referenced samples may be located near the current block or at a specific distance from it. In intra-frame prediction, the prediction mode can include one or more non-directional modes and multiple directional modes. Non-directional modes can include at least one of DC mode or planar mode. Depending on the granularity of the prediction direction, the directional modes can include 33 or 65 directional modes. However, this is just an example; more or fewer directional modes may be used depending on the settings. Intra-frame predictor 222 can also determine the prediction mode to be applied to the current block by using prediction modes applied to a template region.

[0072] Inter-frame predictor 221 can derive the prediction block of the current block based on a reference block (reference sample array) to be specified by motion vectors on a reference frame. In this case, to reduce the amount of motion information to be transmitted in inter-frame prediction mode, motion information can be predicted on a block, sub-block, or sample basis based on the correlation between the template region and the motion information of the current block. Motion information may include motion vectors and reference frame indices. Motion information may also include inter-frame prediction direction information (L0 prediction, L1 prediction, Bi prediction, etc.). For inter-frame prediction, the template region may include a spatial template region (spatial neighbor block) existing in the current frame and a temporal template region (temporal neighbor block) existing in the reference frame. The reference frame including the reference block and the reference frame including the temporal template region may be the same or different. The temporal template region may be referred to as a juxtaposed reference block, a juxtaposed CU (colCU), etc., and the reference frame including the temporal template region may be referred to as a juxtaposed frame (colPic). For example, inter-frame predictor 221 can construct a motion information candidate list based on the template region and generate information indicating which candidate to use to derive the motion vector and / or reference frame index of the current block. Inter-frame prediction can be performed based on various prediction modes, and for example, for skip mode and merge mode, the inter-frame predictor 221 can use the motion information of the template region as the motion information of the current block. For skip mode, unlike merge mode, residual signals may not be sent. For motion vector prediction (MVP) mode, the motion vector of the template region can be used as a motion vector predictor, and the motion vector difference can be signaled to indicate the motion vector of the current block.

[0073] Predictor 220 can generate a prediction signal based on various prediction methods described later. For example, the predictor can not only apply intra-frame prediction or inter-frame prediction to predict a block, but can also apply both intra-frame and inter-frame prediction simultaneously. This can be referred to as the Inter-intra-frame Combined Prediction (CIIP) mode. Alternatively, the predictor can predict blocks based on the Intra-Block Copy (IBC) prediction mode, or it can predict blocks based on a palette mode. The IBC prediction mode or palette mode can be used for content image / video coding such as Screen Content Coding (SCC) in games, etc. IBC essentially performs prediction within the current frame, but it can be performed similarly to inter-frame prediction because it derives a reference block within the current frame. In other words, IBC can use at least one of the inter-frame prediction techniques described herein. The palette mode can be considered an example of intra-frame coding or intra-frame prediction. When applying a palette mode, sample values ​​within the frame can be signaled based on information about the palette table and palette index. The prediction signal generated by predictor 220 can be used to generate a reconstructed signal or a residual signal.

[0074] Transformer 232 can generate transform coefficients by applying transform techniques to the residual signal. For example, the transform techniques may include at least one of Discrete Cosine Transform (DCT), Discrete Sine Transform (DST), Karhunen–Loève Transform (KLT), Graph-Based Transform (GBT), or Conditional Nonlinear Transform (CNT). Here, GBT represents the transform obtained from a graph when the relationship information between pixels is represented as a graph. CNT represents the transform obtained based on generating a prediction signal using all previously reconstructed pixels. Furthermore, the transform processing can be applied to square pixel blocks of the same size, or it can be applied to non-square blocks of variable size.

[0075] Quantizer 233 can quantize the transform coefficients and send them to entropy encoder 240, which can encode the quantized signal (information about the quantized transform coefficients) and output it as a bitstream. This information about the quantized transform coefficients can be called residual information. Quantizer 233 can rearrange the block-form quantized transform coefficients into a one-dimensional vector based on the coefficient scan order, and can generate information about the quantized transform coefficients based on this one-dimensional vector form.

[0076] The entropy encoder 240 can perform various encoding methods such as Golomb, context-adaptive variable-length coding (CAVLC), and context-adaptive binary arithmetic coding (CABAC). The entropy encoder 240 can encode information required for video / image reconstruction other than quantization transform coefficients (e.g., values ​​of syntax elements, etc.) together or separately.

[0077] Encoded information (e.g., encoded video / image information) can be transmitted or stored in bitstream form at the Network Abstraction Layer (NAL) unit level. The video / image information may also include information about various parameter sets such as Adaptive Parameter Set (APS), Picture Parameter Set (PPS), Sequence Parameter Set (SPS), or Video Parameter Set (VPS). Additionally, the video / image information may include general constraint information. Information and / or syntax elements transmitted from the encoding device to / signaled to the decoding device may be included in the video / image information. The video / image information can be encoded and included in the bitstream through the encoding process described above. The bitstream can be transmitted over a network or stored in a digital storage medium. Here, the network may include broadcast networks and / or communication networks, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. A transmitting unit (not shown) for transmitting the signal output from the entropy encoder 240 and / or a storage unit (not shown) for storing the signal may be configured as internal / external components of the encoding device 200, or the transmitting unit may also be included in the entropy encoder 240.

[0078] The quantized transform coefficients output from quantizer 233 can be used to generate a prediction signal. For example, the residual signal (residual block or residual sample) can be reconstructed by applying dequantization and inverse transform to the quantized transform coefficients via dequantizer 234 and inverse transformer 235. Adder 250 can add the reconstructed residual signal to the prediction signal output from inter-frame predictor 221 or intra-frame predictor 222 to generate a reconstructed signal (reconstructed frame, reconstructed block, reconstructed sample array). When there is no residual for the block to be processed (similar to when a skip mode is applied), the prediction block can be used as a reconstructed block. Adder 250 can be referred to as a reconstructor or reconstructed block generator. The generated reconstructed signal can be used for intra-frame prediction of the next block to be processed in the current frame, and can also be used for inter-frame prediction of the next frame by filtering, as described later. Furthermore, luminance mapping and chroma scaling (LMCS) can be applied in frame encoding and / or reconstruction processing.

[0079] Filter 260 can improve subjective / objective image quality by applying filtering to the reconstructed signal. For example, filter 260 can generate a modified reconstructed image by applying various filtering methods to the reconstructed image, and the modified reconstructed image can be stored in memory 270 (specifically, the DPB of memory 270). Various filtering methods can include deblocking filtering, sample adaptive offsetting, adaptive loop filtering, bilateral filtering, etc. Filter 260 can generate various information about the filtering and send it to entropy encoder 240. The information about the filtering can be encoded in entropy encoder 240 and output as a bitstream.

[0080] The modified reconstructed frame sent to memory 270 can be used as a reference frame in inter-frame predictor 221. When inter-frame prediction is applied through it, the encoding device can avoid prediction mismatch between the encoding device 200 and the decoding device, and can also improve encoding efficiency.

[0081] The DPB of memory 270 can store modified reconstructed frames for use as reference frames in inter-frame predictor 221. Memory 270 can store motion information of blocks in the current frame from which motion information is derived (or encoded) and / or of blocks in previously reconstructed frames. The stored motion information can be sent to inter-frame predictor 221 as motion information for spatial template regions or temporal template regions. Memory 270 can store reconstructed samples of reconstructed blocks in the current frame and send them to intra-frame predictor 222.

[0082] Figure 3 A schematic block diagram of a decoding device that can be implemented using embodiments of the present disclosure and performs decoding of video / image signals is shown.

[0083] Reference Figure 3 The decoding device 300 can be configured to include an entropy decoder 310, a residual processor 320, a predictor 330, an adder 340, a filter 350, and a memory 360. The predictor 330 may include an inter-frame predictor 332 and an intra-frame predictor 331. The residual processor 320 may include a dequantizer 321 and an inverse transformer 322.

[0084] According to the implementation, the entropy decoder 310, residual processor 320, predictor 330, adder 340, and filter 350 described above can be configured by a single hardware component (e.g., a decoder chipset or processor). Additionally, the memory 360 may include a decoded screen buffer (DPB) and can be configured by a digital storage medium. The hardware component may also include the memory 360 as an internal / external component.

[0085] When the input includes a bitstream containing video / image information, the decoding device 300 can respond to... Figure 2 The encoding device processes video / image information to reconstruct the image. For example, the decoding device 300 can deduce units / blocks based on block segmentation information obtained from the bitstream. The decoding device 300 can perform decoding using processing units applied in the encoding device. Therefore, the decoding processing unit can be an encoding unit, and the encoding unit can be segmented from the encoding tree unit or a larger encoding unit according to a quadtree structure, binary tree structure, and / or ternary tree structure. At least one transform unit can be derived from the encoding unit. Furthermore, the reconstructed image signal decoded and output by the decoding device 300 can be played back by a playback device.

[0086] Decoding device 300 can receive data in bitstream form from... Figure 2The signal output by the encoding device can be decoded by the entropy decoder 310. For example, the entropy decoder 310 can parse the bitstream to derive the information (e.g., video / image information) required for image reconstruction (or picture reconstruction). The video / image information may also include information about various parameter sets such as Adaptive Parameter Set (APS), Picture Parameter Set (PPS), Sequence Parameter Set (SPS), or Video Parameter Set (VPS). In addition, the video / image information may also include general constraint information. The decoding device can also decode the picture based on the information about the parameter sets and / or general constraint information. The information and / or syntax elements that are signaled / received, as described later herein, can be decoded and obtained from the bitstream through the decoding process. For example, the entropy decoder 310 can decode the information in the bitstream based on encoding methods such as Exponential Golomb coding, CAVLC, CABAC, etc., and output the values ​​of the syntax elements required for image reconstruction and the quantized values ​​of the transform coefficients of the residuals. More specifically, the CABAC entropy decoding method can receive bins corresponding to each syntax element from the bitstream, determine a context model using information about the syntax element to be decoded, decoding information of surrounding blocks and the block to be decoded, or information about symbols / bins decoded in previous steps, perform arithmetic decoding of bins by predicting the occurrence probability of bins based on the determined context model, and generate symbols corresponding to the values ​​of each syntax element. In this case, after determining the context model, the CABAC entropy decoding method can update the context model by using information about the decoded symbols / bins for the context model of the next symbol / bin. Among the information decoded in the entropy decoder 310, information about prediction is provided to the predictors (inter-frame predictor 332 and intra-frame predictor 331), and the residual values ​​(i.e., quantization transform coefficients and related parameter information) from which entropy decoding has been performed in the entropy decoder 310 can be input to the residual processor 320. The residual processor 320 can derive residual signals (residual blocks, residual samples, residual sample arrays). In addition, information about filtering from the information decoded in the entropy decoder 310 can be provided to the filter 350. Furthermore, the receiving unit (not shown) that receives the signal output from the encoding device can be further configured as an internal / external component of the decoding device 300, or the receiving unit can be a component of the entropy decoder 310.

[0087] Furthermore, the decoding device according to this specification may be referred to as a video / image / screen decoding device, and the decoding device may be divided into an information decoder (video / image / screen information decoder) and a sample decoder (video / image / screen sample decoder). The information decoder may include an entropy decoder 310, and the sample decoder may include at least one of a dequantizer 321, an inverse transformer 322, an adder 340, a filter 350, a memory 360, an inter-frame predictor 332, and an intra-frame predictor 331.

[0088] Dequantizer 321 can dequantize the quantized transform coefficients and output the transform coefficients. Dequantizer 321 can rearrange the quantized transform coefficients into two-dimensional blocks. In this case, the rearrangement can be performed based on the coefficient scan order performed in the encoding device. Dequantizer 321 can perform dequantization on the quantized transform coefficients using quantization parameters (e.g., quantization step size information) and obtain the transform coefficients.

[0089] The inverse transformer 322 performs an inverse transformation on the transformation coefficients to obtain the residual signal (residual block, residual sample array).

[0090] Predictor 320 can perform prediction on the current block and generate a prediction block that includes prediction samples of the current block. Predictor 320 can determine whether to apply intra-frame prediction or inter-frame prediction to the current block based on the prediction information output from entropy decoder 310, and determine the specific intra-frame / inter-frame prediction mode.

[0091] Predictor 320 can generate prediction signals based on various prediction methods described later. For example, predictor 320 can not only apply intra-frame prediction or inter-frame prediction to predict a block, but can also apply intra-frame prediction and inter-frame prediction simultaneously. This can be referred to as the Inter-Frame Intra-Frame Combined Prediction (CIIP) mode. Alternatively, the predictor can predict blocks based on the Intra-Frame Block Copy (IBC) prediction mode, or it can predict blocks based on a palette mode. The IBC prediction mode or palette mode can be used for content image / video coding such as Screen Content Coding (SCC) in games, etc. IBC essentially performs prediction within the current frame, but it can be performed similarly to inter-frame prediction because it derives a reference block within the current frame. In other words, IBC can use at least one of the inter-frame prediction techniques described herein. The palette mode can be considered an example of intra-frame coding or intra-frame prediction. When the palette mode is applied, information about the palette table and palette index can be included in the video / image information and signaled.

[0092] Intra-predictor 331 can predict the current block by referencing samples within the current frame. Depending on the prediction mode, the referenced samples can be located near the current block or at a specific distance away. In intra-prediction, the prediction mode can include at least one non-directional mode and multiple directional modes. Intra-predictor 331 can determine the prediction mode applied to the current block by using the prediction mode applied to the template region.

[0093] Inter-frame predictor 332 can deduce the prediction block of the current block based on a reference block (reference sample array) to be specified by motion vectors on a reference frame. In this case, to reduce the amount of motion information to be transmitted in the inter-frame prediction mode, motion information can be predicted on a block, sub-block, or sample basis based on the correlation between the template region and the motion information of the current block. Motion information may include motion vectors and reference frame indices. Motion information may also include inter-frame prediction direction information (L0 prediction, L1 prediction, Bi prediction, etc.). For inter-frame prediction, the template region may include a spatial template region (spatial neighbor block) existing in the current frame and a temporal template region (temporal neighbor block) existing in the reference frame. For example, inter-frame predictor 332 can construct a motion information candidate list based on the template region and deduce the motion vector and / or reference frame index of the current block based on the received candidate selection information. Inter-frame prediction can be performed based on various prediction modes, and the information related to the prediction may include information indicating the inter-frame prediction mode of the current block.

[0094] Adder 340 can add the obtained residual signal to the prediction signal (prediction block, prediction sample array) output from the predictor (including inter-frame predictor 332 and / or intra-frame predictor 331) to generate a reconstruction signal (reconstructed frame, reconstruction block, reconstruction sample array). When there is no residual for the block to be processed (similar to when a skip mode is applied), the prediction block can be used as a reconstruction block.

[0095] Adder 340 can be referred to as a reconstructor or reconstruction block generator. The generated reconstructed signal can be used for intra-frame prediction of the next block to be processed in the current frame, can be output through filtering as described later, or can be used for inter-frame prediction of the next frame. In addition, luminance mapping and chroma scaling (LMCS) can be applied in the frame decoding process.

[0096] Filter 350 can improve subjective / objective image quality by applying filtering to the reconstructed signal. For example, filter 350 can generate a modified reconstructed image by applying various filtering methods to the reconstructed image and send the modified reconstructed image to memory 360 (specifically, the DPB of memory 360). Various filtering methods may include deblocking filtering, sample adaptive offset, adaptive loop filter, bilateral filter, etc.

[0097] The (modified) reconstructed frame stored in the DPB of memory 360 can be used as a reference frame in inter-frame predictor 332. Memory 360 can store motion information of blocks in the current frame from which motion information is derived (or decoded) and / or motion information of blocks in previously reconstructed frames. The stored motion information can be sent to inter-frame predictor 332 as motion information for spatial template regions or temporal template regions. Memory 360 can store reconstructed samples of reconstructed blocks in the current frame and send them to intra-frame predictor 331.

[0098] The embodiments described in this document in the filter 260, inter-frame predictor 221 and intra-frame predictor 222 of the encoding device 200 can also be applied equivalently or correspondingly to the filter 350, inter-frame predictor 332 and intra-frame predictor 331 of the decoding device 300, respectively.

[0099] Figure 4 An image decoding method performed by a decoding device 300 as an embodiment of the present disclosure is illustrated.

[0100] Reference Figure 4 One or more primary intra-prediction modes (MDM) S400 can be derived by applying a predetermined mask kernel to the adjacent regions of the current block.

[0101] This disclosure proposes a mask kernel for measuring the sample gradient of samples belonging to neighboring regions of the current block. Here, the mask kernel may include at least one of a vertical mask kernel for measuring sample gradients in the horizontal direction or a horizontal mask kernel for measuring sample gradients in the vertical direction. The mask kernel can be understood as being replaced by a mask or filter.

[0102] Since changes in neighboring sample values ​​often occur based on directionality, intra-frame prediction modes used in video compression standards can be defined by including orientation modes with predetermined prediction directions. For example, they can be defined as follows: Figure 5 The diagram shows a directional intra-prediction mode definition.

[0103] To derive the MDM according to this disclosure, sample gradients and directions of change of (reconstructed) samples belonging to adjacent regions of the current block can be used. To measure the sample gradients and directions of change, the sample gradients in the horizontal direction and the sample gradients in the vertical direction can be measured separately, and the direction of change of the samples can be determined based on this.

[0104] As shown in Equation 1, this can be based on the relationship between the sample block (P) and the mask kernel (A). The sample gradient (G) is derived through computation.

[0105] [Formula 1]

[0106] In Equation 1, The operation can be derived by multiplying the sum of values ​​obtained by the concatenation of the sample block (P) and the mask kernel (A). For example, when the 3×3 sample block (P) and the mask kernel (A) are represented by Equations 2 and 3 respectively, the value between P and A can be defined as shown in Equation 4. Calculation.

[0107] [Equation 2]

[0108] [Formula 3]

[0109] [Formula 4]

[0110] The horizontal sample gradient (Gx) of a 3×3 sample block (P) can be derived by applying a vertical mask kernel (V) of the same size to the corresponding sample block (P). In this case, when the vertical mask kernel (V) defined in Equation 5 is applied to the 3×3 sample block (P) of Equation 2, the horizontal sample gradient (Gx) can be derived as shown in Equation 6.

[0111] [Formula 5]

[0112] [Formula 6]

[0113] The vertical sample gradient (Gy) of a 3×3 sample block (P) can be derived by applying a horizontal mask kernel (H) of the same size to the corresponding sample block (P). In this case, when the horizontal mask kernel (H) defined in Equation 7 is applied to the 3×3 sample block (P) of Equation 2, the vertical sample gradient (Gy) can be derived as shown in Equation 8.

[0114] [Formula 7]

[0115] [Formula 8]

[0116] This disclosure proposes a mask kernel for measuring gradients in the horizontal and vertical directions that can be used for image compression by proposing filter coefficients (C1, C2) of the mask kernel. In the embodiments described below, C1 can refer to the filter coefficient of any one of the input samples to be applied to the mask kernel, and C2 can refer to the filter coefficient of another input sample to be applied to the input samples.

[0117] Implementation Method 1

[0118] The values ​​of C1 and C2 can be set to 1 and 2, respectively. In this case, the computational complexity required to derive the values ​​of Gx and Gy can be greatly reduced. This is because since the values ​​of C1 and C2 are represented by powers of 2, multiplication with each sample can be replaced by a shift operator. Therefore, the vertical mask kernel (V) and the horizontal mask kernel (H) can be defined by Equations 9 and 10 below, respectively.

[0119] [Formula 9]

[0120] [Formula 10]

[0121] Implementation Method 2

[0122] The values ​​of C1 and C2 can be set to 3 and 10, respectively. In this case, although the computational complexity may increase compared to Implementation 1, the gradient in the horizontal / vertical direction can be measured more accurately, thereby improving the coding compression performance. Therefore, the vertical mask kernel (V) and the horizontal mask kernel (H) can be defined by Equations 11 and 12 below, respectively.

[0123] [Equation 11]

[0124] [Equation 12]

[0125] When the vertical mask kernel (V) according to Equation 11 is applied to the above 3×3 sample block (P), the sample gradient (Gx) in the horizontal direction of the corresponding sample block (P) can be derived as shown in Equation 13 below.

[0126] [Equation 13]

[0127] When the horizontal mask kernel (H) according to Equation 12 is applied to the above 3×3 sample block (P), the sample gradient (Gy) in the vertical direction of the corresponding sample block (P) can be derived as shown in Equation 14 below.

[0128] [Formula 14]

[0129] Implementation Method 3

[0130] The values ​​of C1 and C2 can be set to 47 and 162, respectively. In this case, the most accurate gradient in the horizontal / vertical direction can be measured while maintaining 8-bit precision. Therefore, the vertical mask kernel (V) and the horizontal mask kernel (H) can be defined by Equations 15 and 16 below, respectively.

[0131] [Formula 15]

[0132] [Formula 16]

[0133] When the vertical mask kernel (V) according to Equation 15 is applied to the above 3×3 sample block (P), the sample gradient (Gx) in the horizontal direction of the corresponding sample block (P) can be derived as shown in Equation 17 below.

[0134] [Equation 17]

[0135] When the horizontal mask kernel (H) according to Equation 16 is applied to the above 3×3 sample block (P), the sample gradient (Gy) in the vertical direction of the corresponding sample block (P) can be derived as shown in Equation 18 below.

[0136] [Formula 18]

[0137] Implementation Method 4

[0138] Multiplication can be implemented using shift operations and addition operations. Therefore, to reduce computational complexity, the values ​​of C1 and C2 can be set to 48 and 160, respectively. For example, since the multiplication of (48 × P11) equals {(32 × P11) + (16 × P11)}, it can be represented as (P11 << 5 + P11 << 4). Similarly, since the multiplication of (160 × P12) equals {(128 × P12) + (32 × P2)}, it can be represented as (P12 << 7 + P12 << 5). Therefore, when 48 and 160 are used as the values ​​of C1 and C2, the computational complexity can be greatly reduced within a slight error tolerance compared to implementation method 3.

[0139] The vertical mask kernel (V) and the horizontal mask kernel (H) according to embodiment 4 can be defined by Equations 19 and 20 below, respectively.

[0140] [Formula 19]

[0141] [Formula 20]

[0142] When the vertical mask kernel (V) according to Equation 19 is applied to the above 3×3 sample block (P), the sample gradient (Gx) in the horizontal direction of the corresponding sample block (P) can be derived as shown in Equation 21 below.

[0143] [Equation 21]

[0144] When the horizontal mask kernel (H) according to Equation 20 is applied to the above 3×3 sample block (P), the sample gradient (Gy) in the vertical direction of the corresponding sample block (P) can be derived as shown in Equation 22 below.

[0145] [Equation 22]

[0146] The predetermined mask kernel to be applied to the adjacent regions of the current block can be a mask kernel predefined in the same way in both the encoding and decoding devices. Here, the predefined mask kernel can be any of the mask kernels in Embodiments 1 to 4 described above.

[0147] Alternatively, multiple mask cores can be defined in the encoding and decoding devices, and any one of the multiple mask cores can be selectively used. Here, the multiple mask cores may include at least one of the mask cores in embodiments 1 to 4 described above.

[0148] Based on the characteristics (e.g., size, shape) of sample blocks in the current block and / or adjacent regions, any one of multiple mask kernels can be selectively used. For example, when both the width and height of the block are less than 16, mask kernels with values ​​of 1 and 2 for C1 and C2 can be selected; otherwise, mask kernels with values ​​of non-1 and non-2 for C1 and C2 can be selected. Alternatively, when the product of the width and height of the block is less than 256, mask kernels with values ​​of 1 and 2 for C1 and C2 can be selected; otherwise, mask kernels with values ​​of non-1 and non-2 for C1 and C2 can be selected. The mask kernels with values ​​of non-1 and non-2 for C1 and C2 can be any of the mask kernels in embodiments 2 to 4.

[0149] Alternatively, any one of multiple mask kernels can be selected based on index information. Here, the index information can specify any one of multiple predefined mask kernels. The index information can be encoded in the encoding device and signaled via a bitstream.

[0150] The MDM derivation process according to this disclosure may include a process of determining the intra-prediction mode to be mapped to the sample gradient based on the sample gradient in the horizontal / vertical direction, and a process of calculating the intensity value to be accumulated in the corresponding intra-prediction mode.

[0151] Figure 6 This illustrates the structure for applying vertical / horizontal mask kernels to the adjacent regions of the current block 600. Figure 6 In this context, we assume that the vertical mask kernel of Equation 5 and the horizontal mask kernel of Equation 7 are applied.

[0152] Reference Figure 6 When the vertical mask kernel is applied to a 3×3 sample block centered on the target point (P3), the sample gradient (Gx) in the horizontal direction of the corresponding sample in the 3×3 sample block can be derived as shown in Equation 23 below.

[0153] [Equation 23]

[0154] In addition, when the horizontal mask kernel is applied to a 3×3 sample block centered on the target point (P3), the vertical sample gradient (Gy) of the sample in the corresponding 3×3 sample block can be derived as shown in Equation 24 below.

[0155] [Equation 24]

[0156] The intra-frame prediction mode can be determined based on the sample gradients in the horizontal and vertical directions to determine the direction of change (A) of the samples to be mapped to within a 3×3 sample block.

[0157] For example, the direction of change (A) can be calculated as shown in Equation 25 below.

[0158] [Equation 25]

[0159] In Equation 25, A can represent as follows: Figure 7 The diagram shows the direction of change (or edge direction) of samples within a 3×3 sample block centered at the target point (P3). Additionally, the arctan function is the inverse function of the tangent in trigonometric functions. For example... Figure 7 As shown, arctan(Gy / Gx) can represent the degree formed by the coordinates Q(Gx, Gy) and the x-axis. In other words, since the direction of change (A) can be understood as the edge direction of adjacent samples, the intra-prediction mode to be mapped to the direction of change (A) can be the intra-prediction mode with the direction most similar to the corresponding direction of change (A) among the intra-prediction modes within a predetermined range.

[0160] For example, when Gy and Gx have the same size, the value of A becomes 45 degrees (= ), and the intra-prediction mode to be mapped to can be Figure 5 Intra-frame prediction mode 34.

[0161] The mapping between the value of A and the intra-prediction mode can be performed using an approximation formula or a predefined table (e.g., Table 1) in the encoding and decoding devices. Table 1 is merely an example illustrating the mapping relationship between the value of A and the intra-prediction mode, and the intra-prediction mode to be mapped according to the value of A may be defined differently from Table 1.

[0162] Unlike Equation 25, the value of A can be defined as arctan(Gx / Gy). Alternatively, the value of A can be defined based on both arctan(Gx / Gy) and arctan(Gy / Gx).

[0163] [Table 1]

[0164] The intra-prediction mode mapped to the calculated direction of change (A) is the intra-prediction mode obtained at the target point (P3), and the intensity (or amplitude) value of the corresponding intra-prediction mode can be calculated based on the sample gradient (Gx) in the horizontal direction and the sample gradient (Gy) in the vertical direction.

[0165] For example, the intensity value (G) of the corresponding intra-prediction mode can be calculated as the square root of the sum of squares of Gx and Gy, as shown in Equation 26 below.

[0166] [Equation 26]

[0167] This disclosure proposes the range of adjacent regions for applying a mask kernel.

[0168] An adjacent region can be defined as the first adjacent region, including the region above it but excluding the region to its left. For example, such as... Figure 8 As shown in (a), the MDM for the current block can be derived by calculating the direction of change (A) and intensity value (G) of each target point indicated by the gray dots based on the samples located above the current block. Target points in the first adjacent region can belong to a sample row that is one sample away from the current block.

[0169] Alternatively, an adjacent region can be defined as a second adjacent region that includes the adjacent region to the left but excludes the adjacent region above. For example, such as Figure 8As shown in (b), the MDM for the current block can be derived by calculating the direction of change (A) and intensity value (G) of each target point indicated by the gray dots based on the samples located to the left of the current block. Target points in the second adjacent region can belong to a sample row that is one sample away from the current block.

[0170] Alternatively, an adjacent area can be defined as a third adjacent area that includes the adjacent area above and the adjacent area to the left. For example, such as Figure 8 As shown in (c), the MDM for the current block can be derived by calculating the direction of change (A) and intensity value (G) of each target point indicated by the gray dots based on the samples located above and to the left of the current block. Target points in the third adjacent region can belong to a sample row that is one sample away from the current block.

[0171] Alternatively, adjacent regions can be defined by dividing them into at least two adjacent regions among the first to third adjacent regions described above. In this case, one or more MDMs can be derived from each of the at least two adjacent regions. For example, when adjacent regions are defined as first and second adjacent regions, one or more MDMs can be derived from each of the first and second adjacent regions. When adjacent regions are defined as first to third adjacent regions, one or more MDMs can be derived from each of the first to third adjacent regions.

[0172] As a neighboring region for MDM derivation, at least one of the first, second, or third neighboring regions can be selectively used.

[0173] For example, based on the size and / or shape of the current block, at least one neighboring region from the first to the third neighboring regions can be selected. Here, size can be defined as width, height, the product of width and height, the maximum / minimum value of width and height, or the ratio of width to height. Alternatively, at least one neighboring region from the first to the third neighboring regions can be selected based on index information to be signaled via the bitstream. Here, the index information can be used to specify the location / range of neighboring regions derived for MDM.

[0174] At least two of the methods described above for deriving MDM based on a first adjacent region, a second adjacent region, or a third adjacent region can be defined as individual modes in the encoding / decoding device. For ease of description, the method for deriving MDM based on a first adjacent region will be referred to as Mode A, the method for deriving MDM based on a second adjacent region will be referred to as Mode L, and the method for deriving MDM based on a third adjacent region will be referred to as Mode AL. Any of the predefined modes in the encoding / decoding device can be selectively used. The predefined modes can include at least two of Mode A, Mode L, or Mode AL.

[0175] For example, based on the size and / or shape of the current block, any of the predefined patterns can be selected. Here, size can be defined as width, height, the product of width and height, the maximum / minimum value of width and height, or the ratio of width to height. Alternatively, any of the predefined patterns can be selected based on index information to be signaled via the bitstream. Here, the index information can be used to specify the predefined pattern to be applied to the current block.

[0176] Target points within adjacent regions can belong to multiple sample rows within those adjacent regions. For example, ... Figure 9 As shown, a target point can belong to a sample row that is 1 sample away from the boundary of the current block 900 (hereinafter referred to as the first sample row) and a sample row that is 2 samples away from the current block (hereinafter referred to as the second sample row). Target points belonging to the first sample row are called group A (target point group A), and target points belonging to the second sample row are called group B (target point group B).

[0177] When calculating the intensity value (G) and / or direction of change (A) mentioned above, a predetermined weight can be applied based on the group to which the corresponding target point belongs. For example, a weight of 1 can be applied to the intensity value (G) calculated for a target point belonging to group A. In other words, the intensity value (G) calculated for a target point belonging to group A can be used as is. On the other hand, a weight less than 1 can be applied to the intensity value (G) calculated for a target point belonging to group B. In this case, when a weight of 0.5 is applied, the pre-calculated intensity value (G) can be updated to the value (0.5). G) and use it.

[0178] Deriving the structure of the MDM for the current block by applying vertical / horizontal mask kernels to adjacent regions can be computationally complex. Specifically, the number of target points can significantly impact computational complexity, as the direction of change (A) and intensity value (G) must be derived for each target point. Furthermore, a more accurate MDM can typically be derived with an increase in the number of target points. In the following text, for ease of description, the method for deriving the MDM based on the first adjacent region will be referred to as Mode A, the method for deriving the MDM based on the second adjacent region will be referred to as Mode L, and the method for deriving the MDM based on the third adjacent region will be referred to as Mode AL.

[0179] For pattern A, the MDM can be derived by defining more target points in the adjacent region above the current block. For pattern L, the MDM can be derived by defining more target points in the adjacent region to the left of the current block. Therefore, patterns A, L, and AL can be set to have the same / similar computational complexity.

[0180] For example, Figure 10 This is an example illustrating adjacent regions based on a pattern. Figure 10 (a) can represent the first neighboring region of pattern A. Figure 10 (b) can represent the second adjacent region of pattern L, and Figure 10 (c) can represent the third neighboring region of pattern AL.

[0181] like Figure 10 As shown, the number of target points belonging to the upper adjacent region of pattern A can be greater than the number of target points belonging to the upper adjacent region of pattern AL. The number of target points belonging to the left adjacent region of pattern L can be greater than the number of target points belonging to the left adjacent region of pattern AL.

[0182] The number of target points belonging to the first adjacent region of pattern A can be less than or equal to the number of target points belonging to the third adjacent region of pattern AL. The number of target points belonging to the second adjacent region of pattern L can be less than or equal to the number of target points belonging to the third adjacent region of pattern AL.

[0183] A target point belonging to the first adjacent region of pattern A can belong to multiple sample rows within the first adjacent region. These multiple sample rows can include at least one of the aforementioned first or second sample rows. Similarly, a target point belonging to the second adjacent region of pattern L can belong to multiple sample rows within the second adjacent region. These multiple sample rows can include at least one of the aforementioned first or second sample rows. On the other hand, a target point belonging to the third adjacent region of pattern AL can belong to one sample row within the third adjacent region. Here, one sample row can be either one of the first or second sample rows.

[0184] In Mode A, the length of the sample row containing the target point can be greater than or equal to the width of the current block. In Mode L, the length of the sample row containing the target point can be greater than or equal to the height of the current block. In Mode AL, the length of the sample row containing the target point can be greater than or equal to the sum of the width and height of the current block.

[0185] This disclosure proposes an efficient method for calculating the variables (parameters) required in the process of deriving MDM by applying vertical / horizontal mask kernels to adjacent regions.

[0186] Reference Figure 11 For each search point (e.g., P0 to P8) within the adjacent region of the current block 1100, the sample gradients (Gx) in the horizontal direction and the sample gradients (Gy) in the vertical direction can be derived. Intensity values ​​(G) and values ​​representing the direction of change (A) can be calculated based on the derived Gx and Gy. The probability distribution of the intra-prediction mode can be verified by accumulating the corresponding intensity values ​​within the intra-prediction mode determined based on the value of A.

[0187] Table 2 below shows the parameters required to determine the intensity value (G) for each intra-prediction mode (IPM).

[0188] [Table 2]

[0189] Referring to Table 2, IPM 66 is mapped to target points P0, P4, and P7, and the cumulative intensity value for IPM 66 is 79.18 (=45.25+22.62+11.31). IPM 48 is mapped to target points P1, P2, and P3, and the cumulative intensity value for IPM 48 is 42.48 (=6.71+8.94+26.83). IPM 22 is mapped to target points P5 and P6, and the cumulative intensity value for IPM 22 is 15.65 (=6.71+8.94). IPM 27 is mapped to target point P8, and the cumulative intensity value for IPM 27 is 12.65.

[0190] In this way, the cumulative intensity value for each intra-prediction mode can be calculated by applying a vertical / horizontal mask kernel to adjacent regions. The top N intra-prediction modes in descending order of cumulative intensity value can be derived as MDMs for the current block. Here, N can be an integer 1, 2, 3, or greater. For example, when N is 2, IPM 66 and IPM 48 in Table 2 can be derived as MDMs.

[0191] To reduce the computational complexity of Equation 26, the strength value (G) can be calculated as shown in Equation 27 below.

[0192] [Equation 27]

[0193] In Equation 27, abs(Gx) can represent the absolute value of Gx, and abs(Gy) can represent the absolute value of Gy. In the following text, for ease of description, abs(x) will be indicated as |x|.

[0194] Alternatively, in order to reduce the computational complexity of Equation 26 while maintaining accuracy, the intensity value (G) can be calculated based on the magnitude of the absolute values ​​of Gx and Gy.

[0195] For example, when |Gx| is relatively greater than |Gy|, the intensity value (G) can be set to |Gx|. When |Gx| is greater than K multiplied by |Gy|, it can be determined that |Gx| is relatively greater than |Gy|. On the other hand, when |Gy| is relatively greater than |Gx|, the intensity value (G) can be set to |Gy|. When |Gy| is greater than K multiplied by |Gx|, it can be determined that |Gy| is relatively greater than |Gx|. Here, K can be a constant greater than 1. As an example, K can be 2. When |Gx| is neither relatively greater than |Gy| nor relatively greater than |Gx|, the intensity value (G) can be set to the average of |Gx| and |Gy|.

[0196] Alternatively, when |Gx| is greater than K1 multiplied by |Gy|, the intensity value (G) can be set to |Gx|. When |Gx| is not greater than K1 multiplied by |Gy| but greater than K2 multiplied by |Gy|, the intensity value (G) can be set to |Gx| + 0.5. |Gy|. When |Gx| is not greater than K2 multiplied by |Gy| and |Gy| is less than K2 multiplied by |Gx|, the intensity value (G) can be set to the average of |Gx| and |Gy|. When |Gy| is not less than K2 multiplied by |Gx| but less than K1 multiplied by |Gx|, the intensity value (G) can be set to |Gy| + 0.5. |Gx|. When all the above conditions are not met, the intensity value (G) can be set to |Gy|.

[0197] The intra prediction mode to be determined (mapped) based on the value (A) representing the direction of change can be selected from among the intra prediction modes available for MDM derivation. Here, the available intra prediction modes can be all intra prediction modes (or directional modes) predefined in the encoding / decoding device. Alternatively, to reduce the computational complexity required to derive MDM, only some of the predefined intra prediction modes (or directional modes) in the encoding / decoding device can be defined as available intra prediction modes.

[0198] Specifically, when the number of predefined intra-prediction modes (or directional modes) in the encoding / decoding device is N, the available intra-prediction modes can be defined by uniformly sampling M of the N intra-prediction modes.

[0199] For example, among N intra-prediction modes, the mode with an even number (i.e., 2, 4, 6, ..., 66) can be defined as a usable intra-prediction mode. Alternatively, among N intra-prediction modes, the mode with an odd number (i.e., 1, 3, 5, ..., 65) can be defined as a usable intra-prediction mode.

[0200] Alternatively, among the N intra-prediction modes, a mode with a number corresponding to a multiple of K (e.g., K, 2K, 3K, etc.) can be defined as an available intra-prediction mode. Here, K can be a positive integer defined in the same way in both the encoding and decoding devices. As an example, K can be an integer 3, 4, or greater.

[0201] As mentioned above, when a uniformly sampled intra-prediction mode is defined as an available intra-prediction mode, it may be effective when sample gradients occur randomly in adjacent regions.

[0202] For N predefined intra-prediction modes (or directional modes) in the encoding / decoding device, M predefined intra-prediction modes can also be used as available intra-prediction modes.

[0203] For example, M intra-frame prediction modes can be defined for each frame. When the value of M is 5, 5 intra-frame prediction modes (such as {2, 18, 34, 50, 66}) can be used as available intra-frame prediction modes.

[0204] Alternatively, M intra-prediction modes can be defined for each slice. When the value of M is 10, 10 intra-prediction modes (such as {2, 8, 10, 18, 24, 28, 34, 42, 50, 66}) can be used as available intra-prediction modes.

[0205] Alternatively, M intra-prediction modes can be defined for each coding tree unit (CTU). When the value of M is 3, 3 intra-prediction modes (such as {18, 50, 66}) can be used as available intra-prediction modes.

[0206] In this way, when predefined and available intra-prediction modes are used, the computational complexity can be greatly reduced for images whose gradient changes based on samples in adjacent regions are concentrated on a specific intra-prediction mode.

[0207] Reference Figure 4 A prediction block S410 for the current block can be generated based on one or more MDMs.

[0208] For example, intra-frame prediction can be performed based on an MDM derived in S400 to generate a predicted block for the current block. Alternatively, when two MDMs are derived in S400, intra-frame prediction can be performed based on one of the MDMs to generate a first predicted block, and intra-frame prediction can be performed based on the other MDM to generate a second predicted block. The predicted block for the current block can be generated based on a weighted sum of the first and second predicted blocks. Alternatively, an intra-frame prediction mode for the current block can be derived based on at least one of the multiple MDMs to be derived in S400, and intra-frame prediction can be performed based on the derived intra-frame prediction mode to generate a predicted block for the current block.

[0209] Reference Figure 4 The current block S420 can be reconstructed based on the predicted block of the current block.

[0210] The current block can be reconstructed based on the predicted block and the residual block. The residual block can be generated by performing at least one of dequantization or inverse transform on the transform coefficients to be derived based on the residual information to be signaled via the bitstream.

[0211] Figure 12 An illustrative configuration of a decoding device 300 performing a decoding method according to the present disclosure is shown.

[0212] Reference Figure 12 The decoding device 300 may include an MDM deriver 1200, a prediction block generator 1210, and a reconstructor 1220. The MDM deriver 1200 and the prediction block generator 1210 may be equipped with… Figure 3 The intra-frame predictor 331.

[0213] The MDM deriver 1200 can perform the process of deriving the MDM according to S400. The prediction block generator 1210 can perform the process of generating the prediction block according to S410. The reconstructor 1220 can perform the process of reconstructing the current block according to S420.

[0214] Figure 13 An encoding method performed by the encoding device 200 as an embodiment of the present disclosure is illustrated.

[0215] One or more MDMS1300s can be derived by applying a predetermined mask kernel to the adjacent regions of the current block. Methods and references for deriving MDMs are provided. Figure 4 The description is the same.

[0216] The prediction block S1310 for the current block can be generated based on the MDM derived in S1300. The method for generating the prediction block is referenced. Figure 4 The descriptions are the same.

[0217] The transform coefficients S1320 of the current block can be derived based on the residual block of the current block. The residual block of the current block can be generated based on the prediction block generated in S1310. The transform coefficients can be derived by performing at least one of transformation or quantization on the residual block.

[0218] The residual information associated with the transform coefficients of the current block can be encoded to generate a bit stream S1330.

[0219] Figure 14 An illustrative configuration of an encoding device 200 performing an encoding method according to the present disclosure is shown.

[0220] Reference Figure 14 The encoding device 200 may include an MDM deriver 1400, a prediction block generator 1410, a transform coefficient deriver 1420, and a residual information encoder 1430.

[0221] The MDM deriver 1400 and the prediction block generator 1410 can be equipped with Figure 2 The intra-frame predictor 222. The transform coefficient derivator 1420 may be equipped with Figure 2 The residual processor 230. The residual information encoder 1430 may be equipped with an entropy encoder 240.

[0222] The MDM deriver 1400 can perform the process of deriving the MDM according to S1300. The prediction block generator 1410 can perform the process of generating prediction blocks according to S1310. The transform coefficient deriver 1420 can perform the process of deriving transform coefficients according to S1320. The residual information encoder 1430 can perform the process of encoding residual information according to S1330.

[0223] In the above embodiments, the method is described based on a flowchart as a series of steps or blocks. However, the corresponding embodiments are not limited to this order of steps. Some steps may occur simultaneously with other steps or in a different order, as described above. In addition, those skilled in the art will understand that the steps shown in the flowchart are not exclusive. Other steps may be included, or one or more steps in the flowchart may be deleted, without affecting the scope of the embodiments of this disclosure.

[0224] The methods described above according to embodiments of the present disclosure can be implemented in software, and the encoding and / or decoding devices according to the present disclosure can be included in an apparatus for performing image processing, such as a TV, computer, smartphone, set-top box, display device, etc.

[0225] In this disclosure, when the implementation is implemented as software, the above-described method can be implemented as a module (process, function, etc.) performing the above-described functions. The module can be stored in memory and can be executed by a processor. The memory can be internal or external to the processor and can be connected to the processor by various well-known means. The processor may include an application-specific integrated circuit (ASIC), another chipset, logic circuitry, and / or data processing devices. The memory may include read-only memory (ROM), random access memory (RAM), flash memory, memory cards, storage media, and / or other storage devices. In other words, the implementations described herein can be executed by implementation on a processor, microprocessor, controller, or chip. For example, the functional units shown in the various figures can be executed by implementation on a computer, processor, microprocessor, controller, or chip. In this case, information for implementation (e.g., information about instructions) or algorithms can be stored in a digital storage medium.

[0226] Furthermore, decoding and encoding devices employing embodiments of this disclosure can be included in multimedia broadcasting transmitting and receiving devices, mobile communication terminals, home theater video devices, digital cinema video devices, surveillance cameras, video conferencing devices, real-time communication devices similar to video communication, mobile streaming devices, storage media, cameras, devices for providing video-on-demand (VOD) services, OTT (over-the-top) video devices, devices for providing internet streaming services, three-dimensional (3D) video devices, virtual reality (VR) devices, augmented reality (AR) devices, video telephony devices, transportation terminals (e.g., vehicle (including autonomous vehicle) terminals, aircraft terminals, ship terminals, etc.), and medical video devices, and can be used to process video signals or data signals. For example, OTT (over-the-top) video devices can include game consoles, Blu-ray players, internet-connected televisions, home theater systems, smartphones, tablet PCs, digital video recorders (DVRs), etc.

[0227] Furthermore, the processing methods applying the embodiments of this disclosure can be generated in the form of a computer-executable program and can be stored in a computer-readable recording medium. Multimedia data with data structures according to the embodiments of this disclosure can also be stored in a computer-readable recording medium. Computer-readable recording media include all types of storage devices and distributed storage devices that store computer-readable data. Computer-readable recording media can include, for example, Blu-ray discs (BD), Universal Serial Bus (USB), ROM, PROM, EPROM, EEPROM, RAM, CD-ROM, magnetic tape, floppy disks, and optical media storage devices. Additionally, computer-readable recording media include media implemented in the form of carrier waves (e.g., transmission via the Internet). Furthermore, bitstreams generated by encoding methods can be stored in computer-readable recording media or transmitted via wired / wireless communication networks.

[0228] Furthermore, the embodiments of this disclosure can be implemented by a computer program product using program code, and the program code can be executed on a computer using the embodiments of this disclosure. The program code can be stored on a computer-readable medium.

[0229] Figure 15 An example of a content streaming system to which embodiments of the present disclosure can be applied is shown.

[0230] Reference Figure 15 A content streaming system that applies embodiments of the present disclosure may generally include an encoding server, a streaming server, a network server, a media storage device, a user device, and a multimedia input device.

[0231] An encoding server generates a bitstream by compressing content input from multimedia input devices such as smartphones, cameras, and camcorders into digital data and then sends it to a streaming server. As another example, when multimedia input devices such as smartphones, cameras, and camcorders directly generate bitstreams, the encoding server can be omitted.

[0232] A bitstream can be generated by an encoding method or bitstream generation method that applies the embodiments of this disclosure, and the streaming server can temporarily store the bitstream during the sending or receiving of the bitstream.

[0233] A streaming server sends multimedia data to a user device via a web server based on a user request, and the web server acts as a medium to inform the user what services are available. When a user requests a desired service from the web server, the web server transmits the request to the streaming server, and the streaming server sends the multimedia data to the user. In this scenario, the content streaming system may include a separate control server, which controls the commands / responses between the various devices in the content streaming system.

[0234] A streaming server can receive content from media storage and / or encoding servers. For example, when receiving content from an encoding server, the content can be received in real time. In this case, to provide a smooth streaming service, the streaming server can store the bitstream for a specific time period.

[0235] Examples of user devices may include mobile phones, smartphones, laptops, digital broadcasting terminals, personal digital assistants (PDAs), portable multimedia players (PMPs), navigators, touchscreen PCs, tablet PCs, ultrabooks, wearable devices (e.g., smartwatches, smart glasses, head-mounted displays (HMDs)), digital TVs, desktop computers, digital signage, etc.

[0236] In a content streaming system, each server can operate as a distributed server, and in this case, the data received from each server can be distributed and processed.

[0237] The claims set forth herein can be combined in various ways. For example, the technical features of the method claims of this disclosure can be combined and implemented as an apparatus, and the technical features of the apparatus claims of this disclosure can be combined and implemented as a method. Furthermore, the technical features of the method claims and the apparatus claims of this disclosure can be combined and implemented as an apparatus, and the technical features of the method claims and the apparatus claims of this disclosure can be combined and implemented as a method.

Claims

1. A method comprising the following steps: One or more primary intra-prediction modes (MDMs) are derived by applying a predetermined mask kernel to the adjacent regions of the current block. A prediction block for the current block is generated based on one or more MDMs; as well as The current block is reconstructed based on the predicted block.

2. The method according to claim 1, wherein, Any one of the coefficients in the mask kernel is 1, 3, 47, or 48, and Among the coefficients of the mask kernel, another coefficient is 2, 10, 162, or 160.

3. The method according to claim 1, wherein, The mask kernel is any one of a predefined plurality of mask kernels, and The plurality of mask kernels include at least one of a first mask kernel having coefficients 1 and 2, a second mask kernel having coefficients 3 and 10, a third mask kernel having coefficients 47 and 162, or a fourth mask kernel having coefficients 48 and 160.

4. The method according to claim 3, wherein, The mask kernel is determined to be any one of the plurality of mask kernels based on at least one of the size or shape of the current block.

5. The method according to claim 1, wherein, The adjacent region is at least one of the following: a first adjacent region including the upper adjacent region of the current block, a second adjacent region including the left adjacent region of the current block, or a third adjacent region including both the upper adjacent region and the left adjacent region of the current block.

6. The method according to claim 5, wherein, Based on at least one of the size or shape of the current block, the adjacent region is determined to be any one of the first adjacent region to the third adjacent region.

7. The method according to claim 1, wherein, The steps for deriving one or more MDMs include: Derive the sample gradient at the target point where the mask kernel is applied in the adjacent region, wherein the sample gradient includes at least one of the sample gradient in the horizontal direction or the sample gradient in the vertical direction; Determine the intra-prediction mode to be mapped to the sample gradient; and The intensity value of the intra-prediction mode determined based on the sample gradient is calculated.

8. The method according to claim 7, wherein, A predetermined weight is applied to the intensity value based on the position of the sample row to which the target point belongs within the adjacent region.

9. The method according to claim 7, wherein, The intensity value is calculated based on at least one of the absolute value of the sample gradient in the horizontal direction or the absolute value of the sample gradient in the vertical direction.

10. The method according to claim 7, wherein, The intra-prediction mode to be mapped to the sample gradient is determined in an intra-prediction mode that can be used to derive the MDM.

11. The method according to claim 10, wherein, Available intra-prediction modes are those with even or odd numbers from among N predefined intra-prediction modes.

12. The method according to claim 10, wherein, The available intra-prediction modes are M predefined intra-prediction modes for each frame, slice, or coding tree unit.

13. A method comprising the following steps: One or more primary intra-prediction modes (MDMs) are derived by applying a predetermined mask kernel to the adjacent regions of the current block. A prediction block for the current block is generated based on one or more MDMs; Generate a residual block for the current block based on the predicted block of the current block; The transformation coefficients of the current block are derived based on the residual block; as well as The residual information associated with the transformation coefficients is encoded.

14. A computer-readable storage medium storing a bit stream generated by the method according to claim 13.

15. A method comprising the following steps: Obtain a bitstream of image information, wherein the bitstream is generated based on the following steps: deriving one or more dominant intra-prediction modes (MDMs) by applying a predetermined mask kernel to the adjacent regions of the current block; generating a prediction block of the current block based on the one or more MDMs; generating a residual block of the current block based on the prediction block of the current block; deriving the transform coefficients of the current block based on the residual block; and encoding residual information associated with the transform coefficients; and Send data including the bit stream.

Images