Methods for encoding / decoding images and recording media for storing bit streams

By segmenting and encoding video blocks using diagonal segmentation technology, the problem of low compression efficiency in high-resolution video is solved, and the compression efficiency of inter-frame prediction and data storage capacity are improved.

CN117376558BActive Publication Date: 2026-11-10SAMSUNG ELECTRONICS CO LTD
View PDF -1 Cites 0 Cited by

Patent Information

Application Number
CN202311425458.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2017-10-16
Filing Date
2018-10-16
Publication Date
2026-11-10
Estimated Expiration
2038-10-16

Smart Images

  • Figure CN117376558B_ABST
    Figure CN117376558B_ABST
Patent Text Reader

Abstract

Methods for encoding / decoding images and recording media for storing bitstreams are disclosed. One method for decoding an image includes: segmenting a higher-level coded block in an image into multiple coded blocks based on tree-based block segmentation; segmenting a current block into a first partition and a second partition based on geometric motion partitioning, the current block being one of multiple coded blocks; obtaining a first predicted block of the current block based on motion information of the first partition; obtaining a second predicted block of the current block based on motion information of the second partition; obtaining a final predicted block of the current block by performing a weighted sum of the first and second predicted blocks; reconstructing the current block based on the final predicted block, wherein geometric motion partitioning is performed based on segmentation information indicating the direction of the segmentation lines that segment the current block; at least one of the first and second partitions is triangular in shape; and geometric motion partitioning is applied to the current block only when the current block does not perform sub-block-based inter-frame prediction.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] This application is a divisional application of Chinese Patent Application No. 201880067259.4, filed on October 16, 2018, entitled "Method, Apparatus and Recording Medium for Encoding / Decoding Images and Storing Bit Streams". Technical Field

[0002] This invention relates to image encoding / decoding methods and apparatus. Specifically, this invention relates to image encoding / decoding methods, apparatus for using diagonal segmentation in inter-frame prediction of block structures using combinations of various block segmentation types, and recording media for storing bitstreams generated by the image encoding methods / apparatus of this invention. Background Technology

[0003] The market demand for high-resolution video is increasing, thus requiring technologies capable of effectively compressing high-resolution video. In response to these market demands, the ISO / IEC Moving Picture Experts Group (MPEG) and the ITU-T Video Coding Experts Group (VCEG) jointly established the JCT-VC (Joint Collaboration on Video Coding), completing the development of the HEVC (High Efficiency Video Compression Coding) video compression standard in January 2013, and are actively researching and developing next-generation compression standards.

[0004] Video compression primarily consists of in-frame prediction (or intra-frame prediction), inter-frame prediction (or inter-frame prediction), transform, quantization, entropy coding, and intra-loop filtering. Simultaneously, with the increasing demand for high-resolution video, the demand for stereoscopic video content as a new video service is also increasing. Discussions are underway regarding video compression techniques for effectively providing high-resolution and ultra-high-resolution stereoscopic image content. Summary of the Invention

[0005] Technical issues

[0006] The purpose of this invention is to provide an image encoding / decoding method with improved compression efficiency and a recording medium for storing the bitstream generated by the image encoding method / apparatus of this invention.

[0007] Furthermore, the object of the present invention is to provide an image encoding / decoding method using inter-frame prediction with improved compression efficiency, and a recording medium for storing the bitstream generated by the image encoding method / apparatus of the present invention.

[0008] Furthermore, the object of the present invention is to provide an image encoding / decoding method that efficiently performs prediction using diagonal partitioning on combined block structures employing various block partitioning types, and a recording medium for storing the bitstream generated by the image encoding method / apparatus of the present invention.

[0009] Technical solution

[0010] The image decoding method according to the present invention may include: obtaining diagonal segmentation information for a current block; using the diagonal segmentation information to determine the diagonal segmentation structure of the current block; and performing diagonal segmentation according to the determined diagonal segmentation structure to diagonally divide the current block into a first region and a second region, wherein the current block may be a leaf node of a square segmentation or a non-square segmentation.

[0011] In the image decoding method according to the present invention, the diagonal segmentation information may include a flag indicating whether the current block is segmented along the diagonal.

[0012] In the image decoding method according to the present invention, the diagonal segmentation can be determined as either a lower right diagonal segmentation or an upper right diagonal segmentation based on the direction of the diagonal segmentation.

[0013] In the image decoding method according to the present invention, the diagonal segmentation information may include segmentation direction information indicating the direction of diagonal segmentation, and the diagonal segmentation may be performed by diagonally segmenting the current block in the direction indicated by the segmentation direction information.

[0014] In the image decoding method according to the present invention, performing diagonal segmentation may include: performing a masking operation on the current block, wherein the masking operation may be a predetermined weighted operation performed on samples of the current block.

[0015] In the image decoding method according to the present invention, the current block may be composed of a first sample, a second sample, and a third sample. The first sample is composed of samples in one direction based on the boundary line, which are included in the region not crossed by the diagonally divided boundary line. The second sample is composed of samples crossed by the diagonally divided boundary line. The third sample is composed of samples in the current block that do not belong to the first sample or the second sample. The masking operation on the first region may be: applying a first value to the first sample, applying a second value to the second sample, and applying a third value to the third sample, respectively. The masking operation on the second region may be: applying a third value to the first sample, applying a second value to the second sample, and applying a third value to the third sample, respectively.

[0016] In the image decoding method according to the present invention, when f(x, y) represents the diagonally segmented boundary line, the masking operation on the first region and the masking operation on the second region can satisfy the following equation 1.

[0017] [Formula 1]

[0018]

[0019] Here, MASK_p0(x, y) can represent the mask value of the first region, and MASK_p1(x, y) can represent the mask value of the second region.

[0020] In the image decoding method according to the present invention, the sample values ​​of the current block can satisfy the following equation 2.

[0021] [Equation 2]

[0022] P DMP (x,y)=(P P0 (x, y) × MASK P0 (x, y) + P P1 (x, y) × MASK P1 (x,y)>>shit

[0023] Here, P_DMP(x, y) can represent a specific sample value of the current block, P_p0(x, y) can represent the sample value of the sample belonging to the first region, P_p1(x, y) can represent the sample value of the sample belonging to the second region, and shift can represent the scaling value according to the mask operation.

[0024] In the image decoding method according to the present invention, the square segmentation or non-square segmentation can be one of quadtree segmentation, binary tree segmentation and ternary tree segmentation.

[0025] The image encoding method according to the present invention may include: determining the diagonal segmentation structure of a current block; performing diagonal segmentation according to the determined diagonal segmentation structure to diagonally segment the current block into a first region and a second region; and encoding diagonal segmentation information for the diagonal segmentation structure, wherein the current block may be a leaf node of a square segmentation or a non-square segmentation.

[0026] The image encoding method according to the present invention may further include encoding a flag indicating whether the current block is diagonally divided.

[0027] In the image encoding method according to the present invention, the diagonal segmentation can be determined as either a lower right diagonal segmentation or an upper right diagonal segmentation based on the direction of the diagonal segmentation.

[0028] The image encoding method according to the present invention may further include encoding segmentation direction information indicating the direction of diagonal segmentation.

[0029] In the image encoding method according to the present invention, performing diagonal segmentation may include: performing a masking operation on the current block, wherein the masking operation may be a predetermined weighting operation performed on samples of the current block.

[0030] In the image encoding method according to the present invention, the current block may be composed of a first sample, a second sample, and a third sample. The first sample is composed of samples in one direction based on the boundary line, which are included in the region not crossed by the diagonally divided boundary line. The second sample is composed of samples crossed by the diagonally divided boundary line. The third sample is composed of samples in the current block that do not belong to the first sample or the second sample. The masking operation for the first region may be: applying a first value to the first sample, applying a second value to the second sample, and applying a third value to the third sample. The masking operation for the second region may be: applying a third value to the first sample, applying a second value to the second sample, and applying a third value to the third sample.

[0031] In the image encoding method according to the present invention, when f(x, y) represents the diagonally segmented boundary line, the masking operation for the first region and the masking operation for the second region can satisfy the following equation 3.

[0032] [Formula 3]

[0033]

[0034] Here, MASK_p0(x, y) can represent the mask value of the first region, and MASK_p1(x, y) can represent the mask value of the second region.

[0035] In the image encoding method according to the present invention, the sample values ​​of the current block can satisfy the following equation 4.

[0036] [Formula 4]

[0037] P DMP (x,y)=(P P0 (x, y) × MASK P0 (x, y) + P P1 (x, y) × MASK P1 (x,y)>>shift

[0038] Here, P_DMP(x, y) can represent a specific sample value of the current block, P_p0(x, y) can represent the sample value of the sample belonging to the first region, P_p1(x, y) can represent the sample value of the sample belonging to the second region, and shift can represent the scaling value according to the mask.

[0039] In the image encoding method according to the present invention, the square segmentation or non-square segmentation can be one of quadtree segmentation, binary tree segmentation and ternary tree segmentation.

[0040] A non-transient storage medium for storing bitstreams generated by an image encoding method, wherein the image encoding method may include: determining a diagonal partitioning structure of a current block; performing diagonal partitioning according to the determined diagonal partitioning structure to diagonally partition the current block into a first region and a second region; and encoding diagonal partitioning information for the diagonal partitioning structure, wherein the current block may be a leaf node of a square partition or a non-square partition.

[0041] Beneficial effects

[0042] According to the present invention, an image encoding / decoding method and apparatus with improved compression efficiency, as well as a recording medium for storing the bitstream generated by the image encoding method / apparatus of the present invention, can be provided.

[0043] Furthermore, according to the present invention, an image encoding / decoding method using inter-frame prediction with improved compression efficiency and a recording medium for storing the bitstream generated by the image encoding method / apparatus of the present invention can be provided.

[0044] Furthermore, according to the present invention, an image encoding / decoding method and apparatus that uses diagonal segmentation to efficiently perform inter-frame prediction on block structures using combinations of various block segmentation types, as well as a recording medium for storing the bitstream generated by the image encoding method / apparatus of the present invention, can be provided. Attached Figure Description

[0045] Figure 1 This is a block diagram illustrating the configuration of an embodiment of the encoding device according to which the present invention is applied.

[0046] Figure 2 This is a block diagram illustrating the configuration of an embodiment of the decoding device according to which the present invention is applied.

[0047] Figure 3 It is a diagram used to illustrate the image segmentation structure during image encoding and decoding.

[0048] Figure 4 This is a diagram used to illustrate the geometrical motion division (GEO).

[0049] Figure 5 This is a flowchart illustrating an image decoding method according to an embodiment of the present invention.

[0050] Figure 6 This is a diagram illustrating an image encoding method according to an embodiment of the present invention.

[0051] Figure 7 This is another diagram illustrating an image decoding method according to an embodiment of the present invention.

[0052] Figure 8This is another diagram used to illustrate that diagonal division according to embodiments of the present invention can represent various arbitrary division types.

[0053] Figure 9 This is another figure illustrating a method for determining the direction of diagonal division according to an embodiment of the present invention.

[0054] Figure 10 This is a view used to illustrate the masking operation method according to an embodiment of the present invention.

[0055] Figure 11 This is a view used to illustrate a masking operation method according to another embodiment of the present invention.

[0056] Figure 12 This is a diagram used to illustrate the syntax parameters according to an embodiment of the present invention. Detailed Implementation

[0057] In the following, embodiments of the invention will be described in detail with reference to the accompanying drawings, enabling those skilled in the art to readily implement the invention. However, the invention can be implemented in many different forms and should not be construed as limited to the embodiments set forth herein. For clarity of description, parts irrelevant to the invention have been omitted, and similar parts are indicated by similar reference numerals throughout the specification.

[0058] Throughout the specification, when a claim is made that one part is “connected” to another part, examples are included not only of direct connections between the two parts, but also of electrical connections between the part and another component therebetween.

[0059] Furthermore, when a part of the specification claims to "include" an element throughout the specification, it should be understood that, unless otherwise specifically stated, that element may also include other elements without departing from the other elements.

[0060] Additionally, terms such as "first," "second," etc., may be used to describe various components, but components should not be limited by these terms. These terms are used only for the purpose of distinguishing one component from another.

[0061] Furthermore, in embodiments of the apparatus and method described herein, some components of the apparatus or some steps of the method may be omitted. Additionally, the order of some components of the apparatus or the order of some steps of the method may be changed. Moreover, other components or steps may be inserted into some components of the apparatus or some steps of the method.

[0062] Furthermore, some components or steps of the first embodiment of the present invention may be added to the second embodiment of the present invention, or may replace some components or steps of the second embodiment.

[0063] Furthermore, the components shown in the embodiments of the present invention are illustrated independently to indicate different functional features, and do not imply that each component consists of a separate hardware or software component unit. That is, for ease of description, each component is listed as a single component, and at least two components of each component can be combined to form a single component, or a single component can be divided into multiple components to perform functions. Embodiments of integration and separation of each of these components are also included within the scope of the present invention without departing from its spirit.

[0064] First, the terminology used in this application will be briefly described below.

[0065] The decoding device (video decoding device) described later can be a civilian security camera device, a civilian security system, a military security camera device, a military security system, a personal computer (PC), a laptop computer, a portable multimedia player (PMP), a wireless communication terminal, a smartphone, a device included in a server terminal such as a TV application server and a service server, and can represent a user terminal such as various devices, a communication device such as a modem for performing communication with a wired / wireless communication network, a memory for storing various programs and data for decoding images or performing inter-frame prediction or intra-frame prediction for decoding, and various devices equipped with a microprocessor for executing programs and calculating and controlling programs, etc.

[0066] Additionally, the image encoded into a bitstream by the encoder can be transmitted to an image decoding device via real-time or non-real-time wired / wireless communication networks (e.g., the Internet, local area wireless communication networks, wireless LAN networks, Wi-Fi networks, mobile communication networks), or via various communication interfaces such as cables or Universal Serial Bus (USB), where it is decoded, reconstructed into an image, and reproduced. Alternatively, the bitstream generated by the encoder can be stored in memory. Memory may include both volatile and non-volatile memory. In this specification, memory may be referred to as a recording medium that stores the bitstream.

[0067] Typically, a video can consist of a series of images, and each image can be divided into coding units such as blocks. Furthermore, those skilled in the art to which this embodiment pertains will understand that the term "image" described below can be used by replacing it with another term having an equivalent meaning, such as "picture" or "frame." Additionally, those skilled in the art will understand that the term "coding unit" can be replaced and used with other terms having the same meaning, such as "unit block" and "block."

[0068] In the following description, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. Redundant descriptions of identical components will be omitted in the description of the present invention.

[0069] Figure 1 This is a block diagram illustrating an image encoding apparatus according to the present invention. (Refer to...) Figure 1 A conventional image encoding device 100 may include an image segmentation unit 110, prediction units 120 and 125, a transformation unit 130, a quantization unit 135, a reordering unit 160, an entropy encoding unit 165, an inverse quantization unit 140, an inverse transformation unit 145, a filter unit 150, and a memory 155.

[0070] Image segmentation unit 110 can segment an input image into at least one processing unit. In this case, the processing unit may be a prediction unit (PU), a transform unit (TU), or an encoding unit (CU). In the following, in embodiments of the present invention, the encoding unit may be interpreted as either a unit that performs encoding or a unit that performs decoding.

[0071] A prediction unit can be segmented into at least one square or non-square shape of the same size within a coding unit, or it can be segmented such that any prediction unit segmented within a coding unit has a different shape and / or size than another prediction unit. When generating prediction units that perform intra-frame prediction based on coding units, if the prediction unit is not the smallest coding unit, intra-frame prediction can be performed without segmenting it into multiple prediction units N×N.

[0072] Prediction units 120 and 125 may include an inter-frame prediction unit 120 performing inter-frame or inter-frame prediction and an intra-frame prediction unit 125 performing intra-frame or intra-frame prediction. It can be determined whether inter-frame or intra-frame prediction is used for the prediction unit, and specific information (e.g., intra-frame prediction mode, motion vector, reference image, etc.) based on each prediction method can be determined. The residual value (residual block) between the generated prediction block and the original block can be input to the transform unit 130. Furthermore, prediction mode information, motion vector information, etc., used for prediction can be encoded by the entropy coding unit 165 along with the residual value and sent to the decoder. However, when the motion information derivation technique on the decoder side according to the present invention is applied, since the prediction mode information, motion vector information, etc., are not generated in the encoder, the corresponding information is not sent to the decoder. On the other hand, information indicating the derivation and use of motion information from the decoder side and information about the technique used to derive motion information from the encoder can be transmitted by signal transmission.

[0073] The inter-frame prediction unit 120 can predict prediction units based on information from at least one of the previous or subsequent images of the current image, and in some cases, it can predict prediction units based on information from some encoded regions in the current image. The inter-frame prediction unit 120 may include a reference image interpolation unit, a motion prediction unit, and a motion compensation unit.

[0074] The reference image interpolator can receive reference image information from memory 155 and generate integer pixel or fewer pixel information from the reference image. In the case of luminance pixels, an 8-tap DCT-based interpolation filter with different filter coefficients can be used to generate integer pixel or fewer pixel information in units of 1 / 4 pixel. In the case of chrominance signals, a DCT-based interpolation filter with different filter coefficients can be used to generate integer pixel or fewer pixel information in units of 1 / 8 pixel.

[0075] The motion prediction unit can perform motion prediction based on a reference image interpolated by the reference image interpolation unit. Various methods can be used to compute motion vectors, such as full search-based block matching (FBMA), three-step search (TSS), and novel three-step search (NTS). Motion vectors can have values ​​in units of 1 / 2 or 1 / 4 pixels based on the interpolated pixels. The motion prediction unit can predict the current prediction unit by using different motion prediction methods. Various methods can be used as motion prediction methods, such as skipping methods, merging methods, advanced motion vector prediction (AMVP) methods, and intra-block copying methods. Furthermore, when applying the motion information derivation technique on the decoder side according to the present invention, template matching methods using motion trajectories and bilateral matching methods can be applied as methods performed by the motion prediction unit. This will be discussed later. Figure 3 The template matching method and the bilateral matching method are described in detail.

[0076] Intra-prediction unit 125 can generate prediction units based on reference pixel information surrounding the current block, where the reference pixel information is pixel information in the current image. If the neighboring block of the current prediction unit is a block that has already undergone inter-frame prediction and the reference pixel is a pixel that has already undergone inter-frame prediction, the reference pixel information of the neighboring block that has already undergone inter-frame prediction can be used to replace the reference pixel included in the block that has already undergone inter-frame prediction. That is, when a reference pixel is unavailable, the information of the unavailable reference pixel can be replaced with at least one of the available reference pixels.

[0077] Additionally, residual blocks including residual information can be generated. The residual information is the difference between the original block of the prediction unit that performs the prediction based on the prediction units generated by prediction units 120 and 125. The generated residual blocks can be input to the transformation unit 130.

[0078] Transform unit 130 uses transformation methods such as DCT (Discrete Cosine Transform), DST (Discrete Sine Transform), and KLT to transform the residual block, which includes residual information from the original block and the prediction units generated by prediction units 120 and 125. The choice between applying DCT, DST, or KLT to transform the residual block can be determined based on information about the intra-prediction mode of the prediction unit used to generate the residual block.

[0079] The quantization unit 135 can quantize the values ​​converted from the transform unit 130 to the frequency domain. The quantization coefficients can vary depending on the blocks or importance of the image. The values ​​calculated by the quantization unit 135 can be provided to the inverse quantization unit 140 and the reordering unit 160.

[0080] The reordering unit 160 can reorder the coefficient values ​​associated with the quantized residual values.

[0081] The reordering unit 160 can transform 2D block shape coefficients into 1D vector form using a coefficient scanning method. For example, the reordering unit 160 can use a Zig-Zag scanning method to scan DC coefficients into coefficients in the high-frequency region and transform them into a one-dimensional vector form. Depending on the size of the transform unit and the intra-frame prediction mode, a vertical scan that scans the 2D block shape coefficients in the column direction and a horizontal scan that scans the 2D block shape coefficients in the row direction can be used instead of a zig-shaped scan. That is, the choice between a zig-shaped scan, a vertical scan, and a horizontal scan can be determined based on the size of the transform unit and the intra-frame prediction mode.

[0082] Entropy coding unit 165 can perform entropy coding based on the values ​​calculated by reordering unit 160. Entropy coding can use various coding methods, such as Exponential Golomb codes, CAVLC (Context Adaptive Variable Length Coding), and CABAC (Context Adaptive Binary Arithmetic Coding). In this regard, entropy coding unit 165 can encode the residual coefficient information from the coding units of reordering unit 160 and prediction units 120 and 125. Furthermore, according to the present invention, information indicating the extraction and use of motion information on the decoder side, as well as information about the techniques used to extract the motion information, can be transmitted via signals.

[0083] Inverse quantization unit 140 and inverse transform unit 145 perform inverse quantization on the value quantized by quantization unit 135 and inverse transform on the value transformed by transform unit 130. A reconstruction block can be generated by adding the residual value generated by inverse quantization unit 140 and inverse transform unit 145 to the prediction unit predicted by the motion estimation unit, motion compensation unit, and intra-frame prediction unit included in prediction units 120 and 125.

[0084] Filter unit 150 may include at least one of a deblocking filter, an offset correction unit, and an adaptive loop filter (ALF). The deblocking filter removes block distortion caused by boundaries between blocks in the reconstructed image. The offset correction unit corrects the offset relative to the original image on a pixel-by-pixel basis in the deblocked image. To perform offset correction on a specific image, after segmenting the pixels included in the image into a specific number of regions, a method can be used to determine the regions to be offset and apply the offset to the corresponding regions, or a method can be used to apply the offset considering the edge information of each pixel. The ALF (Adaptive Loop Filter) can be performed based on values ​​obtained by comparing the filtered reconstructed image with the original image. After segmenting the pixels included in the image into predetermined groups, a filter to be applied to the corresponding group can be determined to perform differential filtering on each group.

[0085] The memory 155 can store the reconstructed blocks or images calculated by the filter unit 150, and can provide the stored reconstructed blocks or images to the predictors 120 and 125 when performing inter-frame prediction.

[0086] Figure 2 This is a block diagram illustrating an image decoding apparatus according to the present invention. (Refer to...) Figure 2 The image decoder 200 may include an entropy decoding unit 210, a reordering unit 215, an inverse quantization unit 220, an inverse transform unit 225, a prediction unit 230, 235, a filter unit 240, and a memory 245.

[0087] When an image bitstream is input from an image encoder, the input bitstream can be decoded in the reverse process of the image encoder.

[0088] The entropy decoding unit 210 can perform entropy decoding in a process that is the reverse of the process performed by entropy encoding in the entropy encoding unit of the image encoder. For example, various methods such as exponential Golomb codes, context-adaptive variable-length coding (CAVLC), and context-adaptive binary arithmetic coding (CABAC) can be applied corresponding to the methods performed in the image encoder.

[0089] The entropy decoding unit 210 can decode information related to intra-frame prediction and inter-frame prediction performed by the encoder.

[0090] The reordering unit 215 can perform reordering based on a method of reordering the bitstream entropy-decoded by the entropy decoding unit 210 in the encoding unit. The coefficients expressed in the form of a one-dimensional vector can be reconstructed into coefficients in the form of two-dimensional blocks and rearranged.

[0091] The inverse quantization unit 220 can perform inverse quantization based on the quantization parameters provided by the encoder and the coefficient values ​​of the rearranged blocks.

[0092] The inverse transform unit 225 can perform inverse transforms (i.e., inverse DCT, inverse DST, and inverse KLT) on the transforms (i.e., DCT, DST, and KLT) performed by the transform unit on the quantization result executed by the image encoder. The inverse transform can be performed based on the transmission unit determined by the image encoder. In the inverse transform unit 225 of the image decoder, transform methods (e.g., DCT, DST, KLT) can be selectively performed based on multiple pieces of information, such as the prediction method, the size of the current block, and the prediction direction.

[0093] Prediction units 230 and 235 can generate prediction blocks based on information about prediction block generation provided by entropy decoding unit 210 and previously reconstructed block or image information provided by memory 245.

[0094] As described above, when intra-prediction is performed in the same manner as in the image encoder, if the size of the prediction unit is the same as the size of the transform unit, intra-prediction can be performed on the prediction unit based on the pixels to its left, upper left, and upper sides. However, if the size of the prediction unit is different from the size of the transform unit, intra-prediction can be performed using reference pixels based on the transform unit. Furthermore, intra-prediction using N×N segmentation can be used only for the smallest decoding unit.

[0095] Prediction units 230 and 235 may include a prediction unit determination unit, an inter-frame prediction unit, and an intra-frame prediction unit. The prediction unit determination unit may receive various information input from the entropy decoding unit 210, such as prediction unit information, prediction mode information of the intra-frame prediction method, and motion prediction-related information of the inter-frame prediction method, classify the prediction units from the current coding unit, and determine whether the prediction unit performs inter-frame prediction or intra-frame prediction. On the other hand, if the encoder 100 does not send motion prediction-related information for inter-frame prediction, but instead sends information indicating that motion information is derived and used on the decoder side, as well as information about the techniques used to derive the motion information, the prediction unit determination unit determines the prediction performance of the inter-frame prediction unit 23 based on the information sent from the encoder 100.

[0096] Inter-frame predictor 230 can use the information required for inter-frame prediction of the current prediction unit provided by the image encoder, based on information included in at least one of the previous or subsequent images of the current image including the current prediction unit, to perform inter-frame prediction for the current prediction unit. To perform inter-frame prediction, the motion prediction method of the prediction unit included in the corresponding coding unit can be determined based on the coding unit: skip mode, merge mode, AMVP mode, or intra-block copy mode. Alternatively, inter-frame prediction unit 230 can perform inter-frame prediction by deriving motion information itself from the decoder side according to information provided by the image encoder indicating that the decoder derives and uses motion information, as well as information about the technique used to derive the motion information.

[0097] Intra-prediction unit 235 can generate prediction blocks based on pixel information in the current image. When the prediction unit is one that has already performed intra-prediction, intra-prediction can be performed based on the intra-prediction mode information of the prediction unit provided by the image encoder. Intra-prediction unit 235 may include an adaptive intra-smoothing (AIS) filter, a reference pixel interpolation unit, and a DC filter. The AIS filter is the part that performs filtering on the reference pixels of the current block, and the filter can be applied by determining whether to apply the AIS filter based on the prediction mode of the current prediction unit. AIS filtering can be performed on the reference pixels of the current block using the AIS filter information and prediction mode of the prediction unit provided by the image encoder. When the prediction mode of the current block is a mode in which AIS filtering is not performed, the AIS filter may not be applied.

[0098] When the prediction mode of the prediction unit is to perform intra-frame prediction based on the pixel values ​​of interpolated reference pixels, the reference pixel interpolation unit can interpolate reference pixels to generate reference pixels with integer values ​​or fewer pixel units. If the prediction mode of the current prediction unit is to generate prediction blocks without interpolating reference pixels, then interpolation of reference pixels is not necessary. When the prediction mode of the current block is DC mode, the DC filter can generate prediction blocks through filtering.

[0099] The reconstructed blocks or images can be provided to filter unit 240. Filter unit 240 may include a deblocking filter, an offset correction unit, and an ALF.

[0100] The image encoder can provide information about whether to apply a deblocking filter to a corresponding block or image, and whether to apply a strong or weak filter. The image decoder's deblocking filter can provide information related to the deblocking filter provided by the image encoder, and the image decoder can perform deblocking filtering on the corresponding block.

[0101] The offset correction unit can perform offset correction on the reconstructed image based on the type and offset value information of the offset correction applied to the image during encoding. The decoding unit can apply ALF based on ALF application information, ALF coefficient information, etc., provided by the encoder. This ALF information can be provided by including it in a specific parameter set.

[0102] The memory 245 can store the reconstructed image or block, so that the reconstructed image or block can be used as a reference image or reference block, and also provides the reconstructed image to the output unit.

[0103] Figure 3 It is a diagram used to illustrate the image segmentation structure during image encoding and decoding.

[0104] Reference Figure 3 The description applies various block segmentation forms that can be used in this invention. Figure 3 The process of dividing an image 300 into one or more sub-units 310 and 315 is shown.

[0105] To achieve efficient image encoding and decoding, encoding units can be used as the basic units of encoding / decoding. Alternatively, an encoding unit can be a basic unit of encoding / decoding, including units that distinguish prediction modes, transform and inverse transform units, quantization units, and transform coefficients during image encoding / decoding.

[0106] according to Figure 3 An image 300 can be sequentially segmented into maximum coding units (LCUs) 310 and lower units 315. In this paper, the segmentation structure can be determined in units of LCUs. The segmentation of a unit in an image can imply the segmentation of the corresponding block. Segmentation information can include information about the depth of the coding unit or block. Here, depth can represent the number or extent of segmentation of a particular unit.

[0107] A cell can be divided into multiple sub-cells 315 based on the segmentation information. Each segmented sub-cell can have segmentation information specific to that sub-cell, and can also have depth information or size information for that sub-cell.

[0108] Here, segmentation information can refer to the configuration of coding units (CUs) within a coding tree unit (CTU). Segmentation information can be determined based on whether a CU is segmented into multiple CUs. Multiple CUs generated through segmentation can have positive integer values ​​of 2 or greater in either the horizontal or vertical dimensions. Here, the horizontal and vertical dimensions can be the same or different values.

[0109] Multiple control units (CUs) can be recursively partitioned again. According to the recursive partitioning, the CU to be partitioned can have a horizontal or vertical dimension smaller than at least one of its horizontal or vertical dimensions compared to the CU before partitioning. Here, the recursive partitioning of CUs can be performed recursively until a predefined size is reached. The smallest coding unit with the smallest size that is no longer subject to recursive partitioning can be defined as the smallest coding unit (SCU). The SCU can have a predefined maximum depth value. That is, partitioning starts from the LCU and SCUs can be generated through recursive partitioning.

[0110] Reference Figure 3 An LCU 310 with a depth of 0 can be a 64×64 block. Here, 0 can be the minimum depth. On the other hand, an SCU with a maximum depth can have a depth of 3. Here, 3 can be the maximum depth. The depths of 32×32 and 16×16 can be 1 and 2 respectively. Moreover, the depth value of 32×16 can be 2.

[0111] A CU can be divided into 4 units. When a CU is divided into 4 units, the horizontal and vertical dimensions of the divided CU can each be half the size of the original CU. When a CU is divided into four CUs, it can be said that the coding unit is divided in the form of a quadtree.

[0112] Furthermore, a CU can be divided into two units. When a CU is divided into two CUs, at least one of the horizontal or vertical dimensions of the divided CU can be half the size of the original CU. For example, if a 32×32 CU is divided into two CUs, then each of the two divided CUs can have a size of 32×16. When a CU is divided into two CUs, it can be said that the CU is divided in the form of a binary tree or a binary partitioning tree.

[0113] Furthermore, a CU can be divided into three units. When a CU is divided into three units, at least one of the horizontal and vertical dimensions of the divided CU can be determined by dividing the size of a CU by a specific ratio. For example, if a 32×32 CU is divided into three CUs, it can be divided in a 1:2:1 ratio. In this case, the 32×32 CU can have sizes of 32×8, 32×16, and 32×8. When a CU is divided into three CUs, it can be said that the CU is divided in the form of a ternary tree or a ternary partitioning tree.

[0114] Figure 3The results of segmenting an LCU 310 using quadtrees, binary trees, and ternary trees are shown. In this way, the encoder and decoder can apply one of quadtree segmentation, binary tree segmentation, or ternary tree segmentation to segment the CTU. Here, each segmentation can be applied in a predefined order. For example, quadtree segmentation can be applied preferentially to the LCU. In this case, CUs that can no longer be segmented by quadtree can become leaf nodes of the quadtree segmentation. Leaf nodes of quadtree segmentation can be the root nodes of either binary tree segmentation or ternary tree segmentation.

[0115] The CU corresponding to each node of the quadtree can be determined using quadtree partitioning information. For example, when the quadtree partitioning information has a first value, the quadtree partitioning information can indicate that the CU is recursively partitioned into a quadtree. On the other hand, when the quadtree partitioning information has a second value, the quadtree partitioning information of the corresponding CU can indicate that the corresponding CU is no longer partitioned into a quadtree.

[0116] There may be no priority between binary tree partitioning and ternary tree partitioning. That is, a quadtree leaf node can be partitioned either by binary tree partitioning or ternary tree partitioning, or not by any other method. When there is no priority between binary tree partitioning and ternary tree partitioning for a given CU, it can be said that the CU is partitioned in the form of multi-type tree partitioning.

[0117] Each of the following information—whether a segment has been made, the type of segmentation, the direction of segmentation, etc.—can be signaled using a flag of predetermined length.

[0118] The diagonal partitioning prediction proposed in this invention can be applied to blocks that have undergone at least one of the aforementioned quadtree partitioning, binary tree partitioning, or ternary tree partitioning. Furthermore, since blocks that have undergone diagonal partitioning according to this invention are adjacent to blocks that have undergone at least one of the aforementioned quadtree partitioning, binary tree partitioning, or ternary tree partitioning, various types of predictions can be performed.

[0119] The image encoding / decoding method according to the present invention will be described in detail below based on the above.

[0120] Figure 4 This is a diagram used to illustrate the geometrical motion division (GEO).

[0121] To achieve efficient image encoding / decoding, Geometric Motion Partitioning (GEO) is discussed. GEO partitions a specific block into arbitrary forms and performs prediction based on the segmentation format of the past JCT-VC (Joint Collaborative Team Video Coding) standardization meeting. GEO can segment the target block for encoding into various types to perform image encoding / decoding.

[0122] Existing GEO-based decoding devices can divide a block 400 into two distinct regions 405 and 410. In this case, the boundary line dividing a block into two regions can be defined as the GEO partition boundary line 415. The GEO partition boundary line 415, using a straight line, can be determined by two variables—angel value() (angle value variable) and normal length() (normal length variable). Here, the angle value can represent the angle between the reference line and the normal, and the normal length can represent the distance from the origin to the GEO partition boundary line 415.

[0123] GEO segmentation has the advantage of being able to segment various types of blocks, but its disadvantage is that both the angle value and the normal length must be signaled to determine the GEO boundary lines used for segmentation. Since the encoding device must signal both the angle value and the normal length, there is an overhead problem associated with signaling for block segmentation using GEO.

[0124] This invention proposes a method to improve image encoding / decoding efficiency by segmenting blocks to be encoded that are differently divided into squares or non-squares using diagonals connecting the top left and bottom right corners or connecting the top right and bottom left corners. Hereinafter, the block segmentation using diagonals according to this invention is described as diagonal moving part (hereinafter referred to as DMP), diagonal segmentation, or triangular segmentation.

[0125] The various block partitioning forms described in this specification can represent at least one of the following: conventional quadtree partitioning, quadtree partitioning, binary tree partitioning, ternary tree partitioning, or combinations thereof. In other words, the various block partitioning types can represent various partitioning types that divide a block into multiple square and non-square sub-coded blocks.

[0126] Furthermore, according to the present invention, diagonal segmentation can refer to the method of using diagonals to divide corresponding square and non-square blocks in a structure that applies the various block segmentation types described above, and using it to perform prediction.

[0127] Figure 5 This is a flowchart illustrating an image decoding method according to an embodiment of the present invention.

[0128] Reference Figure 5The decoding device can obtain the segmentation information of the current target block (S500). More specifically, the decoding device can perform entropy decoding on the bit stream sent from the encoding device to obtain the segmentation information of the current block. Thereafter, the decoding device can use the obtained segmentation information to determine the diagonal segmentation structure of the current block (S510), and diagonally segment the current target block according to the determined segmentation structure. Here, the current block can represent an encoded block obtained by recursively applying square or non-square segmentation.

[0129] In other words, the current block here can represent a leaf node that cannot be further divided into square or non-square partitions. Furthermore, a square partition can refer to the aforementioned quadtree partition, and a non-square partition can refer to at least one of the aforementioned binary or ternary tree partitions.

[0130] In other words, the current block can be generated by recursively applying at least one of quadtree partitioning, binary tree partitioning, or ternary tree partitioning to any coding unit. However, the current block is not limited to the coding blocks generated by the above methods, and blocks generated according to various types of block partitioning methods that can be performed by those skilled in the art can be applied to this invention.

[0131] Figure 6 This is a diagram illustrating an image encoding method according to an embodiment of the present invention.

[0132] Reference Figure 6 The encoding device can determine the diagonal segmentation structure of the current block (S600). Here, by utilizing various conventional image encoding methods, the diagonal segmentation structure can be determined as a block structure with optimal rate-distortion (RD). Then, the encoding device can segment the current block diagonally according to the determined diagonal segmentation structure (S610). Finally, the encoding device can encode the diagonal segmentation information (S620).

[0133] For convenience, examples executed by the decoding apparatus of the present invention will be described below; however, the described embodiments can be executed in the same manner in both the encoding and decoding apparatuses. Furthermore, the same embodiment can be applied to each of the luminance and chrominance signals, and embodiments according to the present invention can be applied to only one of the luminance and chrominance signals.

[0134] Furthermore, the present invention can be applied based on the size of the current block or the size of the sub-blocks into which the current block is divided. Here, the size can be limited to the minimum and / or maximum size for applying the above embodiments, or it can be limited to a fixed size for applying the above embodiments. Moreover, the above embodiments of the present invention can only be applied when the size of the current block or its sub-blocks is greater than or equal to the minimum size and less than or equal to the maximum size. That is, the above embodiments can be applied only when the block size is included within a specific range.

[0135] Figure 7 This is another figure used to illustrate the image decoding method according to an embodiment of the present invention. Figure 8 This is another diagram used to illustrate that diagonal division according to embodiments of the present invention can represent various arbitrary division types.

[0136] Reference Figure 7 and Figure 8 The following describes in detail the method by which the decoding device performs diagonal partitioning on the current block (S520). In the following description, the current block may refer to a block in which square or non-square block partitioning is recursively performed on any coding unit, and no further square or non-square block partitioning is performed. That is, the current block may represent a leaf node of square or non-square partitioning. In other words, the current block may have a square shape or a non-square shape.

[0137] Alternatively, in the following description, the current block may refer to a block in which recursively performing quadtree partitioning on any coding unit results in no further quadtree partitioning being performed. That is, in this case, the current block may have a square shape or a non-square shape, and may undergo additional binary or ternary partitioning.

[0138] That is, the current block in this specification is a target block that can be diagonally split, and can represent a block generated by performing at least one square or non-square block split on at least one coding unit.

[0139] That is, as a result of performing at least one of quadtree splitting, binary tree splitting, or ternary tree splitting on any coding unit, a block that can be the target of diagonal splitting in this specification can represent a leaf node of a square split or a non-square splitting that can no longer be further splitted by quadtree splitting, binary tree splitting, or ternary tree splitting.

[0140] The decoding device can divide a block 700 into two distinct regions 705 and 710. In this case, the boundary line dividing a block into two regions can be defined as the diagonal line 715 of the diagonal division. The diagonal boundary line 715 can be defined by two variables.

[0141] The two regions of a block divided based on the boundary line 715 can be defined as the first region and the second region. Although in Figure 7 In the description, the left region 705 is defined as the first region and the right region 710 is defined as the second region, but the meaning of the first and second regions is not limited to this description. The two divided regions can be defined as the first region or the second region regardless of their position and shape.

[0142] Unlike existing GEO-based block segmentation methods that use angle values ​​and normal lengths to represent the boundary line 415 of the GEO segmentation, the present invention can segment any coding block by using square or non-square segmentation for any coding unit and diagonal segmentation for the current block.

[0143] Figure 7 The decoding device is shown to obtain information by using a vertical binary tree partition and a diagonal partition. Figure 4 Examples of block partitioning structures that are the same as those described in the text.

[0144] Figure 7 This illustrates how a square block 700 is divided into two non-square blocks using a vertical binary tree partitioning method. In this paper, diagonal partitioning may not be applied to the left non-square block. That is, Figure 7 An example is shown where diagonal segmentation is performed only on the rightmost non-square block of the two non-square blocks. The decoding device can obtain the same segmentation structure as GEO segmentation by either square segmentation or a combination of non-square segmentation and diagonal segmentation.

[0145] That is, according to the present invention, any block segmentation form can be represented using only the segmentation information of square segmentation or non-square block segmentation and diagonal segmentation.

[0146] Figure 8 This diagram illustrates several coded blocks that are diagonally divided according to the present invention. A decoding device can diagonally divide the current block. Hereinafter, the direction in which a block is divided by connecting the top-left vertex and the bottom-right vertex is defined as a diagonal division along the bottom-right direction. Conversely, the direction in which a block is divided by connecting the top-right vertex and the bottom-left vertex is defined as a diagonal division along the top-right direction.

[0147] The decoding device can perform a diagonal split along the lower right or the upper right for the current block. Figure 8 The first coded block shown illustrates a diagonal split performed by the lower right boundary line 800. Figure 8 The second coded block shown illustrates a diagonal segmentation performed by the upper right boundary line 805. The decoding device can segment the coded block along the lower right or the upper right direction based on the segmentation information.

[0148] according to Figure 8 The decoding device can not only divide square blocks diagonally, but also non-square blocks diagonally.

[0149] Figure 9 This is another figure illustrating a method for determining the direction of diagonal division according to an embodiment of the present invention.

[0150] Figure 9 (a) shows the diagonal division along the lower right direction. Figure 9 (b) shows a diagonal division along the upper right direction. The division boundary used to divide the current block can be defined according to Equation 1 below.

[0151] [Formula 1]

[0152] The boundary is divided diagonally along the lower right direction: f(x,y)=w*yh*x

[0153] The boundary is divided diagonally along the upper right direction: f(x,y)=w*(hy)-h*x

[0154] Here, h and w can be the vertical (height) and horizontal (width) of the current block, which is the target of diagonal division, respectively. Equation 1 is defined by using the coordinates of the upper left corner of the current block as the origin. Equation 1 above can be an example for representing the lower right and upper right diagonals of any square or non-square block. That is, there can be one or more formulas representing diagonal division according to the present invention, and the formulas representing diagonal division according to the present invention are not limited to Equation 1 above.

[0155] In addition, through Figure 9 The two regions generated by the diagonal segmentation shown in (a) or 9(b) can be defined as a first region and a second region, respectively. Based on the diagonal of the diagonal segmentation, the region on the left is defined as the first region, and the region on the right is defined as the second region, but the meaning of the first and second regions is not limited to this description. The two segmented regions can be defined as either the first or the second region regardless of their position and shape.

[0156] Figure 10 This is a view used to illustrate the masking method according to an embodiment of the present invention.

[0157] Reference Figure 10 This paper describes a method for classifying samples located on the boundary lines of a diagonally divided block by a decoding device. When the decoding device divides an arbitrary coded block into two regions, it becomes difficult to determine which region includes samples located on the boundary lines.

[0158] Since a sample is the smallest unit of image coding, it cannot be further segmented, thus making it impossible to divide a sample into two regions. To address this issue, the decoding device can perform a masking operation on the samples within the coding block that serves as the target for diagonal segmentation. Here, masking operations can be performed on samples located on the diagonal of the diagonal segmentation, as well as samples surrounding the diagonal and samples within the diagonal segmentation region.

[0159] In the specification of this invention, masking operation can refer to a method for determining which region, in a first region or a second region, a particular sample included in a diagonally segmented decoded block belongs to.

[0160] Alternatively, the masking operation in the specification of this invention can represent a method of performing a preset weighted operation on each sample of a decoded block that will be diagonally divided. Since individual samples cannot be divided, this can mean representing samples crossed by the boundary line of the diagonal division as a weighted sum of two regions.

[0161] Figure 10 An example of a decoding device applying a masking operation along the upper right direction to a 16×8 sized encoding block is shown. Figure 10 (a) illustrates the masking operation for the first region, which is diagonally divided along the upper right direction. Figure 10 (b) illustrates the masking operation for a second region diagonally segmented along the upper right direction. The masking operation method can be predefined. Alternatively, the masking operation method can be determined differently based on the size of the target block and the segmentation direction.

[0162] As an example, the masking operation can be defined according to Equation 2 below. However, the scope of the invention is not limited by the values ​​provided in Equation 2.

[0163] [Equation 2]

[0164]

[0165] In Equation 2 above, Mask_p0 and Mask_p1 can represent the weighted values ​​used to determine the pixel values ​​of each region sample. In Equation 2, samples satisfying f(x,y)>0 can be defined as first samples, samples satisfying f(x,y)=0 can be defined as boundary region samples, and samples corresponding to the remaining samples can be defined as second samples.

[0166] According to Equation 2, a coding block to be diagonally segmented can be represented by a first sample, a second sample, and a third sample, wherein the first sample consists of samples located in any direction relative to the boundary line from the samples included in the region not crossed by the diagonal segmentation boundary line, the second sample consists of samples crossed by the diagonal segmentation boundary line, and the third sample consists of samples not included in the first and second samples.

[0167] In this paper, the masking operation on the first region can be achieved by applying a first value, a second value, and a third value to the first sample, the second sample, and the third sample, respectively. Similarly, the masking operation on the second region can be achieved by applying a third value, a second value, and a first value to the first sample, the second sample, and the third sample, respectively. In this paper, as described in Equation 2, the first value, the second value, and the third value can be 2, 1, and 0, respectively.

[0168] By combining Equations 1 and 2, the final sample value of the current block can be defined according to Equation 3 below. However, the scope of the invention is not limited to the numerical values ​​or variables provided in Equation 3.

[0169] [Formula 3]

[0170] P DMP (x,y)=(P P0 (x, y) × MASK P0 (x, y) + P P1 (x, y) × MASK P1 (x,y)>>shift

[0171] Here, P_DMP(x, y) can be the sample value of a specific sample in the current block, and P_p0(x, y) and P_p1(x, y) can be the sample values ​​of samples belonging to the first region or the second region, respectively. `shift` can be a value used to scale the sample values ​​according to the mask operation. For example, the value of `shift` can be 1.

[0172] P_p0(x, y) and P_p1(x, y) can refer to the predicted samples obtained by performing motion compensation on the reference image using different motion vectors corresponding to each region of the diagonal segmentation. Furthermore, P_p0(x, y) and P_p1(x, y) can represent the sample values ​​of the reconstructed samples corresponding to each region of the current block.

[0173] Figure 11 This is a view used to illustrate a masking operation method according to another embodiment of the present invention.

[0174] Since samples adjacent to or diagonally segmented boundaries are obtained by weighted summation of samples from physically different regions, discontinuities may occur in the images within these samples. To prevent these discontinuities, the decoding apparatus can perform a masking operation taking into account the discontinuities in sample values. In this application, motion compensation that takes into account such discontinuities in the image encoding / decoding region can be defined as Overlapping Block Compensation (hereinafter referred to as OBMC) or position-based motion compensation. The decoding apparatus can diagonally segment the current block while taking OBMC into account. In the following description, the masking operation that takes into account the discontinuities in sample values ​​adjacent to or diagonally segmented boundaries is described as a masking operation that takes into account sample discontinuities.

[0175] Figure 11 An implementation of the decoding apparatus that performs masking operations on encoded blocks taking into account sample discontinuities is illustrated. Specifically, Figure 11 The diagram illustrates the decoding device performing a masking operation that divides a 16×8-sized encoding block diagonally along the upper right direction. Figure 11 (a) illustrates the masking operation for the first region, which is diagonally divided along the upper right direction. Figure 11 (b) illustrates the masking operation for a second region diagonally segmented along the upper right direction. The masking operation method and weighting settings can be predefined to account for sample discontinuities. Furthermore, the masking operation method and weighting settings, taking into account sample discontinuities, can be determined based on the size and segmentation direction of the target diagonally segmented block, motion vectors, chromaticity components, etc.

[0176] Reference Figure 11 As can be seen in (a) and 11(b), the mask value gradually decreases or increases along a direction different from the boundary direction of the diagonal segmentation. That is, when the diagonal segmentation direction is the upper right direction, the mask weighting value can gradually decrease or increase along the upper right direction. Specifically, the maximum mask value can be assigned to the sample farthest from the boundary line. On the other hand, in the direction opposite to where the maximum mask value based on the boundary line is assigned, the minimum mask value (e.g., 0) can be assigned to the farthest sample.

[0177] That is, in Figure 11 In (a), the mask value of the first region can be assigned to gradually decrease along the lower right direction. Similarly, in Figure 11 In (b), the mask value of the second region can be assigned to gradually increase along the lower right direction. Figure 11 The mask settings shown in A and 11B are examples of the present invention, but the scope of the invention is not limited thereto.

[0178] For example, the weighted setting can be determined as {0, 1, 2, 4, 6, 7, 8}. Considering the scaling value, the weighted setting can be determined as {0, 1 / 8, 2 / 8, 4 / 8, 6 / 8, 8 / 8}. As another example, the weighted setting can be determined as {0, 1, 2, 3, 4, 5, 6, 7, 8}. Considering the scaling value, the weighted setting can be determined as {0, 1 / 8, 2 / 8, 3 / 8, 4 / 8, 5 / 8, 6 / 8, 7 / 8, 8 / 8}. The above weighted settings are examples, and various combinations of values ​​that gradually increase or decrease can be used to determine the weighted setting.

[0179] Furthermore, the decoding device can apply different weighting settings to the luminance and chrominance components. For example, when using {0, 1, 2, 4, 6, 7, 8} or {0, 1 / 8, 2 / 8, 4 / 8, 6 / 8, 8 / 8} as the weighting settings for the luminance component masking operation, the weighting settings of {0, 1, 4, 7, 8} or {0, 1 / 8, 4 / 8, 7 / 8, 8 / 8} can be used for the chrominance component masking operation. As another example, when using {0, 1, 2, 3, 4, 5, 6, 7, 8} or {0, 1 / 8, 2 / 8, 3 / 8, 4 / 8, 5 / 8, 6 / 8, 7 / 8, 8 / 8} as the weighting settings for the luminance component masking operation, the weighting settings of {0, 2, 4, 6, 8} or {0, 2 / 8, 4 / 8, 6 / 8, 8 / 8} can be used for the chrominance component masking operation. The example of the weighting setting described above is just one example, and it can be determined by using various other values ​​in combination with the weighting settings of the luminance component mask operation.

[0180] The decoding device may allow the application of masking operations that take into account sample discontinuities only when the size of the encoded block exceeds, is greater than, or is equal to a preset size. For example, the decoding device may only allow the application of masking operations that take into account sample discontinuities to the diagonal segmentation of the current block or its sub-blocks when the number of samples in a particular encoded block exceeds 256.

[0181] As another example, the decoding device may allow the application of a masking operation that takes into account sample discontinuities to block segmentation only when the horizontal and / or vertical dimensions of the current block exceed or are greater than or equal to a preset size. For example, the decoding device may allow the application of a masking operation that takes into account sample discontinuities to the diagonal segmentation of the current block or its sub-blocks only when at least one of the horizontal or vertical dimensions of the current block is greater than 16.

[0182] The encoding and decoding devices can define an indicator that specifies whether to apply a masking operation that takes into account sample discontinuities to the current block. This indicator can be defined as a flag of size 1 bit. For example, when the flag has a first value, the decoding device can diagonally segment the current block by applying a mask that takes into account sample discontinuities.

[0183] Indicators indicating whether to apply masking operations that take into account sample discontinuities can be sent not only along with flags, but also along with block shape and motion information. Furthermore, the application of masking operations that take into account sample discontinuities can be indirectly indicated by the size of the current block.

[0184] Figure 11 This is a diagram used to illustrate the syntax parameters according to an embodiment of the present invention.

[0185] Several parameters can be proposed to indicate the diagonally segmented blocks or motion information according to the present invention.

[0186] The encoding and decoding devices can define a flag (DMP_flag) indicating whether to apply diagonal partitioning to the current block. When DMP_flag has a first value, the encoding and decoding devices can partition the current block diagonally. Conversely, when DMP_flag has a second value, the encoding and decoding devices cannot partition the current block diagonally.

[0187] Here, flag information indicating whether diagonal partitioning is applied can be sent in units of a coded block. In this paper, a coded block can represent a leaf node in which square partitioning or non-square partitioning can no longer be applied.

[0188] The encoding and decoding devices can define an indicator (DMP_direction) to indicate the direction of diagonal partitioning. When DMP_direction has a first value, the encoding and decoding devices can apply a diagonal partition along the upper right direction to the current block. Conversely, when DMP_direction has a second value, the encoding and decoding devices can apply a diagonal partition along the lower right direction to the current block.

[0189] The segmentation information of this invention may include DMP_flag and DMP_direction.

[0190] The encoding and decoding devices can define a DMP merge flag (DMP_merge_flag) that indicates whether to apply a merge mode to regions where diagonal partitioning has been applied. When the DMP merge flag has a first value, the encoding and decoding devices can encode / decode the diagonal partitioned regions using the merge mode. The DMP merge flag can be determined on a per-block basis or on a per-region basis within which the current block is partitioned.

[0191] When the DMP merge flag has a first value, the encoding and decoding devices can define a merge index (DMP_merge_idx) that indicates merge candidates for diagonally split regions. The DMP merge index can be determined on a block-by-block basis or on a region-by-region basis obtained by performing diagonal splitting.

[0192] The diagonally segmented region can include motion information. This motion information can include, but is not limited to, intra-frame predicted direction, reference image index, motion vectors, differential motion vectors, and AMVP motion information. Furthermore, the motion information can be determined on a block-by-block basis, or it can be determined on a region-by-region basis resulting from diagonal segmentation.

[0193] As another example, the encoding and decoding devices can define a preset diagonal split merge index (dmp_merge_idx) that indicates the merge candidates for each region after diagonal splitting. For example, when the diagonal split merge index specifies a first value, the decoding device can use the splitting direction corresponding to the first value and the preset merge candidates for each region to perform prediction on the current block. For example, the merge index can be predefined as having one of the predefined integer values ​​greater than zero.

[0194] Figure 12 The following example is shown: the current block 1200 is divided into two sub-coded blocks using binary tree partitioning, and one of the sub-coded blocks is divided into a first region 1205 and a second region 1210 by a diagonal partitioning boundary line 1215.

[0195] The decoding device can segment the current block by applying a vertical binary tree split to the current block 1100. In this document, the decoding device can check the binary tree flag (BT_split_flag) used to apply the binary tree split to the current block, and if this value is the first value, the current block can be split into a binary tree. Furthermore, when it is determined by the binary tree flag that the current block is split into a binary tree, the decoding device can determine whether the current block is split vertically or horizontally by checking the split type indicator (BT_split_type).

[0196] When it is determined that there are no other square or non-square partitions of the current block, the decoding device can determine whether the current block is diagonally partitioned. In this paper, the decoding device can check the `DMP_flag` to apply diagonal partitioning to the current block, and if this value is the first value, the current block can be diagonally partitioned. Furthermore, when it is determined by the `DMP_flag` that the current block is diagonally partitioned, the decoding device can determine whether to partition the current block diagonally along the lower right or the upper right by checking the `DMP_direction`. For example, in... Figure 11 In the process, if the decoding device can determine that DMP_Direction has a first value, then the current block is divided diagonally along the upper right direction.

[0197] Then, the decoding device can respectively derive the motion vector MV1 of the first region 1105 and the motion vector MV2 of the second region 1110, which are diagonally divided.

[0198] Obviously, the above parameter variables and values ​​are exemplary, and the scope of the invention is not limited to the terminology used in the specification.

[0199] The inter-frame prediction method according to the present invention can be applied to inter-frame prediction methods based on samples. That is, the inter-frame prediction method according to the present invention may not be applicable to inter-frame prediction methods based on sub-blocks. More specifically, the inter-frame prediction method according to the present invention includes sub-block-based temporal candidate prediction (ATMVP: Alternate Temporal Motion Vector Prediction), sub-block-based spatial-temporal combined candidate prediction (STMVP: Spatial-Temporal Motion Vector Prediction), and frame rate conversion. (This method may not be applicable to FRUC (Frame Rate Upconversion), etc.)

[0200] The various embodiments of this disclosure are not intended to list all possible combinations, but are intended to describe representative aspects of this disclosure, and the contents described in the various embodiments may be applied independently or in combination of two or more.

[0201] Furthermore, various embodiments of this disclosure can be implemented using hardware, firmware, software, or a combination thereof. In the case of hardware implementation, this disclosure can be implemented using one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers, microprocessors, etc.

[0202] The scope of this disclosure includes software or machine-executable instructions (e.g., operating systems, applications, firmware, programs, etc.) that enable operation of methods according to various embodiments to be performed on an apparatus or computer, and such software or instructions include non-transitory computer-readable media stored and executable on an apparatus or computer.

[0203] Industrial applicability

[0204] This invention can be used to encode / decode images.

[0205] In addition, this disclosure also includes the following implementation methods.

[0206] (1) A method for decoding an image, comprising:

[0207] Obtain the diagonal segmentation information for the current block;

[0208] The diagonal segmentation information is used to determine the diagonal segmentation structure of the current block; and

[0209] Diagonal segmentation is performed based on the determined diagonal segmentation structure to divide the current block into a first region and a second region.

[0210] The current block is a leaf node that is either square-divided or non-square-divided.

[0211] (2) The method for decoding images according to (1),

[0212] The diagonal segmentation information includes a flag indicating whether the current block is diagonally segmented.

[0213] (3) The method for decoding images as described in (1),

[0214] Specifically, the diagonal segmentation is determined to be either a lower right diagonal segmentation or an upper right diagonal segmentation based on the direction of the diagonal segmentation.

[0215] (4) The method for decoding images according to (3)

[0216] Wherein, the diagonal segmentation information includes segmentation direction information indicating the direction of the diagonal segmentation, and

[0217] Specifically, performing diagonal segmentation means dividing the current block diagonally along the direction indicated by the segmentation direction information.

[0218] (5) The method for decoding images according to (1),

[0219] The diagonal segmentation includes performing a masking operation on the current block, and

[0220] The masking operation involves performing a predetermined weighted operation on the samples of the current block.

[0221] (6) The method for decoding images according to (5),

[0222] The current block is composed of a first sample, a second sample, and a third sample. The first sample consists of samples in one direction based on the boundary line, from the samples included in the region not crossed by the diagonally divided boundary line. The second sample consists of samples crossed by the diagonally divided boundary line. The third sample consists of samples in the current block that do not belong to the first sample or the second sample.

[0223] The masking operation for the first region involves applying a first value to the first sample, a second value to the second sample, and a third value to the third sample.

[0224] The masking operation for the second region is as follows: applying the third value to the first sample, applying the second value to the second sample, and applying the third value to the third sample.

[0225] (7) The method for decoding an image as described in (5),

[0226] In the case where f(x, y) represents the boundary line of the diagonal division,

[0227] The masking operation for the first region and the masking operation for the second region satisfy the following equation 1.

[0228] [Formula 1]

[0229]

[0230] Where MASK_p0(x,y) represents the mask value of the first region, and MASK_p1(x,y) represents the mask value of the second region.

[0231] (8) The method for decoding an image according to (7),

[0232] Wherein, the sample value of the current block satisfies the following equation 2,

[0233] [Equation 2]

[0234] P DMP (x,y)=(P P0 (x, y) × MASK P0 (x, y) + P P1 (x, y) × MASKP1 (x,y)>>shift

[0235] Wherein, P_DMP(x,y) represents a specific sample value of the current block, P_p0(x,y) represents the sample value of a sample belonging to the first region, P_p1(x,y) represents the sample value of a sample belonging to the second region, and shift represents the scaling value according to the mask operation.

[0236] (9) The method for decoding an image according to (1),

[0237] The square segmentation or the non-square segmentation is one of quadtree segmentation, binary tree segmentation, and ternary tree segmentation.

[0238] (10) A method for encoding an image, comprising:

[0239] Determine the diagonal segmentation structure of the current block;

[0240] Diagonal segmentation is performed based on the determined diagonal segmentation structure to divide the current block into a first region and a second region; and

[0241] The encoding is for the diagonal segmentation information of the diagonal segmentation structure.

[0242] The current block is a leaf node that is either square-divided or non-square-divided.

[0243] (11) The method for encoding an image according to (10),

[0244] The method further includes encoding a flag indicating whether the current block is diagonally divided.

[0245] (12) The method for encoding an image according to (10),

[0246] Specifically, the diagonal segmentation is determined to be either a lower right diagonal segmentation or an upper right diagonal segmentation based on the direction of the diagonal segmentation.

[0247] (13) The method for encoding an image according to (10),

[0248] The encoding step includes encoding segmentation direction information that indicates the direction of the diagonal segmentation.

[0249] (14) The method for encoding an image according to (10),

[0250] The diagonal segmentation includes performing a masking operation on the current block, and

[0251] The masking operation involves performing a predetermined weighted operation on the samples of the current block.

[0252] (15) The method for encoding an image according to (14),

[0253] The current block is composed of a first sample, a second sample, and a third sample. The first sample consists of samples in one direction based on the boundary line, from the samples included in the region not crossed by the diagonally divided boundary line. The second sample consists of samples crossed by the diagonally divided boundary line. The third sample consists of samples in the current block that do not belong to the first sample or the second sample.

[0254] Specifically, the masking operation for the first region involves applying a first value to the first sample, a second value to the second sample, and a third value to the third sample.

[0255] Specifically, the masking operation for the second region involves applying the third value to the first sample, applying the second value to the second sample, and applying the third value to the third sample.

[0256] (16) The method for encoding an image according to (14),

[0257] In the case where f(x,y) represents the boundary line of the diagonal division,

[0258] The masking operation for the first region and the masking operation for the second region satisfy the following equation 3.

[0259] [Formula 3]

[0260]

[0261] Wherein, MASK_p0(x,y) represents the mask value of the first region, and MASK_p1(x,y) represents the mask value of the second region.

[0262] (17) The method for encoding an image according to (16),

[0263] Wherein, the sample value of the current block satisfies the following equation 4,

[0264] [Equation 4]

[0265] P DMP (x,y)=(P P0 (x, y) × MASK P0 (x, y) + P P1 (x, y) × MASKP1 (x,y)>>shift

[0266] Wherein, P_DMP(x,y) represents a specific sample value of the current block, P_p0(x,y) represents the sample value of a sample belonging to the first region, P_p1(x,y) represents the sample value of a sample belonging to the second region, and shift represents the scaling value according to the mask operation.

[0267] (18) The method for decoding an image according to (1),

[0268] The square segmentation or non-square segmentation is one of quadtree segmentation, binary tree segmentation, and ternary tree segmentation.

[0269] (19) A non-transitory storage medium for storing bitstreams generated by an image encoding method.

[0270] The image encoding method includes:

[0271] Determine the diagonal segmentation structure of the current block;

[0272] Diagonal segmentation is performed based on the determined diagonal segmentation structure to divide the current block into a first region and a second region; and

[0273] Encode the diagonal segmentation information for the diagonal segmentation structure, and

[0274] The current block is a leaf node that is either square-divided or non-square-divided.

Claims

1. A method for decoding an image, the method comprising: Based on tree-based block partitioning, the higher-level coded blocks in the image are divided into multiple coded blocks, including the current block; When performing inter-frame prediction in merge mode on the current block, the current block is divided into a first partition and a second partition based on the partition boundary; A first predicted block for the current block is obtained based on first motion information corresponding to a first merge candidate associated with the first partition; A second predicted block for the current block is obtained based on the second motion information corresponding to the second merge candidate associated with the second partition; The final prediction block of the current block is obtained by performing a weighted sum of the first prediction block and the second prediction block; as well as The current block is reconstructed based on the final predicted block. The partition boundary is determined based on the direction indicated by the segmentation information of the current block. Wherein, at least one of the first division and the second division is triangular in shape, and Wherein, when no inter-frame prediction based on sub-blocks is performed on the current block, inter-frame prediction using the merging mode with the first partition and the second partition is performed on the current block.

2. A method for encoding an image, the method comprising: Based on tree-based block partitioning, the higher-level coded blocks in the image are divided into multiple coded blocks, including the current block; When performing inter-frame prediction in merge mode on the current block, the current block is divided into a first partition and a second partition based on the partition boundary; A first predicted block for the current block is obtained based on first motion information corresponding to a first merge candidate associated with the first partition; A second predicted block for the current block is obtained based on the second motion information corresponding to the second merge candidate associated with the second partition; The final prediction block of the current block is obtained by performing a weighted sum of the first prediction block and the second prediction block; Encode the residual block between the original block of the current block and the final predicted block; as well as Generate segmentation information indicating the direction of the partition boundaries in the current block. Wherein, at least one of the first division and the second division is triangular in shape, and Wherein, when no inter-frame prediction based on sub-blocks is performed on the current block, inter-frame prediction using the merging mode with the first partition and the second partition is performed on the current block.

3. A method for storing data associated with an image signal into a computer-readable medium, the method comprising: Based on tree-based block partitioning, the higher-level coded blocks in the image are divided into multiple coded blocks, including the current block; When performing inter-frame prediction in merge mode on the current block, the current block is divided into a first partition and a second partition based on the partition boundary; A first predicted block for the current block is obtained based on first motion information corresponding to a first merge candidate associated with the first partition; A second predicted block for the current block is obtained based on the second motion information corresponding to the second merge candidate associated with the second partition; The final prediction block of the current block is obtained by performing a weighted sum of the first prediction block and the second prediction block; Encode the residual block between the original block of the current block and the final predicted block; as well as The bitstream is stored in the computer-readable medium, wherein the bitstream includes the encoded residual block of the current block and segmentation information indicating the direction of the partition boundaries in the current block. Wherein, at least one of the first division and the second division is triangular in shape, and Wherein, when no inter-frame prediction based on sub-blocks is performed on the current block, inter-frame prediction using the merging mode with the first partition and the second partition is performed on the current block.