Method and apparatus for encoding and decoding a video by using prediction

Abstract

A method and apparatus for encoding and decoding video using prediction are disclosed. During encoding of the current block, a residual signal for the current block is generated based on the current block, a first prediction, and a second prediction. Furthermore, information about the encoded residual signal is generated by encoding the residual signal. The second prediction is a prediction for a portion of the current block. During decoding of the current block, a reconstructed residual signal for the current block is generated. A reconstructed block for the current block is generated based on the reconstructed residual signal, the second prediction, and the first prediction. The second prediction is a prediction for a portion of the current block.

Description

[0001] This application is a divisional application of the invention patent application filed on January 26, 2017, with application number "201780008673.3" and titled "Method and apparatus for encoding and decoding video by using prediction". Technical Field

[0002] The following embodiments generally relate to a video decoding method and apparatus, as well as a video encoding method and apparatus, and more specifically, to a method and apparatus for performing encoding and decoding on video based on the prediction of target blocks. Background Technology

[0003] With the continuous development of the information and communication industry, high-definition (HD) broadcasting services have become widespread throughout the world. Through this widespread adoption, a large number of users have become accustomed to high-resolution and high-definition images and / or videos.

[0004] To meet user demand for high definition, numerous organizations have accelerated the development of next-generation imaging devices. In addition to increased user interest in High Definition TV (HDTV) and Full High Definition (FHD) TV, there is also growing interest in Ultra High Definition (UHD) TV, which boasts more than four times the resolution of Full High Definition (FHD) TV. This increased interest necessitates image encoding / decoding technologies for images with higher resolution and greater definition.

[0005] Image encoding / decoding devices and methods can utilize inter-frame prediction techniques, intra-frame prediction techniques, entropy coding techniques, etc., to perform encoding / decoding on high-resolution and high-definition images. Inter-frame prediction techniques can be used to predict the values ​​of pixels included in the current frame using temporally earlier and / or temporally later frames. Intra-frame prediction techniques can be used to predict the values ​​of pixels included in the current frame using information about the pixels in the current frame. Entropy coding techniques can be used to assign shorter codes to more frequently occurring symbols and longer codes to less frequently occurring symbols.

[0006] In image encoding and decoding, prediction can be interpreted as generating a predicted signal similar to the original signal. Prediction can be primarily categorized as: prediction of an image reconstructed in reference space, prediction of an image reconstructed in reference time, and prediction of an image reconstructed in reference to other symbols. In other words, temporal reference means that the image reconstructed in time is referenced, and spatial reference means that the image reconstructed in space is referenced.

[0007] The current block can be the block that is currently being encoded or decoded. The current block can be referred to as a "target block" or "target unit." During encoding, the current block can be referred to as an "encoded target block" or "encoded target unit." During decoding, the current block can be referred to as a "decoded target block" or "decoded target unit."

[0008] Inter-frame prediction can be a technique used to predict the current block using both temporal and spatial references. Intra-frame prediction can be a technique used to predict the current block using only a spatial reference.

[0009] When performing intra-frame prediction to reduce spatial repetition, image coding / decoding techniques use any one of several prediction modes to encode the current block. For example, in High Efficiency Video Coding (HEVC), 35 intra-frame prediction modes are used.

[0010] The encoding device uses all available prediction modes from a variety of prediction modes to generate a prediction block for the current block, and selects the prediction mode that performs best from all available prediction modes as the prediction mode for encoding the current block. However, even when using the prediction mode that performs best, an error still exists between the original block and the prediction block. This error is represented as a residual block.

[0011] For some pixels, there is a large error between the original block and the predicted block. Due to this large error, spatial repetition may not be sufficiently reduced even after applying transform and / or quantization to the residual block.

[0012] Since intra-frame prediction is a prediction of the entire block along one direction, large errors may occur in a part of the block. Summary of the Invention

[0013] Technical issues

[0014] The embodiments aim to provide a method and apparatus for reducing prediction errors based on prediction units due to prediction.

[0015] The embodiments are intended to provide a method and apparatus for reducing prediction errors caused by the prediction direction of a prediction pattern.

[0016] The embodiments aim to provide a method and apparatus for performing a selective second prediction on a prediction error corresponding to a first prediction made along a first prediction mode.

[0017] The embodiments aim to provide a method and apparatus for partitioning a current block into two parts having the same number of pixels and performing a second prediction on the two parts.

[0018] The embodiments aim to provide a method and apparatus for using a second prediction on a portion of a plurality of portions of a current block where the prediction error caused by a first prediction is large.

[0019] The embodiments are intended to provide a method and apparatus for reducing prediction errors in the current block by using a second prediction.

[0020] The embodiments are intended to provide a method and apparatus for providing higher compression performance for the current block by using a second prediction.

[0021] Solution

[0022] According to one aspect, an encoding method is provided, comprising: generating a residual signal of the current block based on a current block, a first prediction, and a second prediction; generating information about the encoded residual signal by encoding the residual signal, wherein the second prediction is a prediction for a portion of the current block.

[0023] A decoding apparatus is provided, comprising: a reconstruction residual signal generation unit for generating a reconstruction residual signal for a current block; and a reconstruction block generation unit for generating a reconstruction block for the current block based on the reconstruction residual signal, a second prediction, and a first prediction, wherein the second prediction is a prediction for a portion of the current block.

[0024] A decoding method is provided, comprising: generating a reconstructed residual signal for a current block; generating a reconstructed block for the current block based on the reconstructed residual signal, a second prediction, and a first prediction, wherein the second prediction is a prediction for a portion of the current block.

[0025] Both the first and second predictions can be intra-frame predictions.

[0026] The portion may have a non-square shape.

[0027] The aforementioned portion can be determined based on a prediction model derived from the first prediction.

[0028] The portion can be determined based on a straight line passing through the center of the current block along the predicted direction of the first prediction.

[0029] The first and second predictions can have different prediction directions.

[0030] The portion can be one of two portions generated by partitioning the current block.

[0031] The two parts can have the same number of pixels.

[0032] Each of the two parts may have a non-square shape.

[0033] The two parts can be determined based on a straight line passing through the center of the current block along the predicted direction of the first prediction.

[0034] The selected portion may be based on selected portion information indicating one of the two portions.

[0035] When the prediction mode of the first prediction is a non-directional mode, the second prediction may not be used.

[0036] The second prediction may not be used when the second prediction usage information indicating whether the second prediction will be used to encode the current block indicates that the second prediction will not be used.

[0037] When the second prediction is not used, the first prediction can be a prediction for the entire current block.

[0038] The second prediction signal generated by the second prediction for the aforementioned portion can be added to the reconstructed residual signal.

[0039] The first prediction can be a prediction for the remaining portion of the current block excluding the aforementioned portion.

[0040] The first prediction signal generated by the first prediction for the remaining portion can be added to the reconstructed residual signal.

[0041] The reconstructed block can be generated based on a second prediction signal generated via a second prediction for the portion and a first prediction signal generated via a first prediction for the remaining portion.

[0042] Beneficial effects

[0043] A method and apparatus for reducing prediction errors based on prediction units due to prediction are provided.

[0044] A method and apparatus are provided to reduce prediction errors caused by the prediction direction of a prediction pattern.

[0045] A method and apparatus are provided for performing a selective second prediction on a prediction error corresponding to a first prediction made along a first prediction mode.

[0046] A method and apparatus are provided for partitioning a current block into two parts having the same number of pixels and performing a second prediction on the two parts.

[0047] A method and apparatus are provided for using a second prediction on a portion of a current block where the prediction error caused by a first prediction is large.

[0048] Embodiments of a method and apparatus designed to reduce prediction errors in the current block by using a second prediction are provided.

[0049] A method and apparatus are provided for providing higher compression performance for the current block by using a second prediction. Attached Figure Description

[0050] Figure 1 This is a block diagram illustrating the configuration of an embodiment of an encoding device to which the present invention is applied;

[0051] Figure 2 This is a block diagram illustrating the configuration of an embodiment of the decoding device to which the present invention is applied;

[0052] Figure 3 It is a schematic diagram illustrating the partitioning structure of an image as it is encoded and decoded;

[0053] Figure 4 This is a diagram showing the shape of the prediction unit (PU) that the coding unit (CU) can include;

[0054] Figure 5 This is a diagram showing the shape of a transformation unit (TU) that can be included in a CU;

[0055] Figure 6 This is a diagram illustrating an embodiment used to explain the intra-frame prediction process;

[0056] Figure 7 It is a diagram used to explain the location of reference samples used in the intra-frame prediction process;

[0057] Figure 8 This is a diagram illustrating an embodiment used to explain the inter-frame prediction process;

[0058] Figure 9 This is a diagram illustrating the prediction error when the original image is predicted along the vertical direction according to an embodiment;

[0059] Figure 10 This is a configuration diagram of an encoding device according to an embodiment;

[0060] Figure 11 This is a flowchart of the encoding method according to an embodiment;

[0061] Figure 12 This is a diagram illustrating the use of intra-frame prediction for first and second predictions according to an embodiment;

[0062] Figure 13 This is a flowchart of a residual block generation method according to an embodiment;

[0063] Figure 14 This is a flowchart of a reconstruction block generation method according to an embodiment;

[0064] Figure 15 This is a diagram illustrating the encoding process according to an embodiment;

[0065] Figure 16 This is a configuration diagram of the decoding device according to an embodiment;

[0066] Figure 17 This is a flowchart of a decoding method according to an embodiment.

[0067] Best practice

[0068] The following exemplary embodiments will be described in detail with reference to the accompanying drawings, which illustrate specific embodiments. These embodiments are described to enable those skilled in the art to readily practice them. It should be noted that the various embodiments differ from one another but are not necessarily mutually exclusive. For example, the particular shapes, structures, and characteristics described herein may be implemented in other embodiments without departing from the spirit and scope of the various embodiments associated with one embodiment. Furthermore, it should be understood that the position or arrangement of the various components in each disclosed embodiment can be changed without departing from the spirit and scope of the embodiments. Therefore, the appended detailed description is not intended to limit the scope of this disclosure, and the scope of the exemplary embodiments is defined only by the appended claims and their equivalents (provided they are properly described).

[0069] In the accompanying drawings, similar reference numerals are used to designate the same or similar functions in various respects. The shape, size, etc., of the components in the drawings may be exaggerated for clarity of description.

[0070] It will be understood that when a component is referred to as being "connected" or "coupled" to another component, the component may be directly connected or coupled to the other component, or there may be an intermediate component. Furthermore, it should be noted that in exemplary embodiments, the expression describing a component as "comprising" a particular component means that other components may be included within the scope or spirit of the practice of the exemplary embodiments, but does not exclude the presence of components other than the particular component.

[0071] Terms such as "first" and "second" may be used to describe various components, but components are not limited by these terms. These terms are used only to distinguish one component from another. For example, without departing from the scope of this specification, a first component may be referred to as a second component. Similarly, a second component may be referred to as a first component.

[0072] Furthermore, the components described in the embodiments are shown independently to represent different functional features, but this does not mean that each component is formed by a single piece of hardware or software. That is, for ease of description, multiple components are arranged and included separately. For example, at least two of the multiple components may be integrated into a single component. Conversely, a component may be divided into multiple components. Embodiments in which multiple components are integrated or some components are separated are included within the scope of this specification, provided they do not depart from its spirit.

[0073] Furthermore, some components are not essential for performing necessary functions, but may be optional components used only to improve performance. Embodiments may be implemented using only the essential components necessary to achieve the essence of the embodiment. For example, a structure that includes only essential components and excludes optional components used only to improve performance is also included within the scope of the embodiments.

[0074] The embodiments will now be described in detail with reference to the accompanying drawings, enabling those skilled in the art to readily practice the embodiments. In the following description of the embodiments, detailed descriptions of well-known functions or configurations that are considered to obscure key points of this specification will be omitted.

[0075] In the following text, "image" may refer to a single frame that constitutes part of a video, or it may refer to the video itself. For example, "encoding and / or decoding of an image" may mean "encoding and / or decoding of a video," and may also mean "encoding and / or decoding of any one of the multiple images that constitute a video."

[0076] In the following text, the terms “video” and “moving footage” may be used to have the same meaning and may be used interchangeably.

[0077] In the following text, the terms “image,” “picture,” “frame,” and “screen” may be used to have the same meaning and may be used interchangeably.

[0078] In the following embodiments, specific information, data, flags, elements, and attributes may have their own values. The value 0 corresponding to each of the information, data, flags, elements, and attributes may indicate logical false or a first predefined value. In other words, the value "0" (logical false) and the first predefined value are interchangeable. The value "1" corresponding to each of the information, data, flags, elements, and attributes may indicate logical true or a second predefined value. In other words, the value "1" (logical true) and the second predefined value are interchangeable.

[0079] When variables such as i or j are used to indicate rows, columns, or indices, the value i can be an integer 0 or greater than 0, or an integer 1 or greater than 1. In other words, in the embodiments, each of the rows, columns, and indices can be counted starting from 0, or can be counted starting from 1.

[0080] The terminology used in the embodiments will be described below.

[0081] Unit: "Unit" can refer to a unit of image encoding and decoding. The terms "unit" and "block" can have the same meaning. Furthermore, the terms "unit" and "block" are interchangeable.

[0082] A cell (or block) can be an M×N sample matrix. M and N can both be positive integers. The term "cell" typically refers to an array of two-dimensional (2D) samples. The term "sample" can be a pixel or a pixel value.

[0083] The terms “pixel” and “sample” can be used interchangeably and have the same meaning.

[0084] In the process of image encoding and decoding, a "unit" can be a region created by partitioning an image. A single image can be partitioned into multiple units. During image encoding and decoding, predefined processing can be performed for each unit, depending on its type. Based on function, units can be classified as macrounits, coding units (CUs), prediction units (PUs), and transform units (TUs). A single unit can also be further partitioned into lower-level units with sizes smaller than the unit itself.

[0085] Cell partitioning information may include information about the depth of the cell. Depth information may indicate the number and / or extent to which the cell is partitioned.

[0086] A single unit can be hierarchically partitioned into multiple lower-level units, each possessing depth information based on a tree structure. In other words, the unit and the lower-level units created by partitioning that unit can correspond to a node and its child nodes, respectively. Each partitioned lower-level unit can possess depth information. The depth information of a unit indicates the number and / or degree to which it has been partitioned; therefore, the partitioning information of a lower-level unit can include information about its size.

[0087] In a tree structure, the top node corresponds to the initial node before partitioning. The top node can be called the "root node." Furthermore, the root node can have a minimum depth value. Here, the depth of the top node can be level "0".

[0088] A node with a depth of level "1" can represent a cell created when the initial cell is partitioned once. A node with a depth of level "2" can represent a cell created when the initial cell is partitioned twice.

[0089] – A leaf node of depth “n” can represent the cells generated when the initial cell is partitioned n times.

[0090] – A leaf node can be a bottom node, and it cannot be further partitioned. The depth of a leaf node can be the maximum level. For example, a predefined value for the maximum level could be 3.

[0091] – Transform Unit (TU): A TU can be the basic unit for residual signal encoding and / or residual signal decoding (such as transform, inverse transform, quantization, dequantization, transform coefficient encoding, and transform coefficient decoding). A single TU can be partitioned into multiple TUs, wherein each of the multiple TUs has a smaller size.

[0092] – Prediction Unit (PU): A PU can be the basic unit in the execution of prediction or compensation. A PU can be partitioned into multiple partitions. These multiple partitions can also be the basic units in the execution of prediction or compensation. The partitions created by partitioning the PU can also be prediction units.

[0093] – Reconstructed neighboring units: Reconstructed neighboring units can be units that have been previously encoded or decoded and reconstructed near the target unit being encoded or decoded. Reconstructed neighboring units can be units that are spatially or temporally adjacent to the target unit.

[0094] – Prediction cell partitioning: Prediction cell partitioning can represent the shape of the PU being partitioned.

[0095] – Parameter set: The parameter set can correspond to header information about the structure of the bitstream. For example, the parameter set may include sequence parameter set, frame parameter set, adaptation parameter set, etc.

[0096] – Rate-distortion optimization: Encoding devices can use rate-distortion optimization to provide higher coding efficiency by utilizing a combination of the following: CU size, prediction mode, prediction cell size, motion information, and TU size.

[0097] – Rate-distortion optimization scheme: This scheme calculates the rate-distortion cost of each combination to select the optimal combination from these combinations. The rate-distortion cost can be calculated using Equation 1 below. Generally, the combination that minimizes the rate-distortion cost is selected as the optimal combination under the rate-distortion optimization method.

[0098] [Equation 1]

[0099] D+λ*R

[0100] Here, D can represent distortion. D can be the average of the squared differences (mean squared error) between the original transform coefficients and the reconstructed transform coefficients in the transform block.

[0101] R represents the bit rate, which can be represented using relevant context information.

[0102] λ represents the Lagrange multiplier. R can include not only coding parameter information (such as prediction mode, motion information, and coding block flags), but also bits generated from encoding the transform coefficients.

[0103] Encoding devices perform processes such as inter-frame prediction and / or intra-frame prediction, transform, quantization, entropy coding, dequantization, and inverse transform in order to compute accurate D and R, but these processes greatly increase the complexity of the encoding device.

[0104] – Reference frame: A reference frame can be an image used for inter-frame prediction or motion compensation. A reference frame can be a frame that includes reference units referenced by the target unit to perform inter-frame prediction or motion compensation. The terms "frame" and "image" can have the same meaning. Therefore, the terms "frame" and "image" are interchangeable.

[0105] – Reference Frame List: The reference frame list can be a list that includes reference frames used for inter-frame prediction or motion compensation. The type of reference frame list can be a merged list (LC), list 0 (L0), list 1 (L1), etc.

[0106] – Motion Vector (MV): MV can be a 2D vector used for inter-frame prediction. For example, it can be expressed as (mv... x ,mv y MV is represented in the form of ) x It can indicate the horizontal component, mv y It can indicate the vertical component.

[0107] –MV can represent the offset between the target frame and the reference frame.

[0108] – Search Range: The search range can be a 2D region where a search for the MV is performed during inter-frame prediction. For example, the size of the search range can be M×N. M and N can both be positive integers.

[0109] Figure 1 This is a block diagram illustrating the configuration of an embodiment of an encoding device to which the present invention is applied.

[0110] Encoding device 100 can be a video encoding device or an image encoding device. Video may include one or more images (frames). Encoding device 100 can encode one or more images of the video sequentially over time.

[0111] Reference Figure 1 The encoding device 100 includes an inter-frame prediction unit 110, an intra-frame prediction unit 120, a switcher 115, a subtractor 125, a transform unit 130, a quantization unit 140, an entropy coding unit 150, an inverse quantization unit 160, an inverse transform unit 170, an adder 175, a filtering unit 180, and a reference frame buffer 190.

[0112] The encoding device 100 can perform encoding on the input image in intra-frame mode and / or inter-frame mode. The input image may be referred to as the "current image" as the target to be encoded.

[0113] Furthermore, the encoding device 100 can generate a bit stream including information about the encoding by encoding the input image, and can output the generated bit stream.

[0114] When intra-frame mode is used, switcher 115 can switch to intra-frame mode. When inter-frame mode is used, switcher 115 can switch to inter-frame mode.

[0115] The encoding device 100 can generate prediction blocks for input blocks in the input image. Furthermore, after generating the prediction blocks, the encoding device 100 can encode the residual between the input blocks and the prediction blocks. The input block can be referred to as the "current block" as the target to be encoded.

[0116] When the prediction mode is intra-frame mode, the intra-frame prediction unit 120 can use the pixel values ​​of previously encoded neighboring blocks around the current block as reference pixels. The intra-frame prediction unit 120 can use the reference pixels to perform spatial prediction on the current block and generate prediction samples for the current block via spatial prediction.

[0117] The inter-frame prediction unit 110 may include a motion prediction unit and a motion compensation unit.

[0118] When the prediction mode is inter-frame mode, the motion prediction unit can search for the region in the reference image that best matches the current block during motion prediction, and can derive motion vectors for the current block and the found region. The reference image can be stored in the reference frame buffer 190. More specifically, the reference image can be stored in the reference frame buffer 190 when the encoding and / or decoding of the reference image is processed.

[0119] The motion compensation unit can generate prediction blocks by performing motion compensation using motion vectors. Here, the motion vectors can be two-dimensional (2D) vectors used for inter-frame prediction. Furthermore, the motion vectors can represent the offset between the current image and the reference image.

[0120] Subtractor 125 generates a residual block, which is the residual between the input block and the prediction block. The residual block is also referred to as the "residual signal".

[0121] Transform unit 130 can generate transform coefficients by transforming the residual block and can output the generated transform coefficients. Here, the transform coefficients can be coefficient values ​​generated by transforming the residual block. When using transform skip mode, transform unit 130 can omit the operation of transforming the residual block.

[0122] By quantizing the transform coefficients, a quantized transform coefficient level can be generated. Here, in the embodiment, the quantized transform coefficient level can also be referred to as the "transform coefficient".

[0123] The quantization unit 140 can generate quantized transform coefficient levels by quantizing the transform coefficients according to quantization parameters. The quantization unit 140 can output the quantized transform coefficient levels. In this case, the quantization unit 140 can use a quantization matrix to quantize the transform coefficients.

[0124] The entropy coding unit 150 can generate a bitstream by performing probability-based entropy coding based on values ​​calculated by the quantization unit 140 and / or coding parameter values ​​calculated during the encoding process. The entropy coding unit 150 can output the generated bitstream.

[0125] In addition to the pixel information of the image, the entropy coding unit 150 can also perform entropy coding on the information required to decode the image. For example, the information required to decode the image may include syntax elements, etc.

[0126] Encoding parameters can be information required for encoding and / or decoding. Encoding parameters may include information encoded by the encoding device and transmitted to the decoding device, and may also include information derived during the encoding or decoding process. For example, information transmitted to the decoding device may include syntax elements.

[0127] For example, the encoding device may include values ​​or statistics such as prediction mode, motion vectors, reference frame index, coded block pattern, presence or absence of residual signals, transform coefficients, quantized transform coefficients, quantization parameters, block size, and block partitioning information. The prediction mode can be an intra-frame prediction mode or an inter-frame prediction mode.

[0128] The residual signal can represent the difference between the original signal and the predicted signal. Optionally, the residual signal can be a signal generated by transforming the difference between the original signal and the predicted signal. Optionally, the residual signal can be a signal generated by transforming and quantizing the difference between the original signal and the predicted signal. The residual block can be a block-based residual signal.

[0129] When applying entropy coding, fewer bits can be allocated to more frequently occurring symbols, and more bits can be allocated to less frequently occurring symbols. Because symbols are represented through this allocation, the size of the bit string used to encode the target symbol can be reduced. Therefore, entropy coding can improve the compression performance of video coding.

[0130] Furthermore, for entropy coding, coding methods such as Exponential Golomb, Context Adaptive Variable Length Coding (CAVLC), or Context Adaptive Binary Arithmetic Coding (CABAC) can be used. For example, entropy coding unit 150 can use a variable-length code / code (VLC) table to perform entropy coding. For example, entropy coding unit 150 can derive a binarization method for the target symbol. Furthermore, entropy coding unit 150 can derive a probabilistic model for the target symbol / bit. Entropy coding unit 150 can use the derived binarization method or probabilistic model to perform entropy coding.

[0131] Since the encoding device 100 performs encoding via inter-frame prediction, the encoded current image can be used as a reference image for another image to be processed subsequently. Therefore, the encoding device 100 can decode the encoded current image and store the decoded image as a reference image. For decoding, inverse quantization and inverse transform of the encoded current image can be performed.

[0132] The quantized coefficients can be dequantized by the dequantization unit 160 and inverse transformed by the inverse transform unit 170. The dequantized and inverse transformed coefficients can be added to the prediction block by the adder 175. The dequantized and inverse transformed coefficients are then added to the prediction block to generate the reconstructed block.

[0133] The reconstructed blocks can be filtered by the filtering unit 180. The filtering unit 180 can apply one or more filters, including a deblocking filter, a sample adaptive offset (SAO) filter, and an adaptive loop filter (ALF), to the reconstructed blocks or reconstructed images. The filtering unit 180 may also be referred to as an "adaptive in-loop filter".

[0134] Deblocking filters eliminate block distortion at block boundaries. SAO filters add appropriate offset values ​​to pixel values ​​to compensate for coding errors. ALF filters perform filtering based on comparisons between reconstructed and original blocks. Reconstructed blocks filtered by filtering unit 180 can be stored in reference frame buffer 190.

[0135] Figure 2 This is a block diagram illustrating the configuration of an embodiment of the decoding device to which the present invention is applied.

[0136] Decoding device 200 can be a video decoding device or an image decoding device.

[0137] Reference Figure 2 The decoding device 200 may include an entropy decoding unit 210, an inverse quantization unit 220, an inverse transform unit 230, an intra-frame prediction unit 240, an inter-frame prediction unit 250, an adder 255, a filtering unit 260, and a reference frame buffer 270.

[0138] Decoding device 200 can receive a bitstream output from encoding device 100. Decoding device 200 can perform decoding on the bitstream in intra-frame mode and / or inter-frame mode. Furthermore, decoding device 200 can generate a reconstructed image via decoding and can output the reconstructed image.

[0139] For example, a switcher can be used to switch between intra-frame mode and inter-frame mode based on the prediction mode used for decoding. When the prediction mode used for decoding is intra-frame mode, the switcher can be operated to switch to intra-frame mode. When the prediction mode used for decoding is inter-frame mode, the switcher can be operated to switch to inter-frame mode.

[0140] The decoding device 200 can obtain reconstructed residual blocks from the input bitstream and can generate prediction blocks. When the reconstructed residual blocks and prediction blocks are obtained, the decoding device 200 can generate reconstructed blocks by adding the reconstructed residual blocks to the prediction blocks.

[0141] The entropy decoding unit 210 can generate symbols by performing entropy decoding on the bitstream based on a probability distribution. The generated symbols may include quantized coefficient format symbols. Here, the entropy decoding method can be similar to the entropy encoding method described above. That is, the entropy decoding method can be the inverse process of the entropy encoding method described above.

[0142] The quantized coefficients can be dequantized by the dequantization unit 220. Furthermore, the dequantized coefficients can be inversely transformed by the inverse transform unit 230. As a result of the dequantization and inverse transform of the quantized coefficients, a reconstructed residual block can be generated. Here, the dequantization unit 220 can apply the quantization matrix to the quantized coefficients.

[0143] When using intra-frame mode, intra-frame prediction unit 240 can generate a prediction block by performing spatial prediction using the pixel values ​​of previously encoded neighboring blocks around the current block.

[0144] The inter-frame prediction unit 250 may include a motion compensation unit. When using inter-frame mode, the motion compensation unit can generate prediction blocks by performing motion compensation using motion vectors and a reference image. The reference image may be stored in a reference frame buffer 270.

[0145] The reconstructed residual block and the predicted block can be added to each other by adder 255. Adder 255 produces a reconstructed block by adding the reconstructed residual block and the predicted block.

[0146] The reconstructed blocks can be filtered by the filtering unit 260. The filtering unit 260 can apply one or more of a deblocking filter, a SAO filter, and an ALF filter to the reconstructed blocks or reconstructed frames. The filtering unit 260 can output a reconstructed image (frame). The reconstructed image can be stored in the reference frame buffer 270 and can then be used for inter-frame prediction.

[0147] Figure 3 This is a schematic diagram illustrating the image partitioning structure when an image is encoded and decoded.

[0148] To effectively partition an image, coding units (CUs) can be used in encoding and decoding. The term "unit" can be used to collectively specify 1) a block comprising image samples and 2) a syntax element. For example, "partition of a unit" can mean "partition of a block corresponding to a unit".

[0149] Reference Figure 3 Image 300 is sequentially partitioned into units corresponding to maximum coding units (LCUs), and the partitioning structure of image 300 can be determined based on the LCUs. Here, LCU can be used to have the same meaning as coding tree unit (CTU).

[0150] The partitioning structure represents the distribution of coding units (CUs) in the LCU 310 for effectively encoding an image. This distribution can be determined based on whether a single CU will be partitioned into four CUs. The horizontal and vertical dimensions of each CU resulting from partitioning can be half the horizontal and vertical dimensions of the CU before partitioning. Each partitioned CU can be recursively partitioned into four CUs, and in the same manner, the horizontal and vertical dimensions of the four CUs are halved.

[0151] Here, the partitioning of the CU can be performed recursively until a predefined depth is reached. Depth information can be information indicating the size of the CU. Depth information can be stored for each CU. For example, the depth of the LCU can be 0, and the depth of the minimum coding unit (SCU) can be a predefined maximum depth. Here, as mentioned above, the LCU can be a CU with the maximum coding unit size, and the SCU can be a CU with the minimum coding unit size.

[0152] Partitioning begins at LCU 310. The depth of a CU can be increased by "1" whenever its horizontal and vertical dimensions are halved through partitioning. For each depth, an unpartitioned CU can have a size of 2N×2N. Furthermore, when a CU is partitioned, a 2N×2N CU can be partitioned into four CUs, each with a size of N×N. The size N is halved each time the depth increases by 1.

[0153] Reference Figure 3 An LCU with a depth of 0 can have 64×64 pixels. 0 can be the minimum depth. An SCU with a depth of 3 can have 8×8 pixels. 3 can be the maximum depth. Here, a CU with 64×64 pixels as an LCU can be represented by depth 0. A CU with 32×32 pixels can be represented by depth 1. A CU with 16×16 pixels can be represented by depth 2. A CU with 8×8 pixels as an SCU can be represented by depth 3.

[0154] Furthermore, information about whether a corresponding CU is partitioned can be represented using the CU's partition information. Partition information can be 1 bit. All CUs except the SCU can include partition information. For example, when a CU is not partitioned, the value of its partition information can be 0. When a CU is partitioned, the value of its partition information can be 1.

[0155] Figure 4 This is a diagram showing the shape of the prediction unit (PU) that the coding unit (CU) can include.

[0156] Within the control units (CUs) partitioned from the control unit (LCU), CUs that are no longer partitioned can be divided into one or more prediction units (PUs). This partitioning can also be referred to as "partitioning".

[0157] A PU (Program Unit) can be the basic unit used for prediction. A PU can be encoded and decoded in any of the following modes: skip mode, inter-frame mode, and intra-frame mode. A PU can be partitioned into various shapes according to each mode.

[0158] In skip mode, partitioning may not be required in the CU. Skip mode also supports a 2N×2N mode 410 without partitioning, where the PU and CU have the same size.

[0159] In inter-frame mode, eight types of partition shapes can exist in the CU. For example, in inter-frame mode, 2N×2N mode 410, 2N×N mode 415, N×2N mode 420, N×N mode 425, 2N×nU mode 430, 2N×nD mode 435, nL×2N mode 440 and nR×2N mode 445 are supported.

[0160] In intra-frame mode, 2N×2N mode 410 and N×N mode 425 are supported.

[0161] In 2N×2N mode 410, a PU with a size of 2N×2N can be encoded. A PU with a size of 2N×2N can represent a PU with the same size as the CU. For example, a PU with a size of 2N×2N can have a size of 64×64, 32×32, 16×16, or 8×8.

[0162] In N×N mode 425, PUs with an N×N size can be encoded.

[0163] For example, in intra-frame prediction, when the PU size is 8×8, the PUs from four partitions can be encoded. The size of the PU from each partition can be 4×4.

[0164] When encoding a PU in intra-frame mode, the PU can be encoded using any of several intra-frame prediction modes. For example, HEVC technology provides 35 intra-frame prediction modes, and the PU can be encoded in any of these 35 intra-frame prediction modes.

[0165] The rate-distortion cost can be used to determine which of the 2N×2N modes 410 and N×N modes 425 will be used to encode the PU.

[0166] Encoding device 100 can perform encoding operations on a PU of size 2N×2N. Here, the encoding operation can be an operation of encoding the PU in each of a plurality of intra-prediction modes that can be used by encoding device 100. Through the encoding operation, the optimal intra-prediction mode for the PU of size 2N×2N can be obtained. The optimal intra-prediction mode can be the intra-prediction mode that results in the minimum rate-distortion cost when encoding the PU of size 2N×2N among the plurality of intra-prediction modes that can be used by encoding device 100.

[0167] Furthermore, the encoding device 100 can sequentially perform encoding operations on each PU obtained by performing N×N partitioning. Here, the encoding operation can be an operation of encoding the PU in each of a plurality of intra-prediction modes that can be used by the encoding device 100. Through the encoding operation, the optimal intra-prediction mode for the N×N PU can be obtained. The optimal intra-prediction mode can be the intra-prediction mode that produces the minimum rate-distortion cost when encoding the N×N PU among the plurality of intra-prediction modes that can be used by the encoding device 100.

[0168] The encoding device 100 can determine which of the PUs, one of size 2N×2N and one of size N×N, will be encoded based on a comparison between the rate-distortion cost of a PU of size 2N×2N and the rate-distortion cost of a PU of size N×N.

[0169] Figure 5 This is a diagram showing the shape of a transformation unit (TU) that can be included in a CU.

[0170] A transform unit (TU) is the basic unit in a CU used for processes such as transform, quantization, inverse transform, dequantization, entropy coding, and entropy decoding. A TU can have a square or rectangular shape.

[0171] Within a CU partitioned from an LCU, CUs that are no longer designated as CUs can be partitioned into one or more TUs. Here, the partitioning structure of a TU can be a quadtree structure. For example, ... Figure 5 As shown, a single CU 510 can be partitioned once or more according to a quadtree structure. Through this partitioning, a single CU 510 can be composed of TUs of various sizes.

[0172] In the encoding device 100, a 64×64 coding tree unit (CTU) can be partitioned into multiple smaller CUs using a recursive tree structure. A single CU can be partitioned into four CUs of the same size. Each CU can be recursively partitioned and can have a quadtree structure.

[0173] A CU can have a given depth. When a CU is partitioned, the CU resulting from the partitioning operation can have a depth obtained by increasing the depth of the partitioned CU by 1.

[0174] For example, the depth of the CU can have a value ranging from 0 to 3. Depending on the depth of the CU, the size of the CU can range from 64×64 to 8×8.

[0175] By recursively partitioning the CU, the optimal partitioning method that results in the minimum rate distortion cost can be selected.

[0176] Figure 6This is a diagram illustrating an embodiment used to explain the intra-frame prediction process.

[0177] from Figure 6 The radially extending arrow from the center of the diagram indicates the prediction direction of the intra-prediction mode. Furthermore, the numbers shown near the arrows represent examples of mode values ​​assigned to the intra-prediction mode or the prediction direction assigned to the intra-prediction mode.

[0178] Intra-frame coding and / or decoding can be performed using reference samples from cells adjacent to the target cell. A neighboring cell can be a neighboring reconstructed cell. For example, intra-frame coding and / or decoding can be performed using the values ​​of reference samples included in each neighboring reconstructed cell or the coding parameters of the neighboring reconstructed cells.

[0179] Encoding device 100 and / or decoding device 200 can generate prediction blocks by performing intra-frame prediction on target units based on information about samples in the current frame. When intra-frame prediction is performed, encoding device 100 and / or decoding device 200 can generate prediction blocks for target units by performing intra-frame prediction based on information about samples in the current frame. When intra-frame prediction is performed, encoding device 100 and / or decoding device 200 can perform directional prediction and / or non-directional prediction based on at least one reconstructed reference sample.

[0180] A prediction block can represent a block generated as a result of performing intra-frame prediction. A prediction block can correspond to at least one of CU, PU, ​​and TU.

[0181] The cells of the prediction block may have a size corresponding to at least one of CU, PU, ​​and TU. The prediction block may have a square shape with a size of 2N×2N or N×N. The size N×N may include sizes such as 4×4, 8×8, 16×16, 32×32, 64×64, etc.

[0182] Optionally, the prediction block can be a square block with a size of 2×2, 4×4, 16×16, 32×32, 64×64, etc., or a rectangular block with a size of 2×8, 4×8, 2×16, 4×16, 8×16, etc.

[0183] Intra-prediction can be performed based on the intra-prediction mode used for the target cell. The number of intra-prediction modes that a target cell can have can be a predefined fixed value, or it can be a value determined differently based on the attributes of the prediction block. For example, the attributes of the prediction block may include the size of the prediction block, the type of the prediction block, etc.

[0184] For example, the number of intra-prediction modes can be fixed at 35, regardless of the size of the prediction cell. Alternatively, the number of intra-prediction modes can be, for example, 3, 5, 9, 17, 34, 35, or 36.

[0185] like Figure 6 As shown, the intra-frame prediction modes can include two non-directional modes and 33 directional modes. The two non-directional modes can include DC mode and planar mode.

[0186] For example, in the vertical mode with a mode value of 26, prediction can be performed along the vertical direction based on the pixel values ​​of the reference sample. Similarly, in the horizontal mode with a mode value of 10, prediction can be performed along the horizontal direction based on the pixel values ​​of the reference sample. And again, in the vertical mode with a mode value of 26, prediction can be performed along the vertical direction based on the pixel values ​​of the reference sample.

[0187] Even in orientation modes other than those described above, the encoding device 100 and the decoding device 200 can still perform intra-frame prediction on the target cell using reference samples based on the angle corresponding to the orientation mode.

[0188] Intra-prediction modes located to the right of the vertical mode can be called "vertical-right mode". Intra-prediction modes located below the horizontal mode can be called "horizontal-bottom mode". For example, in Figure 6 In the frame, an intra-prediction mode with a mode value of 27, 28, 29, 30, 31, 32, 33, and 34 can be a vertical-right mode 613. An intra-prediction mode with a mode value of 2, 3, 4, 5, 6, 7, 8, and 9 can be a horizontal-down mode 616.

[0189] Non-directional modes can include DC mode and planar mode. For example, the mode value for DC mode can be 1. The mode value for planar mode can be 0.

[0190] Orientation modes can include corner modes. Among various intra-frame prediction modes, modes other than DC mode and planar mode can be orientation modes.

[0191] In DC mode, a prediction block can be generated based on the average pixel values ​​of multiple reference samples. For example, the pixel values ​​of the prediction block can be determined based on the average pixel values ​​of multiple reference samples.

[0192] The number of intra-prediction modes and the mode values ​​of each intra-prediction mode described above are merely exemplary. The number of intra-prediction modes and the mode values ​​of each intra-prediction mode described above may be defined differently depending on the embodiment, implementation, and / or requirements.

[0193] The number of intra-frame prediction modes can vary depending on the type of color component. For example, the number of prediction modes can vary depending on whether the color component is a luma signal or a chroma signal.

[0194] Figure 7 It is a diagram used to explain the location of reference samples used in intra-frame prediction.

[0195] Figure 7 The locations of reference samples used for intra-frame prediction of the target cell are shown. (Refer to...) Figure 7 The reconstruction reference samples used for intra-frame prediction of the current block may include, for example, the lower left reference sample 731, the left reference sample 733, the upper left reference sample 735, the upper reference sample 737, and the upper right reference sample 739.

[0196] For example, left reference sample 733 may represent a reconstructed reference sample adjacent to the left side of the target cell. Top reference sample 737 may represent a reconstructed reference sample adjacent to the top of the target cell. Top-left reference sample 735 may represent a reconstructed reference sample located at the top-left corner of the target cell. Bottom-left reference sample 731 may represent a reference sample located below the left sample line formed by left reference sample 733, among the samples on that line. Top-right reference sample 739 may represent a reference sample located to the right of the top sample line, among the samples on the same line as the top sample line formed by top reference sample 737.

[0197] When the size of the target cell is N×N, the number of reference samples 731 at the lower left, 733 at the left, 737 at the top, and 739 at the upper right can all be N.

[0198] A prediction block can be generated by performing intra-frame prediction on the target cell. The process of generating a prediction block may include determining the values ​​of the pixels in the prediction block. The target cell and the prediction block can have the same size.

[0199] The reference sample used for intra-prediction of the target cell can be changed according to the intra-prediction mode of the target cell. The direction of the intra-prediction mode can represent the dependency between the reference sample and the pixels of the prediction block. For example, the value of a specified reference sample can be used as the value of one or more specified pixels in the prediction block. In this case, the specified reference sample and the one or more specified pixels in the prediction block can be samples and pixels located on a straight line along the direction of the intra-prediction mode. In other words, the value of the specified reference sample can be copied as the value of a pixel located in the direction opposite to the direction of the intra-prediction mode. Optionally, the value of a pixel in the prediction block can be the value of a reference sample located in the direction of the intra-prediction mode relative to the position of that pixel.

[0200] For example, when the intra-prediction mode of the target cell is the vertical mode with a mode value of 26, the upper reference sample 737 can be used for intra-prediction. When the intra-prediction mode is vertical, the value of a pixel in the prediction block can be the value of a reference pixel vertically above that pixel. Therefore, the upper reference sample 737 adjacent to the top of the target cell can be used for intra-prediction. Furthermore, the value of a pixel in a row of the prediction block can be the same as the value of the pixel at the upper reference sample 737.

[0201] For example, when the intra-prediction mode of the current block is the horizontal mode with a mode value of 10, the left reference sample 733 can be used for intra-prediction. When the intra-prediction mode is horizontal, the value of a pixel in the prediction block can be the value of a reference pixel horizontally to the left of that pixel. Therefore, the left reference sample 733 adjacent to the left of the target cell can be used for intra-prediction. Furthermore, the value of a pixel in a column of the prediction block can be the same as the value of the pixel in the left reference sample 733.

[0202] For example, when the mode value of the intra-prediction mode for the current block is 18, at least some of the left reference samples 733, the top-left reference sample 735, and the top reference sample 737 can be used for intra-prediction. When the mode value of the intra-prediction mode is 18, the value of a pixel in the prediction block can be the value of a reference pixel located diagonally to the top-left corner of that pixel.

[0203] Furthermore, when an intra-prediction mode with mode values ​​corresponding to 27, 28, 29, 30, 31, 32, 33 or 34 is used, at least some of the upper right reference samples 739 can be used for intra-prediction.

[0204] Furthermore, when an intra-prediction mode with mode values ​​corresponding to 2, 3, 4, 5, 6, 7, 8, or 9 is used, at least some of the lower left reference samples 731 can be used for intra-prediction.

[0205] Furthermore, when an intra-prediction mode with a corresponding mode value of any one of 11 to 25 is used, the upper left reference sample 735 can be used for intra-prediction.

[0206] The number of reference samples used to determine the pixel value of a pixel in the prediction block can be 1, 2, or more.

[0207] As described above, the pixel value of a pixel in a prediction block can be determined based on the pixel's position and the position of a reference sample indicated by the direction of the intra-prediction mode. When both the pixel's position and the position of the reference sample indicated by the direction of the intra-prediction mode are integer positions, the value of a reference sample indicated by the integer position can be used to determine the pixel value of the pixel in the prediction block.

[0208] When the pixel position and the position of the reference sample indicated by the direction of the intra-prediction mode are not integer positions, an interpolated reference sample can be generated based on the two reference samples closest to that reference sample position. The value of the interpolated reference sample can be used to determine the pixel value of the pixel in the prediction block. In other words, when the pixel position in the prediction block and the position of the reference sample indicated by the direction of the intra-prediction mode indicate the position between two reference samples, an interpolated value based on the values ​​of those two samples can be generated.

[0209] The predicted block generated by prediction may differ from the original target cell. In other words, there may be prediction errors, which are the differences between the target cell and the predicted cell, and there may also be prediction errors between the pixels of the target cell and the pixels of the predicted block. For example, in the case of directional intra-frame prediction, the greater the distance between the pixels of the predicted block and the reference sample, the greater the possible prediction error. Such prediction errors can lead to discontinuities between the generated predicted block and its neighboring blocks.

[0210] To reduce prediction error, filtering operations can be used for prediction blocks. These filtering operations can be configured to adaptively apply filters to regions within the prediction block that are considered to have large prediction errors. For example, regions considered to have large prediction errors could be the boundaries of the prediction block. Furthermore, the regions within the prediction block considered to have large prediction errors can vary depending on the intra-prediction mode, and the characteristics of the filters can also vary depending on the intra-prediction mode.

[0211] Figure 8 This is a diagram illustrating an embodiment used to explain the intra-frame prediction process.

[0212] Figure 8 The rectangles shown can represent images (or screens). Furthermore, in... Figure 8 In the image, the arrow indicates the prediction direction. That is, each image can be encoded and / or decoded based on the prediction direction.

[0213] Images (or frames) can be classified according to their encoding type into intra-frame frames (I-frames), one-way predictive frames or predictive-coded frames (P-frames), and two-way predictive frames or two-way predictive-coded frames (B-frames). Each frame can be encoded according to its encoding type.

[0214] When the image to be encoded is an I-frame, the image itself can be encoded without inter-frame prediction. When the image to be encoded is a P-frame, it can be encoded via inter-frame prediction using only reference frames in the forward direction. When the image to be encoded is a B-frame, it can be encoded via inter-frame prediction using reference frames in both the forward and backward directions, or via inter-frame prediction using reference frames in either the forward or backward direction.

[0215] P-frames and B-frames that are encoded and / or decoded using reference frames can be considered as images using inter-frame prediction.

[0216] The following will describe in detail the inter-frame prediction in inter-frame mode according to the embodiments.

[0217] In inter-frame mode, encoding device 100 and decoding device 200 can perform prediction and / or motion compensation on encoded target units and decoded target units. For example, encoding device 100 or decoding device 200 can perform prediction and / or motion compensation by using motion information of neighboring reconstructed blocks as motion information of encoded target units or decoded target units. Here, the encoded target unit or the decoded target unit may represent a prediction unit and / or a prediction unit partition.

[0218] Inter-frame prediction can be performed using reference frames and motion information. Furthermore, inter-frame prediction can utilize the skip modes described above.

[0219] The reference frame can be at least one frame preceding or following the current frame. Here, inter-frame prediction can be performed on blocks in the current frame based on the reference frame. Here, the reference frame can represent the image used to predict the blocks.

[0220] Here, a region in the reference frame can be specified by using the reference frame index refIdx, which indicates the reference frame, and the motion vector, which will be described later.

[0221] Inter-frame prediction can select a reference frame and a reference block within that reference frame corresponding to the current block, and can use the selected reference block to generate a prediction block for the current block. The current block can be a block within the current frame that is the target to be encoded or decoded.

[0222] Motion information can be derived by each of the encoding device 100 and the decoding device 200 during inter-frame prediction. Furthermore, the derived motion information can be used to perform inter-frame prediction.

[0223] Here, the encoding device 100 and the decoding device 200 can improve encoding efficiency and / or decoding efficiency by using motion information of neighboring reconstructed blocks and / or motion information of col blocks (col blocks). A col block can be a block corresponding to the current block in a previously reconstructed col frame (col frame).

[0224] A neighboring reconstructed block can be a block existing in the current frame, and can be a block that has previously been reconstructed via encoding and / or decoding. A reconstructed block can be a neighboring block adjacent to the current block and / or a block located at the outer corner of the current block. Here, "a block located at the outer corner of the current block" can mean a block that is vertically adjacent to a neighboring block that is horizontally adjacent to the current block, or a block that is horizontally adjacent to a neighboring block that is vertically adjacent to the current block.

[0225] For example, a neighboring reconstruction unit (block) can be a unit located to the left of the target unit, a unit located above the target unit, a unit located at the lower left corner of the target unit, a unit located at the upper right corner of the target unit, or a unit located at the upper left corner of the target unit.

[0226] Each of the encoding device 100 and the decoding device 200 can determine a block in the col frame that exists at a spatial position corresponding to the current block, and can determine a predefined relative position based on the determined block. The predefined relative position can be an internal and / or external position of the block existing at the spatial position corresponding to the current block. Furthermore, each of the encoding device 100 and the decoding device 200 can deduce the col block based on the determined predefined relative position. Here, the col frame can be any one of one or more reference frames included in the reference frame list.

[0227] A block in the reference frame can exist at a spatially corresponding position in the reconstructed reference frame. In other words, the position of the current block in the current frame and the position of the block in the reference frame can correspond to each other. Hereinafter, the motion information of the blocks included in the reference frame may be referred to as "temporal motion information".

[0228] The method used to derive motion information can vary depending on the prediction mode of the current block. For example, as prediction modes applied to inter-frame prediction, there may be Advanced Motion Vector Prediction Factor (AMVP) mode, merging mode, etc.

[0229] For example, when the AMVP mode is used as the prediction mode, each of the encoding device 100 and the decoding device 200 can use the motion vectors of neighboring reconstructed blocks and / or the motion vectors of col blocks to generate a candidate list of prediction motion vectors. The motion vectors of neighboring reconstructed blocks and / or the motion vectors of col blocks can be used as candidates for prediction motion vectors.

[0230] The bitstream generated by encoding device 100 may include a predicted motion vector index. The predicted motion vector index may represent the best predicted motion vector selected from the predicted motion vector candidates included in the predicted motion vector candidate list. The predicted motion vector index can be transmitted from encoding device 100 to decoding device 200 via the bitstream.

[0231] The decoding device 200 can use the predicted motion vector index to select the predicted motion vector of the current block from the predicted motion vector candidates included in the predicted motion vector candidate list.

[0232] Encoding device 100 can calculate the motion vector difference (MVD) between the motion vector of the current block and the predicted motion vector, and can encode the MVD. The bitstream can include the encoded MVD. The MVD can be sent from encoding device 100 to decoding device 200 via the bitstream. Here, decoding device 200 can decode the received MVD. Decoding device 200 can use the sum of the decoded MVD and the predicted motion vector to deduce the motion vector of the current block.

[0233] The bitstream may include a reference frame index, etc., for indicating a reference frame. The reference frame index can be sent from the encoding device 100 to the decoding device 200 via the bitstream. The decoding device 200 can use motion information of neighboring blocks to predict the motion vector of the current block, and can use the difference between the predicted motion vector and the motion vector (MVD) to deduce the motion vector of the current block. The decoding device 200 can generate a predicted block for the current block based on the deduced motion vector and the reference frame index information.

[0234] Since motion information from neighboring reconstructed blocks can be used to encode and decode the target unit, the encoding device 100 may not encode the motion information of the target unit separately in a specific inter-frame prediction mode. If the motion information of the target unit is not encoded, the number of bits sent to the decoding device 200 can be reduced, and encoding efficiency can be improved. For example, skip modes and / or merge modes may exist as inter-frame prediction modes that do not encode the motion information of the target unit. Here, each of the encoding device 100 and the decoding device 200 may use an identifier and / or index indicating one of a plurality of neighboring reconstructed blocks, wherein the motion information of said neighboring reconstructed block will be used as the motion information of the target unit.

[0235] There exists a merging method as another example of a method for deriving motion information. The term "merging" can refer to merging the motion of multiple blocks. The term "merging" can also mean that the motion information of one block is also applied to other blocks. When merging is applied, each of the encoding device 100 and the decoding device 200 can use the motion information of neighboring reconstructed blocks and / or the motion information of the col block to generate a list of merging candidates. The motion information may include at least one of the following: 1) motion vector, 2) index of the reference image, and 3) prediction direction. The prediction direction can be unidirectional or bidirectional.

[0236] Here, merging can be applied on either the CU or PU basis. When merging is performed on either the CU or PU basis, the encoding device 100 can send predefined information to the decoding device 200 via a bitstream. The bitstream can include the predefined information. The predefined information can include: 1) information about whether merging is performed on each block partition, and 2) information about the neighboring blocks among the multiple adjacent blocks of the current block that will be used for merging. For example, the neighboring blocks of the current block can include the left neighboring block of the current block, the upper neighboring block of the current block, the time-adjacent block of the current block, etc.

[0237] The merge candidate list can represent a list storing multiple pieces of motion information. Furthermore, the merge candidate list can be generated before performing the merge. The motion information stored in the merge candidate list can be 1) motion information of neighboring blocks adjacent to the current block and 2) motion information of corresponding blocks in the reference image. Additionally, the motion information stored in the merge candidate list can be new motion information generated by combining multiple pieces of motion information previously existing in the merge candidate list.

[0238] A skip mode can be a mode that applies information about neighboring blocks to the current block without alteration. The skip mode can be one of several modes used for inter-frame prediction. When a skip mode is used, the encoding device 100 can send only information about blocks to the decoding device 200 via the bitstream, wherein the motion information of those blocks will be used as the motion information for the current block. The encoding device 100 may not send other information to the decoding device 200. For example, the other information may be syntax information. Syntax information may include motion vector difference (MVD) information.

[0239] Figure 9 A second prediction of the partitioning of the current block according to an embodiment is shown.

[0240] Figure 9 The current block shown can be a PU.

[0241] When performing optimal intra-frame prediction on the current block, prediction is performed on the entire current block in only one direction, which may result in large prediction errors on a portion of the current block.

[0242] The scheme used to partition the current block into non-square shapes can be applied to parts where large prediction errors occur.

[0243] In other words, if a second prediction in a different prediction mode is performed on a portion with a large prediction error after the first prediction has been performed on the square prediction unit, the prediction error in the current block can be reduced.

[0244] For example, both the first prediction mode and the second prediction mode can be intra-frame prediction modes.

[0245] exist Figure 9 In the diagram, the current block can have a square shape. The prediction direction of the first prediction performed on the entire current block is shown. For example, the prediction mode of the first prediction could be a vertical mode with a mode value of 26.

[0246] The current block can be divided into a first part and a second part. Each of the first and second parts can have a non-square shape.

[0247] exist Figure 9 The diagram illustrates the prediction direction for the second prediction of the first part and the prediction direction for the second prediction of the second part. For example, the prediction mode for the second prediction of the first part could be a horizontal mode with a mode value of 10. The prediction mode for the second prediction of the second part could be a lowest diagonal mode with a mode value of 34.

[0248] The shape of the portion with a large prediction error can correspond to the direction of the best prediction mode used for the current block. In the following embodiments, methods and apparatus for improving prediction performance and reducing prediction error by effectively utilizing the tendency of the shape of the portion with a large prediction error to correspond to the direction of the best prediction mode used for the current block can be described.

[0249] Figure 10 This is a configuration diagram of an encoding device according to an embodiment.

[0250] The encoding device 1000 can be a general-purpose computer system that performs encoding.

[0251] like Figure 10As shown, the encoding device 1000 may include at least one processor 1010, a memory 1030, a user interface (UI) input device 1050, a UI output device 1060, and a storage device 1040 that communicate with each other via a bus 1090. The encoding device 1000 may also include a communication unit 1020 connected to a network 1099. The processor 1010 may be a central processing unit (CPU) or a semiconductor device for executing processing instructions stored in the memory 1030 or the storage device 1040. Each of the memory 1010 and the storage device 1040 may be any of a variety of volatile or non-volatile storage media. For example, the memory may include at least one of a read-only memory (ROM) 1031 and a random access memory (RAM) 1032.

[0252] The processor 1010 may include an inter-frame prediction unit 110, an intra-frame prediction unit 120, a switcher 115, a subtractor 125, a transform unit 130, a quantization unit 140, an entropy coding unit 150, an inverse quantization unit 160, an inverse transform unit 170, an adder 175, a filtering unit 180, and a reference frame buffer 190 of the encoding device 100.

[0253] The storage medium may store at least one module for the operation of the encoding device 1000. The memory 1030 may be configured to store at least one module and allow the module to be executed by at least one processor 1010.

[0254] According to an embodiment, at least some of the inter-frame prediction unit 110, intra-frame prediction unit 120, switcher 115, subtractor 125, transform unit 130, quantization unit 140, entropy coding unit 150, inverse quantization unit 160, inverse transform unit 170, adder 175, filtering unit 180, and reference frame buffer 190 of the encoding device 1000 may be program modules and are capable of communicating with external devices or systems. These program modules may be included in the encoding device 1000 in the form of an operating system, application module, and other program modules.

[0255] The program modules can be physically stored in various types of known storage devices. Furthermore, at least some of the program modules can also be stored in a remote storage device capable of communicating with the encoding device 1000.

[0256] Program modules may include, but are not limited to, routines, subroutines, programs, objects, components, and data structures for performing functions or operations according to an embodiment or for implementing abstract data types according to an embodiment.

[0257] The program module can be implemented using instructions or code executed by processor 1010.

[0258] The communication unit 1020 can perform functions related to communication of data or information with the encoding device 1000.

[0259] Figure 11 This is a flowchart of the encoding method according to an embodiment.

[0260] The encoding method according to the embodiment can be executed by encoding device 100 or encoding device 1000.

[0261] In step 1110, the prediction unit may generate the residual signal of the current block based on the current block, the first prediction, and the second prediction.

[0262] The prediction unit may include an inter-frame prediction unit 110, an intra-frame prediction unit 120, and a subtractor 125.

[0263] For example, a residual signal can represent a residual block.

[0264] For example, a residual block can be the difference between the current block and the predicted block. Alternatively, a residual block can be generated based on the difference between the current block and the predicted block.

[0265] For example, a prediction block can be the sum of a first prediction block and a second prediction block. A residual block can be obtained by subtracting the sum of the first and second prediction blocks from the current block.

[0266] The first prediction block can be a block generated via a first prediction. The second prediction block can be a block generated via a second prediction. When the second prediction is not used, the second prediction block can be an empty block. An empty block can be a block where all pixels have a value of 0.

[0267] The prediction unit can generate residual blocks for the current block by performing inter-frame prediction or intra-frame prediction.

[0268] In an embodiment, the prediction unit may generate a second prediction block based on the current block and / or the first prediction block.

[0269] In an embodiment, the prediction unit can generate a residual signal for the current block by performing a first prediction on the current block and a second prediction on a first prediction error that is a result of the first prediction. Here, the first prediction error can be the difference between the current block and the first prediction signal. The residual signal can be the difference between the first prediction error and the second prediction signal.

[0270] In this embodiment, the first prediction and the second prediction can be different types of predictions.

[0271] In this embodiment, the first prediction and the second prediction can be the same type of prediction. For example, both the first prediction and the second prediction can be intra-frame predictions. When the first prediction is an intra-frame prediction, the second prediction can also be set to an intra-frame prediction.

[0272] In this embodiment, the prediction directions of the first prediction and the second prediction can be the same as each other. The prediction direction of the second prediction can be set to the same prediction direction as the first prediction.

[0273] For example, in an embodiment, both the first prediction and the second prediction can be inter-frame predictions.

[0274] In an embodiment, the block that is the target of the first prediction and the block that is the target of the second prediction may be different from each other. Here, the content represented by the expression "block that is the target of prediction" may be different. Here, the expression "block that is the target of prediction" may refer to the block that will generate the prediction block via "prediction".

[0275] For example, the type of block targeted in the first prediction and the type of block targeted in the second prediction can be different from each other. The block type can include original block, luma block, chroma block, depth block, residual block, etc.

[0276] For example, the block that is the target of the first prediction can be the current block or the original block.

[0277] For example, the first prediction error generated by the first prediction can represent a first prediction error block. The first prediction error can be a first residual signal. Furthermore, the first residual signal can represent a first residual block. The block targeted by the second prediction can be either the first prediction error block or the first residual block. In this case, a first residual signal can be generated via the first prediction, and a second prediction error can be generated via the second prediction. The second prediction error can be a second residual signal. The second residual signal can represent a second residual block.

[0278] Reference blocks can be used for prediction. In an embodiment, a first reference block used for a first prediction and a second reference block used for a second prediction may be different from each other. The reference blocks may differ in type and / or location.

[0279] In this embodiment, the positions of the first reference block used for the first prediction and the second reference block used for the second prediction may be different from each other. Here, the position of the first reference block may be relative to the position of the block that serves as the target of the first prediction. The position of the second reference block may be relative to the position of the block that serves as the target of the second prediction.

[0280] In an embodiment, a first reference block for a first prediction may include multiple reference blocks. A second reference block for a second prediction may also include multiple reference blocks. At least some of the multiple first reference blocks for the first prediction and the multiple second reference blocks for the second prediction may be different from each other.

[0281] The positions of at least some of the plurality of first reference blocks used for the first prediction and the plurality of second reference blocks used for the second prediction may be included only in the positions of the plurality of first reference blocks and the positions of the plurality of second reference blocks.

[0282] In an embodiment, the type of the first reference block used for the first prediction and the type of the second reference block used for the second prediction may be different from each other. The block type may include a reconstruction block, a reconstruction luma block, a reconstruction chroma block, a reconstruction depth block, a reconstruction first residual block, and a reconstruction second residual block.

[0283] For example, the first reference block used for the first prediction can be a reconstructed block. The second reference block used for the second prediction can be a reconstructed first residual block. The first reference sample used for the first prediction can be a pixel in the reconstructed block. The second reference sample used for the second prediction can be a pixel in the reconstructed first residual block.

[0284] For example, the first reference block used for the first prediction can be a neighboring reconstructed block adjacent to the current block. The second reference block used for the second prediction can be a reconstructed first residual block adjacent to the current block.

[0285] The neighboring reconstruction residual block adjacent to the current block can be obtained by adding the second predicted block of the neighboring reconstruction block to the reconstruction residual block used for the neighboring reconstruction block. Optionally, the neighboring reconstruction residual block adjacent to the current block can be the difference between the neighboring reconstruction block and the first predicted block of the neighboring reconstruction block.

[0286] For example, when the target of the first prediction is the current block, the neighboring reconstructed blocks of the current block can be used as reference blocks to perform the prediction of the current block. When the target of the second prediction is the first residual block, the neighboring reconstructed residual blocks of the current block or the first residual block can be used as the second reference block to perform the prediction of the first residual block.

[0287] In an embodiment, the first predicted region and the second predicted region may be different from each other. Here, "predicted region" may refer to the region in the block that serves as the target of the prediction where the predicted value is generated. Alternatively, "predicted region" may refer to the region in the prediction block corresponding to the block that serves as the target of the prediction where the predicted value is assigned via the prediction.

[0288] In an embodiment, the second prediction may be a prediction for a portion of the current block. The first prediction may be a prediction for the entire current block. Alternatively, the first prediction may be a prediction for the remaining portion of the current block excluding the portion predicted via the second prediction.

[0289] For example, the second predicted region can be a portion of the current block. The first predicted region can be the entire current block. Alternatively, the first predicted region can be the remaining portion of the current block excluding the second predicted portion.

[0290] For example, the second predicted region can be a non-square portion of the current block. The first predicted region can be the entire current block. Alternatively, the first predicted region can be the remaining non-square regions in the current block excluding the second predicted non-square region.

[0291] For example, the predicted value determined by the first prediction may be assigned only to pixels within the region of the first prediction in the block that is the target of the first prediction. The predicted value determined by the first prediction may not be assigned to pixels within the region of the first prediction in the block that is the target of the first prediction. Optionally, a predefined value may be assigned to pixels within the region of the first prediction in the block that is the target of the first prediction. This predefined value may be 0.

[0292] For example, the predicted value determined by the second prediction may be assigned only to pixels within the region of the second prediction in the block that is the target of the second prediction. The predicted value determined by the second prediction may not be assigned to pixels within the region of the second prediction in the block that is the target of the second prediction. Optionally, a predefined value may be assigned to pixels within the region of the second prediction in the block that is the target of the second prediction. This predefined value may be, for example, 0.

[0293] In an embodiment, the region of the first prediction can be determined based on the type of the first prediction. For example, the region of the first prediction can be determined based on whether the first prediction is an inter-frame prediction. Optionally, the region of the first prediction can be determined based on whether the first prediction is an intra-frame prediction. Optionally, the region of the first prediction can be determined based on the prediction direction of the first prediction.

[0294] In an embodiment, the region of the second prediction can be determined based on the type of the second prediction. For example, the region of the second prediction can be determined based on whether the second prediction is an inter-frame prediction. Optionally, the region of the second prediction can be determined based on whether the second prediction is an intra-frame prediction. Optionally, the region of the second prediction can be determined based on the prediction direction of the second prediction.

[0295] In step 1120, the encoding unit may perform encoding on the residual signal. The encoding unit may generate information about the encoded residual signal by performing encoding on the residual signal.

[0296] The encoding unit may include a transformation unit 130, a quantization unit 140, and an entropy encoding unit 150.

[0297] Step 1120 may include steps 1121, 1122 and 1123.

[0298] In step 1121, the transformation unit 130 can generate transformation coefficients for the residual signal.

[0299] In step 1122, quantization unit 1140 can generate quantized transformation coefficients by performing quantization on the transformation coefficients.

[0300] In step 1123, the entropy coding unit 150 can generate information about the coded residual signal by performing entropy coding on the quantized transform coefficients.

[0301] In step 1130, the entropy coding unit 150 can generate a bit stream of information about the encoded residual signal.

[0302] The bitstream may include information about the encoded residual signal, and may also include prediction-related information.

[0303] Entropy coding unit 150 can include prediction-related information in the bitstream. This prediction-related information can be entropy-coded.

[0304] For example, prediction-related information may include prediction scheme information that indicates the scheme used to encode the current block.

[0305] For example, prediction scheme information can indicate which of intra-frame prediction and inter-frame prediction was used to encode the current block. Optionally, prediction scheme information can indicate whether the current block has already been encoded via intra-frame prediction. Optionally, prediction scheme information can indicate whether the current block has already been encoded via inter-frame prediction.

[0306] For example, prediction-related information may include intra-prediction mode information that indicates the prediction mode of intra-prediction.

[0307] For example, the prediction-related information may include second prediction usage information, which indicates whether the second prediction will be used to encode the current block.

[0308] For example, the prediction-related information may include first prediction pattern information indicating the prediction pattern of the first prediction and second prediction pattern information indicating the prediction pattern of the second prediction.

[0309] For example, the prediction-related information may include selected portion information indicating the selected portion to which the second prediction will be applied, wherein the selected portion is one of two portions generated by partitioning the current block.

[0310] For example, prediction-related information may include current block type information indicating the type of the current block, first prediction block type information indicating the type of the first predicted block, second prediction block type information indicating the type of the second predicted block, first reference block type information indicating the type of the first reference block, second reference block type information indicating the type of the second reference block, first reference sample type information indicating the type of the first reference sample, and second reference sample type information indicating the type of the second reference sample.

[0311] For example, prediction-related information may include first prediction region information indicating the region of the first prediction and / or second prediction region information indicating the region of the second prediction.

[0312] The prediction-related information may include information used for the predictions described in the embodiments. The entropy coding unit 150 may include multiple pieces of prediction-related information in the bitstream according to the order described in the embodiments or the order generated in the embodiments.

[0313] In step 1140, the communication unit 1020 may send a bit stream to the decoding device 200 or the decoding device 1500.

[0314] In step 1150, the reconstruction residual signal generation unit can generate a reconstruction residual signal based on information about the residual signal.

[0315] The reconstructed residual signal generation unit may include an inverse quantization unit 160 and an inverse transform unit 170.

[0316] Step 1150 may include steps 1151 and 1152.

[0317] In step 1151, the dequantization unit 160 can generate reconstructed transform coefficients by performing dequantization on the quantized transform coefficients.

[0318] In step 1152, the transformation unit 130 can generate the reconstruction residual signal by performing an inverse transformation on the reconstruction transformation coefficients.

[0319] In step 1160, the reconstruction block generation unit can generate reconstruction blocks based on the reconstruction residual signal, the first prediction, and the second prediction.

[0320] The rebuild block generation unit may include adder 175.

[0321] In the example, the sum of the reconstructed residual signal and the predicted signal can represent the reconstructed block. Alternatively, the reconstructed block can be generated based on the sum of the reconstructed residual signal and the predicted signal.

[0322] In the example, the reconstructed block can be the sum of the reconstructed residual block and the predicted block. The predicted block can be the sum of the first predicted block and the second predicted block. The second prediction signal can be a signal generated by performing a second prediction on a portion of the current block. The first prediction signal can be a signal generated by performing a first prediction on the current block or a signal generated by performing a first prediction on the remainder of the current block excluding the portion with the second prediction.

[0323] Alternatively, the reconstruction block can be generated based on the sum of the reconstruction residual block and the prediction block.

[0324] For example, a reconstruction block can be generated based on a reconstructed residual signal, a first prediction block, and a second prediction block. The reconstruction block generation unit can generate a reconstructed first residual signal by adding the reconstructed residual signal to the second prediction signal. The reconstruction block generation unit can generate a reconstruction block by adding the reconstructed first residual signal to the first prediction signal.

[0325] Optionally, the reconstruction block generation unit can generate a reconstructed first residual block by adding the reconstructed residual block to the second prediction block. The reconstruction block generation unit can generate a reconstructed block by adding the reconstructed first residual block to the first prediction block.

[0326] The reconstructed block and the first residual signal of the reconstruction can be used as a reference block for encoding other blocks.

[0327] Figure 12 This illustrates the partitioning operation of the current block according to an embodiment.

[0328] exist Figure 12 The image shows the current block with a size of N×N.

[0329] The current block can be divided into two parts. These two parts can be determined based on a straight line passing through the center of the current block along the predicted direction of the first prediction. Figure 12 In this case, the prediction direction of the first prediction can be the prediction direction of the prediction mode with a mode value of 3.

[0330] The first prediction may be a prediction using the first best prediction mode obtained by performing intra-frame prediction on the current block.

[0331] The current block can be partitioned into two parts (regions) using the prediction mode of the first prediction, such that the two parts have the same number of pixels.

[0332] The reason for partitioning the current block along the prediction direction of the first prediction is that when the prediction of the current block is performed using intra-frame prediction, there may be a tendency for large prediction errors to occur in the part specified by the prediction direction.

[0333] exist Figure 12 In this diagram, S1 and S2 are used to indicate the two parts, respectively. Among the points with x-coordinate value i, the point with y-coordinate value j2 can be included in S1. Among the points with x-coordinate value i, the point with y-coordinate value j1 can be included in S2.

[0334] The predicted direction of the prediction model can be calculated using the following equations 2, 3, 4, 5, and 6, as well as Table 1 below.

[0335] Since prediction modes with a mode value of 0 or 1 are non-directional prediction modes, they can be excluded from the partition.

[0336] [Equation 2]

[0337]

[0338] "predMode" can represent the numerical value of the prediction mode. "Vertical" indicates that the prediction mode is vertical. "Horizontal" indicates that the prediction mode is horizontal.

[0339] [Equation 3]

[0340] absAngle = |predAngle|

[0341] [Equation 4]

[0342]

[0343] [Equation 5]

[0344]

[0345] [Equation 6]

[0346] Tan = TanSign × TanValue

[0347] The value of absAngle can be defined using the following Table 1.

[0348] Table 1

[0349] absAngle 0 1 2 3 4 5 6 7 8 value 0 2 5 9 13 17 21 26 32

[0350] When the value of Tan of the prediction mode is calculated, the pixel located at coordinates (i, j) in the current block can be included in at least one of part S0 and part S1 by means of the following equation 7:

[0351] [Equation 7]

[0352]

[0353] When the prediction mode `predMode` has a mode value of 3, 18, or 34, the value of `Tan` can be -1 or +1. In this case, pixels on the line that partitions the current block can be included in each of the two parts, S0 and S1.

[0354] In other words, when the current block is divided into two parts along a straight line passing through the center of the current block, the pixels in the current block that lie on that straight line can be included in each of the two parts.

[0355] Figure 13 This is a flowchart of a residual block generation method according to an embodiment.

[0356] Reference above Figure 11 The described step 1110 may include steps 1310, 1320, 1330, 1335, 1340, 1345, 1350, 1355, 1360, 1370 and 1380.

[0357] In this embodiment, the block targeted by the first prediction can be the current block. The first prediction can be an intra-frame prediction.

[0358] In step 1310, the prediction unit can generate a first prediction signal by performing a first prediction on the current block.

[0359] The first prediction signal can represent the first prediction block.

[0360] The prediction unit can perform a first prediction in each of the multiple prediction modes in order to find the best prediction mode for the first prediction to encode the current block. That is, step 1310 can be performed in each of the multiple prediction modes.

[0361] Multiple prediction models can correspond to different prediction directions.

[0362] The prediction unit can generate a first prediction error based on the current block and the first prediction signal.

[0363] The first prediction error can be represented by the first residual block.

[0364] The first residual block can represent the difference (residual) between the current block and the first prediction block. That is, the prediction unit can generate the first prediction error by subtracting the first prediction block represented by the first prediction signal from the current block.

[0365] The forecasting unit can use the first forecast error to calculate the rate distortion cost for the first forecast.

[0366] The prediction unit may store the calculated first prediction error and / or the calculated rate distortion cost for the first prediction in memory 1030, storage 1040 or buffer.

[0367] The prediction unit can perform a first prediction in each of multiple prediction modes to find the optimal prediction mode for encoding the current block. The prediction unit can use the first prediction errors under multiple prediction modes to calculate multiple rate-distortion costs for the multiple prediction modes. The prediction unit can store the minimum rate-distortion cost among the calculated multiple rate-distortion costs as the minimum first rate-distortion cost RDcost_1. For example, RDcost_1 could be the rate-distortion cost of the mode with the minimum rate-distortion cost in the main intra-frame prediction.

[0368] The prediction unit can store RDcost_1 and the first prediction error corresponding to RDcost_1.

[0369] In step 1320, the prediction unit may determine a first prediction mode for the first prediction. The first prediction mode may indicate the prediction direction of the first prediction.

[0370] The prediction department can determine the prediction mode with the lowest rate-distortion cost among multiple prediction modes as the first prediction mode. In the following text, the prediction mode of the first prediction can be the prediction mode with the lowest rate-distortion cost among multiple prediction modes. The prediction direction of the first prediction can be the prediction direction of the prediction mode of the first prediction.

[0371] In step 1330, it can be determined whether to perform a second prediction on the current block.

[0372] In an embodiment, the determination in step 1330 may not be a final decision regarding whether to use the second prediction to encode the current block. In step 1130, the prediction unit may determine whether the second prediction can be performed on the current block.

[0373] For example, if it is determined that the second prediction should not be performed, the current block can be encoded without using the second prediction. If it is determined that the second prediction should be performed, a second prediction signal and a second prediction error can be generated via the second prediction, but whether to use the second prediction to encode the current block can be determined in step 1360 by comparing the rate distortion cost.

[0374] If it is determined that the second prediction will not be performed, the residual block for the current block can be a block represented by the first prediction error. In other words, the first prediction error can be provided as a residual block.

[0375] If it is determined that a second prediction will be performed, the residual block for the current block can be a block represented by the second prediction error, as will be described later. In other words, the second prediction error can be provided as a residual block.

[0376] The forecasting unit can determine whether to perform a second forecast based on predefined conditions.

[0377] In an embodiment, the prediction unit may determine whether to perform a second prediction based on the prediction pattern of the first prediction.

[0378] For example, when the forecasting mode of the first forecast is a non-directional mode, the forecasting unit may not execute the second forecast. Alternatively, when the forecasting mode of the first forecast is a directional mode, the forecasting unit may execute the second forecast.

[0379] When the prediction mode of the first prediction is a non-directional mode among multiple prediction modes, since the direction of the prediction mode does not exist, the current block does not need to be partitioned along the prediction direction.

[0380] If it is determined that the second prediction will be performed, then step 1335 can be executed.

[0381] If it is determined that the second prediction is not performed, then step 1370 or step 1120 can be performed. When step 1120 is performed, the residual signal in step 1120 can be the first prediction error. That is, when only the first prediction is performed and the second prediction is not performed, the first prediction error generated by the first prediction can be used as the residual signal for the current block.

[0382] In step 1335, the prediction unit may partition the current block based on a first prediction mode. The prediction unit may generate two parts by partitioning the current block based on the first prediction mode.

[0383] In one embodiment, the prediction unit can generate two non-square portions by partitioning the current block into two parts by drawing a straight line through the center of the current block along the prediction direction of the first prediction. Hereinafter, the two non-square portions may be designated as the "first portion" and the "second portion".

[0384] When the current block is partitioned, a second prediction can be performed on each part resulting from the partitioning operation.

[0385] In step 1340, the prediction unit can generate a second prediction signal for the first part by performing a second prediction on the first part.

[0386] The second prediction signal for the first part can represent the second prediction block for the first part.

[0387] The second prediction for the first part can be an intra-frame prediction.

[0388] The prediction unit can perform a second prediction in each of the multiple prediction modes in order to find the optimal prediction mode for encoding the first part. In other words, step 1340 can be performed in each of the multiple prediction modes.

[0389] Multiple prediction models can correspond to different prediction directions.

[0390] The prediction unit can generate a second prediction error for the first part based on the first part and the second prediction signal.

[0391] The second prediction error for the first part can be represented as the second residual block for the first part.

[0392] The second residual block for the first part can represent the difference (residual) between the first part and the second prediction block. That is, the prediction unit can generate a second prediction error for the first part by subtracting the second prediction block for the first part from the first part, wherein the second prediction block for the first part is represented by the second prediction signal for the first part.

[0393] The prediction unit can use the second prediction error for the first part to calculate the rate distortion cost of the second prediction for the first part.

[0394] The prediction unit may store the calculated second prediction error for the first part and / or the calculated rate distortion cost of the second prediction for the first part in memory 1030, memory 1040 or buffer.

[0395] The prediction unit can perform a second prediction in each of multiple prediction modes to find the optimal prediction mode for the second prediction of the first part to encode the first part. The prediction unit can use multiple second prediction errors of multiple prediction modes to calculate multiple rate-distortion costs for multiple prediction modes. The prediction unit can store the minimum rate-distortion cost among the calculated multiple rate-distortion costs as the minimum second rate-distortion cost RDcost_2 of the first part. For example, RDcost_2 can be the rate-distortion cost of the mode with the minimum rate-distortion cost in the minor intra-frame prediction of the first part.

[0396] The prediction unit can store the RDcost_2 of the first part and the second prediction error of the first part corresponding to the RDcost_2 of the first part.

[0397] In step 1345, the prediction unit may determine a second prediction mode for the second prediction of the first part. The second prediction mode for the first part may indicate the prediction direction for the second prediction of the first part.

[0398] The prediction department can determine the prediction mode with the lowest rate-distortion cost among multiple prediction modes as the second prediction mode for the first part. In the following text, the prediction mode for the second prediction of the first part can be the prediction mode with the lowest rate-distortion cost among multiple prediction modes. The prediction direction for the second prediction of the first part can be the prediction direction of the prediction mode for the second prediction of the first part.

[0399] In step 1350, the prediction unit can generate a second prediction signal for the second part by performing a second prediction on the second part.

[0400] The second prediction signal for the second part can represent the second prediction block for the second part.

[0401] The second prediction for the second part can be an intra-frame prediction.

[0402] The prediction unit can perform a second prediction in each of the multiple prediction modes to find the optimal prediction mode for encoding the second part. In other words, step 1340 can be performed in each of the multiple prediction modes.

[0403] Multiple prediction models can correspond to different prediction directions.

[0404] The prediction unit can generate a second prediction error for the second part based on the second part and the second prediction signal.

[0405] The second prediction error for the second part can be represented as the second residual block for the second part.

[0406] The second residual block for the second part can represent the difference (residual) between the second part and the second prediction block. That is, the prediction unit can generate a second prediction error for the second part by subtracting the second prediction block for the second part from the second part, wherein the second prediction block for the second part is represented by the second prediction signal for the second part.

[0407] The prediction unit can use the second prediction error for the second part to calculate the rate distortion cost of the second prediction for the second part.

[0408] The prediction unit may store the calculated second prediction error for the second part and / or the calculated rate distortion cost for the second prediction for the second part in memory 1030, memory 1040 or buffer.

[0409] The prediction unit can perform a second prediction in each of multiple prediction modes to find the optimal prediction mode for encoding the second part. The prediction unit can use multiple second prediction errors from multiple prediction modes to calculate multiple rate-distortion costs for multiple prediction modes. The prediction unit can store the minimum rate-distortion cost among the calculated multiple rate-distortion costs as the minimum second rate-distortion cost RDcost_2 for the second part. For example, RDcost_2 could be the rate-distortion cost of the mode with the minimum rate-distortion cost in the minor intra-frame prediction of the second part.

[0410] The prediction unit can store the RDcost_2 of the second part and the second prediction error of the second part corresponding to the RDcost_2 of the second part.

[0411] In step 1355, the prediction unit may determine a second prediction mode for the second prediction of the second part. The second prediction mode for the second part may indicate the prediction direction for the second prediction of the second part.

[0412] The prediction department can determine the prediction mode with the lowest rate-distortion cost among multiple prediction modes as the second prediction mode for the second part. In the following text, the prediction mode for the second prediction in the second part can be the prediction mode with the lowest rate-distortion cost among multiple prediction modes. The prediction direction for the second prediction in the second part can be the prediction direction of the prediction mode for the second prediction in the second part.

[0413] In step 1360, the prediction unit may determine whether to use the second prediction to encode the current block.

[0414] The prediction unit can determine whether to use a second prediction to encode the current block based on predefined conditions.

[0415] For example, if the rate distortion cost is further reduced by using the second prediction, the prediction unit may determine to use the second prediction. If the rate distortion cost is not further reduced even if the second prediction is used, the prediction unit may determine not to use the second prediction.

[0416] For example, if the minimum rate distortion cost calculated when using the second prediction is less than the minimum rate distortion cost calculated when not using the second prediction, the prediction unit may determine to use the second prediction.

[0417] In an embodiment, the prediction unit may determine whether to use the second prediction based on a comparison between the rate-distortion cost of the prediction mode of the first prediction, the rate-distortion cost of the prediction mode of the second prediction for the first part, and the rate-distortion cost of the prediction mode of the second prediction for the second part.

[0418] In an embodiment, the prediction unit may select a portion to be encoded from a first portion and a second portion. The prediction unit may select a portion to be encoded from the first portion and the second portion based on a comparison between the rate-distortion cost of the second prediction mode for the first portion and the rate-distortion cost of the second prediction mode for the second portion.

[0419] For example, the prediction unit can select the part with the higher rate-distortion cost from the first part and the second part. When the rate-distortion cost of the second prediction mode for the first part is higher than the rate-distortion cost of the second prediction mode for the second part, the prediction unit can select the first part. When the rate-distortion cost of the second prediction mode for the first part is lower than the rate-distortion cost of the second prediction mode for the second part, the prediction unit can select the second part.

[0420] For example, when the rate-distortion cost of the second prediction mode for the first part is the same as the rate-distortion cost of the second prediction mode for the second part, if the second prediction mode for the first part is different from the first prediction mode, the prediction unit can select the first part; if the second prediction mode for the second part is different from the first prediction mode, the prediction unit can select the second part. In other words, the prediction unit can use the rate-distortion cost as the primary criterion and the prediction mode as the secondary criterion to select a part between the first part and the second part.

[0421] In the following text, the portion of the first part and the second part selected by the prediction unit is referred to as the "selected portion". Furthermore, "second prediction" refers to the second prediction for the first part and the second prediction for the second part specifically for the selected portion. The portion of the first part and the second part not selected by the prediction unit is referred to as the remaining portion.

[0422] In an embodiment, the prediction unit may determine whether to use the second prediction based on the second prediction mode of the selected portion and the first prediction mode of the current block.

[0423] For example, when the second prediction mode of the selected portion is the same as the first prediction mode of the current block, the result obtained by using the second prediction may be the same as the result obtained by not using the second prediction. Therefore, in this case, the effect obtained by using the second prediction may not exist. If the second prediction mode of the selected portion is the same as the first prediction mode of the current block, the prediction unit can determine that the second prediction is not used for the selected portion.

[0424] In an embodiment, if the rate-distortion cost of the prediction for the current block is reduced by the second prediction for the selected portion, the prediction unit may determine to use the second prediction for the selected portion. If the rate-distortion cost of the prediction for the current block is not reduced by the second prediction for the selected portion, the prediction unit may determine not to use the second prediction for the selected portion.

[0425] In the example, when the rate-distortion cost of the second prediction for the selected portion is RDcost_SS, the rate-distortion cost of the first prediction for the remaining portion is RDcost_SNS, and the rate-distortion cost of the first prediction for the current block is RDcost, if the sum of RDcost_SS and RDcost_SNS is less than RDcost, the prediction unit can determine to use the second prediction for the selected portion. Furthermore, if the sum of RDcost_SS and RDcost_SNS is equal to or greater than RDcost, the prediction unit can determine not to use the second prediction for the selected portion.

[0426] In the example, if the rate-distortion cost of the second prediction for the selected portion is less than the rate-distortion cost of the first prediction for the selected portion, it is determined that the second prediction will not be used for the selected portion. Furthermore, if the rate-distortion cost of the second prediction for the selected portion is equal to or greater than the rate-distortion cost of the first prediction for the selected portion, it is determined that the second prediction will not be used for the selected portion.

[0427] If it is determined that the second prediction will not be used to encode the current block, step 1370 can be executed.

[0428] If it is determined that the second prediction will be used to encode the current block, then step 1380 can be executed.

[0429] In step 1370, the prediction unit may execute a setting that indicates the second prediction will not be used.

[0430] The prediction unit can set the value of the second prediction usage information so that the second prediction usage information indicates that the second prediction will not be used. For example, the second prediction usage information can be an Additional Minor Intra-Frame Prediction (ASIP) flag.

[0431] For example, when the value of the second prediction usage information is 0, the second prediction usage information can indicate that the second prediction will not be used.

[0432] The forecasting department can set the first forecasting mode information for the first forecasting mode.

[0433] When step 1370 is executed, the residual signal from step 1120 can be the residual signal of the current block. The residual signal of the current block can be the first prediction error. In other words, when a second prediction is not performed on the current block, the first prediction error generated based on the first prediction can be used as the residual signal of the current block.

[0434] In step 1380, the prediction unit may execute settings indicating that a second prediction will be used.

[0435] In an embodiment, the prediction unit may set the value of the second prediction usage information so that the second prediction usage information indicates that the second prediction will be used.

[0436] In an embodiment, when the value of the second prediction usage information is, for example, 1, the second prediction usage information may indicate that the second prediction will be used.

[0437] The prediction unit can set the value of the selected portion information so that the selected portion indicates the selected portion to which the second prediction is applied. For example, the selected portion information can be a Selected Portion Flag (SSF).

[0438] For example, the prediction unit can set the value of selected portion information so that the selected portion information indicates one of two portions generated by partitioning the current block. When the selected portion is the above reference... Figure 11 When describing part S1, the prediction unit can set the value of the selected part information to 0. When the selected part is the above reference... Figure 11 When describing part S2, the prediction unit can set the value of the selected part of the information to 1.

[0439] The forecasting unit can set the first forecasting mode information for the first forecasting mode. The forecasting unit can set the second forecasting mode information for the second forecasting mode.

[0440] When step 1370 is executed, the residual signal in step 1120 can be the residual signal of the current block. The residual signal of the current block can be a signal corresponding to the difference between the current block and the predicted block. The predicted block can be the sum of the predicted block generated via the second prediction for the selected portion and the predicted block generated via the first prediction for the remaining portion.

[0441] In other words, when a second prediction is performed for the current block, the prediction error generated based on the first and second predictions can be used as a residual signal.

[0442] Optionally, when step 1370 is performed, the residual signal in step 1120 can be a residual signal for a selected portion and a residual signal for the remaining portion. The residual signal for the selected portion can be a signal corresponding to the difference between the selected portion and a second prediction block generated via a second prediction for the selected portion. The residual signal for the remaining portion can be a signal corresponding to the difference between the remaining portion and a third prediction block generated via a first prediction for the remaining portion.

[0443] Figure 14 This is a flowchart of a reconstruction block generation method according to an embodiment.

[0444] Reference above Figure 11 The described step 1160 may include the following steps 1410, 1420 and 1430.

[0445] In step 1410, the reconstructed block generation unit can determine whether the second prediction has been used to encode the current block.

[0446] If it is determined that the second prediction has been used to encode the current block, step 1420 can be executed.

[0447] If it is determined that the second prediction was not used to encode the current block, step 1430 can be executed.

[0448] In step 1420, the reconstruction block generation unit may add the second prediction signal for the selected portion to the reconstruction residual signal.

[0449] The second prediction signal can be a signal generated via a second prediction for a selected portion.

[0450] In step 1430, the reconstruction block generation unit can generate a reconstruction block by adding the first prediction signal for the current block or the remaining portion to the reconstruction residual signal.

[0451] In an embodiment, when the second prediction is not used, the reconstruction block generation unit can generate a reconstruction block by adding the first prediction signal for the current block to the reconstruction residual signal. The first prediction signal may be a signal generated via the first prediction for the current block.

[0452] In an embodiment, when the second prediction is not used, the reconstruction block generation unit can generate a reconstruction block by adding the first prediction signal for the remaining portion to the reconstruction residual signal. The first prediction signal may be a signal generated via the first prediction for the remaining portion.

[0453] For example, when the second prediction is used, the signal indicating the reconstructed block can be the sum of the reconstructed residual signal, the second prediction signal for the selected portion, and the first prediction signal for the remaining portion. As described above with reference to steps 1420 and 1430, when the second prediction is used, the second prediction signal for the selected portion and the first prediction signal for the remaining portion can be added to the reconstructed residual signal. Alternatively, when the second prediction is used, the reconstructed block can be generated based on the reconstructed residual signal, the second prediction signal for the selected portion, and the first prediction signal for the remaining portion.

[0454] For example, when the second prediction is not used, the signal indicating the reconstructed block can be the sum of the reconstructed residual signal and the first prediction signal for the current block. Alternatively, when the second prediction is not used, the reconstructed block can be generated based on the reconstructed residual signal and the first prediction signal for the current block.

[0455] Figure 15 This is a configuration diagram of a decoding device according to an embodiment.

[0456] The decoding device 1500 can be a general-purpose computer system that performs the decoding.

[0457] like Figure 15 As shown, the decoding device 1500 may include at least one processor 1510, a memory 1530, a user interface (UI) input device 1550, a UI output device 1560, and a storage device 1540, all communicating with each other via a bus 1590. The decoding device 1500 may also include a communication unit 1520 connected to a network 1599. The processor 1510 may be a CPU or a semiconductor device for executing processing instructions stored in the memory 1530 or the storage device 1540. Each of the memory 1530 and the storage device 1540 may be any of a variety of volatile or non-volatile storage media. For example, the memory may include at least one of ROM 1531 and RAM 1532.

[0458] The processor 1510 may include an entropy decoding unit 210, an inverse quantization unit 220, an inverse transform unit 230, an intra-frame prediction unit 240, an inter-frame prediction unit 250, an adder 255, a filtering unit 260, and a reference frame buffer 270 of the decoding device 200.

[0459] The storage medium may store at least one module used for operation of the decoding device 1500. The memory 1530 may be configured to store at least one module and allow the module to be executed by at least one processor 1510.

[0460] According to an embodiment, at least some of the entropy decoding unit 210, inverse quantization unit 220, inverse transform unit 230, intra-frame prediction unit 240, inter-frame prediction unit 250, adder 255, filtering unit 260, and reference frame buffer 270 of the decoding device 1500 may be program modules and are capable of communicating with external devices or systems. Program modules may be included in the decoding device 1500 in the form of operating systems, application modules, and other program modules.

[0461] The program modules can be physically stored in various types of known storage devices. Furthermore, at least some of the program modules can also be stored in a remote storage device capable of communicating with the decoding device 1500.

[0462] Program modules may include, but are not limited to, routines, subroutines, programs, objects, components, and data structures for performing functions or operations according to an embodiment or for implementing abstract data types according to an embodiment.

[0463] Program modules can be implemented using instructions or code executed by processor 1510.

[0464] The communication unit 1520 can perform functions related to communication of data or information with the decoding device 1500.

[0465] Figure 16 This is a flowchart of a decoding method according to an embodiment.

[0466] The decoding method according to the embodiment can be executed by decoding device 200 or decoding device 1500.

[0467] In step 1610, the communication unit 1520 may receive a bit stream from the encoding device 100 or the encoding device 1000.

[0468] The bitstream may include information about the encoded residual signal and may also include prediction-related information.

[0469] Information about the encoded residual signal may include the quantized transform coefficients that have been entropy encoded.

[0470] Information related to prediction can be encoded by entropy.

[0471] In step 1620, the entropy decoding unit 210 can generate quantized transform coefficients by performing entropy decoding on the bitstream. Furthermore, the entropy decoding unit 210 can generate prediction-related information by performing entropy decoding on the bitstream.

[0472] In the example, prediction-related information may include prediction scheme information that indicates the scheme used to encode the current block.

[0473] In the example, the prediction scheme information can indicate which of intra-frame prediction and inter-frame prediction was used to encode the current block. Optionally, the prediction scheme information can indicate whether the current block has already been encoded via intra-frame prediction. Optionally, the prediction scheme information can indicate whether the current block has already been encoded via inter-frame prediction.

[0474] In the example, prediction-related information may include intra-prediction mode information that indicates the prediction mode of intra-prediction.

[0475] In the example, the prediction-related information may include second prediction usage information, which indicates whether the second prediction will be used to encode the current block.

[0476] In the example, the prediction-related information may include first prediction pattern information indicating the prediction pattern of the first prediction and second prediction pattern information indicating the prediction pattern of the second prediction.

[0477] In the example, the prediction-related information may include selected portion information indicating the selected portion to which the second prediction will be applied, wherein the selected portion is one of two portions generated by partitioning the current block.

[0478] In the example, prediction-related information may include current block type information indicating the type of the current block, first prediction block type information indicating the type of the first predicted block, second prediction block type information indicating the type of the second predicted block, first reference block type information indicating the type of the first reference block, second reference block type information indicating the type of the second reference block, first reference sample type information indicating the type of the first reference sample, and second reference sample type information indicating the type of the second reference sample.

[0479] In the example, the prediction-related information may include first prediction region information indicating the region of the first prediction and / or second prediction region information indicating the region of the second prediction.

[0480] Furthermore, the prediction-related information may include information used for the predictions described in the embodiments. Multiple pieces of prediction-related information may be included in the bitstream in the order described in the embodiments or in the order generated in the embodiments. The reconstructed residual signal generation unit may obtain the multiple pieces of prediction-related information from the bitstream in the order described in the embodiments or in the order generated in the embodiments.

[0481] In step 1630, the reconstruction residual signal generation unit can generate a reconstruction residual signal for the current block based on the quantized transform coefficients.

[0482] The reconstructed residual signal generation unit may include an inverse quantization unit 220 and an inverse transform unit 230.

[0483] Step 1630 may include steps 1631 and 1632.

[0484] In step 1631, the dequantization unit 220 can generate reconstructed transform coefficients by performing dequantization on the quantized transform coefficients.

[0485] In step 1632, the inverse transform unit 230 can generate the reconstructed residual signal by performing an inverse transform on the dequantized transform coefficients.

[0486] In steps 1610, 1720 and 1730, a reconstruction residual signal for the current block can be generated.

[0487] In step 1640, the reconstruction block generation unit can generate reconstruction blocks based on the reconstruction residual signal, the second prediction, and the first prediction.

[0488] The reconstruction block generation unit may include an adder 225, an intra-frame prediction unit 240, an inter-frame prediction unit 250, a filter unit 260, and a reference frame buffer 270.

[0489] The reconstructed residual signal can be a reconstructed residual block.

[0490] In the example, the reconstructed block can be the sum of the reconstructed residual block and the predicted block. Alternatively, the reconstructed block can be generated based on the sum of the reconstructed residual signal and the predicted block.

[0491] In the example, the prediction block can be the sum of a first prediction block and a second prediction block. Optionally, the prediction signal can be the sum of a first prediction signal and a second prediction signal. The second prediction signal can be a signal generated via a second prediction for a selected portion of the current block. The first prediction signal can be a signal generated via a first prediction for the current block or the remainder of the current block.

[0492] Alternatively, the reconstruction block can be generated based on the sum of the reconstruction residual block and the prediction block.

[0493] For example, a reconstruction block can be generated based on a reconstructed residual signal, a first prediction block, and a second prediction block. The reconstruction block generation unit can generate a reconstructed first residual signal by adding the reconstructed residual signal to the second prediction signal. The reconstruction block generation unit can generate a reconstruction block by adding the reconstructed first residual signal to the first prediction signal.

[0494] Optionally, the reconstruction block generation unit can generate a reconstructed first residual block by adding the reconstructed residual block to the second prediction block. The reconstruction block generation unit can generate a reconstructed block by adding the reconstructed first residual block to the first prediction block.

[0495] In an embodiment, when the second prediction is not used, the second prediction signal can be an empty signal, or the second prediction block can be an empty block. An empty block can be a block where all pixels have a value of 0.

[0496] The reconstruction block generation unit can generate reconstruction blocks by performing inter-frame prediction or intra-frame prediction.

[0497] In this embodiment, the first prediction and the second prediction can be different types of predictions.

[0498] In this embodiment, the first prediction and the second prediction can be the same type of prediction. For example, both the first prediction and the second prediction can be intra-frame predictions. When the first prediction is an intra-frame prediction, the second prediction can also be set to an intra-frame prediction.

[0499] In this embodiment, the prediction directions of the first prediction and the second prediction can be the same as each other. The prediction direction of the second prediction can be set to the same prediction direction as the first prediction.

[0500] For example, in an embodiment, both the first prediction and the second prediction can be inter-frame predictions.

[0501] In an embodiment, the block that is the target of the first prediction and the block that is the target of the second prediction may be different from each other. Here, the content represented by the expression "block that is the target of prediction" may be different. Here, the expression "block that is the target of prediction" may refer to the block that will generate the prediction block via "prediction".

[0502] For example, the type of block targeted in the first prediction and the type of block targeted in the second prediction can be different from each other. The block type can include original block, luma block, chroma block, depth block, residual block, etc.

[0503] For example, the block that is the first target of prediction could be the reconstruction block.

[0504] Reference blocks can be used for prediction. In an embodiment, a first reference block used for a first prediction and a second reference block used for a second prediction may be different from each other. The reference blocks may differ in type and / or location.

[0505] In this embodiment, the positions of the first reference block used for the first prediction and the second reference block used for the second prediction may be different from each other. Here, the position of the first reference block may be relative to the position of the block that serves as the target of the first prediction. The position of the second reference block may be relative to the position of the block that serves as the target of the second prediction.

[0506] In an embodiment, a first reference block for a first prediction may include multiple reference blocks. A second reference block for a second prediction may also include multiple reference blocks. At least some of the multiple first reference blocks for the first prediction and the multiple second reference blocks for the second prediction may be different from each other.

[0507] The position of at least one of the plurality of first reference blocks used for the first prediction and the plurality of second reference blocks used for the second prediction may be included only in the position of the plurality of first reference blocks and the position of the plurality of second reference blocks.

[0508] In an embodiment, the type of the first reference block used for the first prediction and the type of the second reference block used for the second prediction may be different from each other. The block type may include a reconstruction block, a reconstruction luma block, a reconstruction chroma block, a reconstruction depth block, a reconstruction first residual block, and a reconstruction second residual block.

[0509] For example, the first reference block used for the first prediction can be a reconstructed block. The second reference block used for the second prediction can be a reconstructed first residual block. The first reference sample used for the first prediction can be a pixel in the reconstructed block. The second reference sample used for the second prediction can be a pixel in the reconstructed first residual block.

[0510] For example, the first reference block used for the first prediction can be a neighboring reconstructed block adjacent to the current block. The second reference block used for the second prediction can be a reconstructed first residual block adjacent to the current block.

[0511] The neighboring reconstruction residual block adjacent to the current block can be obtained by adding the second predicted block of the neighboring reconstruction block to the reconstruction residual block used for the neighboring reconstruction block. Optionally, the neighboring reconstruction residual block adjacent to the current block can be the difference between the neighboring reconstruction block and the first predicted block of the neighboring reconstruction block.

[0512] For example, when the target of the first prediction is the current block, the neighboring reconstructed blocks of the current block can be used as reference blocks to perform the prediction of the current block. When the target of the second prediction is the first residual block, the neighboring reconstructed residual blocks of the current block or the first residual block can be used as the second reference block to perform the prediction of the first residual block.

[0513] In an embodiment, the first predicted region and the second predicted region may be different from each other. Here, "predicted region" may refer to the region in the block that serves as the target of the prediction where the predicted value is generated. Alternatively, "predicted region" may refer to the region in the prediction block corresponding to the block that serves as the target of the prediction where the predicted value is assigned via the prediction.

[0514] In an embodiment, the second prediction may be a prediction for a selected portion of the current block. The first prediction may be a prediction for the entire current block. Alternatively, the first prediction may be a prediction for the remaining portion of the current block excluding the selected portion for the second prediction.

[0515] In the example, the second predicted region can be a selected portion of the current block. The first predicted region can be the entire current block. Alternatively, the first predicted region can be the remaining portion of the current block excluding the selected portion of the second predicted region.

[0516] In the example, the second predicted region can be a non-square portion of the current block. The first predicted region can be the entire current block. Alternatively, the first predicted region can be the remaining non-square regions in the current block excluding the second predicted non-square region.

[0517] In the example, the predicted value determined by the first prediction may be assigned only to pixels within the region of the first prediction in the block that is the target of the first prediction. The predicted value determined by the first prediction may not be assigned to pixels within the region of the first prediction in the block that is the target of the first prediction. Optionally, a predefined value may be assigned to pixels within the region of the first prediction in the block that is the target of the first prediction. This predefined value may be, for example, 0.

[0518] In the example, the predicted value determined by the second prediction may be assigned only to pixels within the region of the second prediction in the block that is the target of the second prediction. The predicted value determined by the second prediction may not be assigned to pixels within the region of the second prediction in the block that is the target of the second prediction. Optionally, a predefined value may be assigned to pixels within the region of the second prediction in the block that is the target of the second prediction. This predefined value may be, for example, 0.

[0519] In an embodiment, the region of the first prediction can be determined based on the type of the first prediction. For example, the region of the first prediction can be determined based on whether the first prediction is an inter-frame prediction. Optionally, the region of the first prediction can be determined based on whether the first prediction is an intra-frame prediction. Optionally, the region of the first prediction can be determined based on the prediction direction of the first prediction.

[0520] In an embodiment, the region of the second prediction can be determined based on the type of the second prediction. For example, the region of the second prediction can be determined based on whether the second prediction is an inter-frame prediction. Optionally, the region of the second prediction can be determined based on whether the second prediction is an intra-frame prediction. Optionally, the region of the second prediction can be determined based on the prediction direction of the second prediction.

[0521] Figure 17 This is a flowchart of a reconstruction block generation method according to an embodiment.

[0522] Reference above Figure 16 The described step 1640 may include the following steps 1710, 1710, 1730, 1740 and 1750.

[0523] A second prediction can be selectively performed based on predefined conditions.

[0524] In step 1710, the reconstruction block generation unit may determine whether to use the second prediction to generate reconstruction blocks.

[0525] The reconstruction block generation unit can determine whether to use a second prediction to generate reconstruction blocks based on predetermined conditions.

[0526] In an embodiment, the reconstruction block generation unit may determine whether to use a second prediction based on the prediction pattern of the first prediction.

[0527] For example, the reconstructed block generation unit can obtain first prediction mode information from the bitstream that indicates the prediction mode of the first prediction. When the prediction mode of the first prediction is a non-directional mode, the reconstructed block generation unit may not use the second prediction.

[0528] In an embodiment, after determining whether to use the second prediction using the prediction mode, the position of the current block, and / or the number of neighboring reconstructed blocks, the reconstructed block generation unit may determine whether to use the second prediction based on whether the second prediction has been used to encode the current block.

[0529] For example, the reconstructed block generation unit can obtain second prediction usage information from the bitstream. When the second prediction usage information indicates that the second prediction will be used, the reconstructed block generation unit can use the second prediction. When the second prediction usage information indicates that the second prediction will not be used, the reconstructed block generation unit can choose not to use the second prediction.

[0530] When the second prediction is to be used to encode the current block, step 1720 can be executed.

[0531] When the second prediction will not be used to encode the current block, step 1740 can be executed.

[0532] In step 1720, the reconstruction block generation unit can generate a second prediction signal by performing a second prediction.

[0533] The reconstruction block generation unit can generate a second prediction signal by performing a second prediction on a selected portion.

[0534] The second prediction can correspond to the above reference. Figure 11 and Figure 13 The second prediction described is for a selected portion during the encoding process of the current block. For example, the second prediction could be a prediction for a selected portion of the current block. The second prediction signal may correspond to the above reference. Figure 11 and Figure 13 The second prediction described for the selected portion.

[0535] The second prediction signal can represent the second prediction block.

[0536] Both the first prediction and the second prediction can be intra-frame predictions, and the second prediction mode of the second prediction can be different from the first prediction mode of the first prediction. Optionally, the prediction direction of the second prediction can be different from the second prediction direction.

[0537] The reference block used for the first prediction and the reference block used for the second prediction can be a reconstruction neighbor block adjacent to the current block.

[0538] In an embodiment, the selected portion to be subjected to the second prediction may be one of two portions generated by partitioning the current block.

[0539] In this embodiment, each of the two portions may have a non-square shape. Selected portions may have a non-square shape.

[0540] The selected portion to be subject to the second prediction can be determined based on the prediction direction of the prediction pattern of the first prediction.

[0541] In the embodiments, based on the above references Figure 12 The described scheme allows for partitioning of the current block along a prediction direction. The two portions can be determined based on a straight line passing through the center of the current block along the prediction direction of the first prediction. Each of the two portions can be one of two regions resulting from partitioning the current block along a straight line passing through the center of the current block along the prediction direction of the first prediction.

[0542] In one embodiment, the two portions created by partitioning can be of equal size. Alternatively, the two portions can have the same number of pixels.

[0543] In an embodiment, each of the two portions created by partitioning may have a non-square shape.

[0544] In an embodiment, the selected portion of the two portions to which the second prediction will be applied can be selected based on selected portion information. The reconstructed block generation unit can obtain the selected portion information from the bitstream. The selected portion can be selected based on selected portion information indicating one of the two portions. The portion indicated by the unselected portion information of the two portions can be the remaining portion.

[0545] For example, the value of the selected portion information can indicate which of the two portions will be subject to the second prediction. When the value of the selected portion information is 0, the selected portion can be the one mentioned above. Figure 11 S1 as described. When the value of the selected information is 1, the selected part can be the above reference. Figure 11 The described S2.

[0546] In step 1730, the reconstruction block generation unit may add the second prediction signal for the selected portion to the reconstruction residual signal.

[0547] The second prediction signal can be a signal generated via a second prediction for a selected portion.

[0548] In step 1740, the reconstruction block generation unit may generate a first prediction signal for the current block or the remainder of the current block.

[0549] The reconstruction block generation unit can generate a first prediction signal by performing a first prediction on the current block or the remaining portion.

[0550] The first prediction signal can represent the first prediction block.

[0551] In the first prediction, the reconstructed neighboring blocks of the current block can be used as reference blocks. Furthermore, the forms in the reconstructed neighboring blocks of the current block can be used as reference samples.

[0552] In an embodiment, when the second prediction is not used, the first prediction signal for the current block may be generated by performing a first prediction on the current block. The first prediction signal may be a signal generated via the first prediction of the current block. When the second prediction is not used to decode the current block, the first prediction may be a prediction for the entire current block.

[0553] In an embodiment, when the second prediction is not used, the first prediction signal for the remaining portion is generated by performing a first prediction on the remaining portion. The first prediction signal may be a signal generated via the first prediction for the remaining portion. When the second prediction is used to decode the current block, the first prediction may be a prediction for the remaining portion of the current block excluding the selected portion used for the second prediction.

[0554] In step 1750, the reconstruction block generation unit can generate a reconstruction block by adding the first prediction signal for the current block or the remaining portion to the reconstruction residual signal.

[0555] In an embodiment, when the second prediction is not used, the reconstruction block generation unit can generate a reconstruction block by adding the first prediction signal for the current block to the reconstruction residual signal. The first prediction signal may be a signal generated via the first prediction for the current block.

[0556] In an embodiment, when the second prediction is not used, the reconstruction block generation unit can generate a reconstruction block by adding the first prediction signal for the remaining portion to the reconstruction residual signal. The first prediction signal may be a signal generated via the first prediction for the remaining portion.

[0557] According to steps 1710, 1720, 1730, 1740, and 1750 described above, when the second prediction is used, the signal representing the reconstructed block can be the sum of the reconstructed residual signal, the second prediction signal for the selected portion, and the first prediction signal for the remaining portion. As described above with reference to steps 1730 and 1750, when the second prediction is used, the second prediction signal for the selected portion and the first prediction signal for the remaining portion can be added to the reconstructed residual signal. Optionally, when the second prediction is used, the reconstructed block can be generated based on the reconstructed residual signal, the second prediction signal for the selected portion, and the first prediction signal for the remaining portion.

[0558] According to steps 1710, 1720, 1730, 1740, and 1750 described above, when the second prediction is not used, the signal representing the reconstructed block can be the sum of the reconstructed residual signal and the first prediction signal for the current block. Optionally, when the second prediction is not used, the reconstructed block can be generated based on the reconstructed residual signal and the first prediction signal for the current block.

[0559] The reconstructed block can be used as a reference block for decoding other blocks.

[0560] According to the embodiments described above, when the second prediction is used, the selected portion of the first prediction applied to it and the remaining portion of the second prediction applied to it can be separated.

[0561] In an embodiment, when the second prediction is used, the first prediction can be applied to the entire current block. In this case, a reconstructed block for a selected portion to which the second prediction is applied can be generated based on the reconstruction reference signal, the second prediction signal generated via the second prediction, and the sum of the first prediction signal generated via the first prediction. Furthermore, a reconstructed block for the remaining portion to which the second prediction is not applied can be generated based on the reconstructed residual block and the sum of the first prediction signal generated via the first prediction.

[0562] In this scenario, the second prediction can be a prediction of the residual. The second prediction can be a prediction of the first residual signal, where the first residual signal is the difference between the current block and the first predicted signal. The reference block used for the second prediction can be a reconstructed residual block, and the reference pixel used for the second prediction can be a pixel within the reconstructed residual block. In other words, a second residual signal, which is the difference between the first residual signal and the second predicted signal, can be generated via a second prediction of the first residual signal, and the second residual signal can be used to generate the encoding information for the current block.

[0563] Optionally, in this case, the reconstruction block for the selected portion to which the second prediction is applied can be a weighted sum of the reconstruction residual block, the second prediction signal, and the first prediction signal. Prediction-related information may include a first weight for the reconstruction residual signal, a second weight for the second prediction signal, and a third weight for the first prediction signal. It can be assumed that not using the second prediction means the second weight for the second prediction signal is set to 0. Optionally, it can be assumed that for the remaining portion to which the second prediction is not applied, the second weight for the second prediction is set to 0.

[0564] The description of the encoding of the current block described above can also be applied to the decoding of the current block. (Repeated description omitted.) Furthermore, the description of the decoding of the current block described above can also be applied to the encoding of the current block. (Repeated description omitted.)

[0565] In the embodiments described above, although the method has been described based on a flowchart as a series of steps or units, the present invention is not limited to the order of the steps, and some steps may be performed in a different order than those described or simultaneously with other steps. Furthermore, those skilled in the art will understand that the steps shown in the flowchart are not exclusive and may include other steps, or one or more steps in the flowchart may be omitted without departing from the scope of the invention.

[0566] The embodiments of the invention described above can be implemented as programs executable by various computer devices and recorded on a computer-readable storage medium. The computer-readable storage medium may individually or in combination include program instructions, data files, and data structures. The program instructions recorded on the storage medium may be specifically designed or configured for the present invention, or may be known or available to those skilled in the art of computer software. Examples of computer storage media may include all types of hardware devices specifically configured to record and execute program instructions, such as magnetic media (e.g., hard disks, floppy disks, and magnetic tapes), optical media (e.g., CD-ROMs and DVDs), and magneto-optical media (e.g., floppy disks, ROMs, RAMs, and flash memory). Examples of program instructions include machine code (e.g., code created by a compiler) and high-level language code executable by a computer using an interpreter. The hardware device may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

[0567] As described above, although the invention has been described based on specific details (such as detailed components and a limited number of embodiments and drawings), these specific details are provided only for ease of understanding of the invention. The invention is not limited to these embodiments, and those skilled in the art will practice various changes and modifications based on the above description.

[0568] Therefore, it should be understood that the spirit of this embodiment is not limited to the above embodiments, and the appended claims and their equivalents and modifications thereof fall within the scope of this invention.

Claims

1. A video decoding method, comprising: Perform a prediction on the current block to generate a prediction block; A reconstruction block is generated based on the predicted block; Filtering is performed on the reconstructed block. The prediction includes at least one of a first prediction and a second prediction. Determine whether the second prediction was used for the prediction of the current block. If it is determined that the second prediction will not be used, the first prediction is performed for the entire current block. If it is determined that a second prediction will be used, the second prediction is performed on a portion of the current block. Determine the first prediction pattern to use for the entire current block. After determining the first prediction pattern for the entire current block, the next step is to determine whether to perform partitioning for the current block. If it is determined that the partition will be executed, the second prediction mode used for the second prediction will be set to be equal to the first prediction mode.

2. The video decoding method as described in claim 1, wherein, Each of the first and second predictions is an intra-frame prediction.

3. The video decoding method as described in claim 1, wherein, The second prediction is used only if the first prediction pattern of the first prediction is a predefined pattern.

4. The video decoding method as described in claim 1, wherein, The first prediction is inter-frame prediction. The second prediction is intra-frame prediction. The reconstructed block is generated by applying a first weight to a first prediction block and a second weight to a second prediction block, wherein the first prediction block is generated by performing a first prediction and the second prediction block is generated by performing a second prediction.

5. The video decoding method as described in claim 1, wherein, The first reference block of the first prediction reference is the reconstructed brightness block. The second reference block, which is the second prediction reference, is the reconstructed chroma block.

6. The video decoding method as described in claim 1, wherein, The reconstructed block is generated by applying weights to a first prediction block and a second prediction block, wherein the first prediction block is generated by performing a first prediction, and the second prediction block is generated by performing a second prediction. The reconstructed blocks are generated using bidirectional inter-frame prediction.

7. A video encoding method, comprising: Perform a prediction on the current block to generate a prediction block; A reconstruction block is generated based on the predicted block; Filtering is performed on the reconstructed block. The prediction includes at least one of a first prediction and a second prediction. Determine whether the second prediction was used for the prediction of the current block. If it is determined that the second prediction will not be used, the first prediction is performed for the entire current block. If it is determined that a second prediction will be used, the second prediction is performed on a portion of the current block. Determine the first prediction pattern to use for the entire current block. After determining the first prediction pattern for the entire current block, the next step is to determine whether to perform partitioning for the current block. If it is determined that the partition will be executed, the second prediction mode used for the second prediction will be set to be equal to the first prediction mode.

8. The video encoding method as described in claim 7, wherein, Each of the first and second predictions is an intra-frame prediction.

9. The video encoding method as described in claim 7, wherein, The second prediction is used only if the first prediction pattern of the first prediction is a predefined pattern.

10. The video encoding method as described in claim 7, wherein, The first prediction is inter-frame prediction. The second prediction is intra-frame prediction. The reconstructed block is generated by applying a first weight to a first prediction block and a second weight to a second prediction block, wherein the first prediction block is generated by performing a first prediction and the second prediction block is generated by performing a second prediction.

11. The video encoding method as described in claim 7, wherein, The first reference block of the first prediction reference is the reconstructed brightness block. The second reference block, which is the second prediction reference, is the reconstructed chroma block.

12. The video encoding method as described in claim 7, wherein, The reconstructed block is generated by applying weights to a first prediction block and a second prediction block, wherein the first prediction block is generated by performing a first prediction, and the second prediction block is generated by performing a second prediction. The reconstructed blocks are generated using bidirectional inter-frame prediction.

13. A computer-readable recording medium for storing a bitstream generated by a computer program, said computer program, when executed by a processor, causing a video encoding device to perform the following steps: Perform a prediction for the current block based on the prediction pattern; and Generate a bitstream that includes prediction pattern information; in, The prediction pattern information indicates that the prediction pattern is used for the current block. The prediction includes at least one of a first prediction and a second prediction. Determine whether the second prediction was used for the prediction of the current block. If it is determined that the second prediction will not be used, the first prediction is performed for the entire current block. If it is determined that a second prediction will be used, the second prediction is performed on a portion of the current block. Determine the first prediction pattern to be used for the entire current block. After determining the first prediction pattern for the entire current block, the next step is to determine whether to perform partitioning for the current block. If it is determined that the partition will be executed, the second prediction mode used for the second prediction will be set to be equal to the first prediction mode.

14. A computer-readable recording medium configured to store a bit stream generated by a computer program, wherein, When the computer program is executed by the processor, it causes the video decoding device to perform the following steps: The prediction mode for the current block is determined based on the prediction mode information; Based on the prediction mode, a prediction block is generated by performing a prediction on the current block; A reconstruction block is generated based on the predicted block; Filtering is performed on the reconstructed block. The prediction includes at least one of a first prediction and a second prediction. Determine whether the second prediction was used for the prediction of the current block. If it is determined that the second prediction will not be used, the first prediction is performed for the entire current block. If it is determined that a second prediction will be used, the second prediction is performed on a portion of the current block. Determine the first prediction pattern to be used for the entire current block. After determining the first prediction pattern for the entire current block, the next step is to determine whether to perform partitioning for the current block. If it is determined that the partition will be executed, the second prediction mode used for the second prediction will be set to be equal to the first prediction mode.

15. The computer-readable recording medium of claim 14, wherein, Each of the first and second predictions is an intra-frame prediction.

16. The computer-readable recording medium of claim 14, wherein, The second prediction is used only if the prediction pattern of the first prediction is a predefined pattern.

17. The computer-readable recording medium of claim 14, wherein, The first prediction is inter-frame prediction. The second prediction is intra-frame prediction. The reconstructed block is generated by applying a first weight to a first prediction block and a second weight to a second prediction block, wherein the first prediction block is generated by performing a first prediction and the second prediction block is generated by performing a second prediction.

18. The computer-readable recording medium of claim 14, wherein, The first reference block of the first prediction reference is the reconstructed brightness block. The second reference block, which is the second prediction reference, is the reconstructed chroma block.

19. The computer-readable recording medium of claim 14, wherein, The reconstructed block is generated by applying weights to a first prediction block and a second prediction block, wherein the first prediction block is generated by performing a first prediction, and the second prediction block is generated by performing a second prediction. The reconstructed blocks are generated using bidirectional inter-frame prediction.

20. A method for transmitting a bit stream, the method comprising: Send bit stream, The bitstream includes prediction mode information for the current block. The prediction mode information is used to determine the prediction mode for the current block. The prediction mode is used to perform a prediction for the current block to generate a prediction block. The prediction block is used to generate the reconstruction block. Among these steps, filtering of the reconstructed block is performed. The prediction includes at least one of a first prediction and a second prediction. Specifically, determining whether the second prediction is used for the prediction of the current block. In cases where it is determined that the second prediction will not be used, the first prediction is performed for the entire current block. Specifically, if it is determined that a second prediction will be used, the second prediction will be performed on a portion of the current block. Among these, the first prediction pattern for the entire current block is determined. After determining the first prediction pattern for the entire current block, the next step is to determine whether to perform partitioning for the current block. Wherein, when it is determined to perform the partition, the second prediction mode used for the second prediction is set to be equal to the first prediction mode.

21. A video decoding apparatus, comprising: A prediction unit is configured to perform a first prediction on the current block to generate a first prediction block, and to perform a second prediction on the current block to generate a second prediction block; A reconstruction block generation unit is used to generate reconstruction blocks based on a first prediction block and a second prediction block. A filtering unit is used to perform filtering on the reconstructed block. The first prediction is inter-frame prediction. The second prediction is intra-frame prediction. Obtain the second prediction usage information from the bitstream. The second prediction uses information to indicate whether intra-frame prediction of the second prediction has been performed. After determining whether the inter-frame prediction of the first prediction has been performed, it is then determined whether the intra-frame prediction of the second prediction has been performed. The reconstruction block is generated based on a weighted sum of the first and second prediction blocks, assuming that intra-frame prediction of the second prediction is performed.

22. A video encoding apparatus, comprising: A prediction unit is configured to perform a first prediction on the current block to generate a first prediction block, and to perform a second prediction on the current block to generate a second prediction block; A reconstruction block generation unit is used to generate reconstruction blocks based on a first prediction block and a second prediction block. A filtering unit is used to perform filtering on the reconstructed block. The first prediction is inter-frame prediction. The second prediction is intra-frame prediction. After determining whether the inter-frame prediction of the first prediction has been performed, it is then determined whether the intra-frame prediction of the second prediction has been performed. Generate a bitstream that includes information used in the second prediction. The second prediction uses information to indicate whether intra-frame prediction of the second prediction has been performed. The reconstruction block is generated based on a weighted sum of the first and second prediction blocks, assuming that intra-frame prediction of the second prediction is performed.

23. A computer-readable recording medium for storing a bitstream generated by a computer program, said computer program, when executed by a processor, causing a video encoding device to perform the following steps: Based on the prediction pattern, perform the first prediction for the current block to generate the first prediction block; Based on the prediction mode, a second prediction is performed on the current block to generate a second prediction block; and Generate a bitstream that includes prediction pattern information and second prediction usage information. in, The prediction pattern information indicates that the prediction pattern is used for the current block. The first prediction is inter-frame prediction. The second prediction is intra-frame prediction. After determining whether the inter-frame prediction of the first prediction has been performed, it is then determined whether the intra-frame prediction of the second prediction has been performed. The second prediction uses information to indicate whether intra-frame prediction of the second prediction has been performed. The reconstructed block used for the current block is generated based on a weighted sum of the first and second prediction blocks, and the reconstructed block is filtered.

24. A computer-readable recording medium configured to store a bit stream generated by a computer program, wherein, When the computer program is executed by the processor, it causes the video decoding device to perform the following steps: The prediction pattern for the current block is determined based on the prediction pattern information. The first prediction block is generated by performing the first prediction; The second prediction block is generated by performing the second prediction; Reconstructed blocks are generated based on the first and second prediction blocks; Filtering is performed on the reconstructed block. in, The first prediction is an inter-frame prediction performed on the entire current block. The second prediction is intra-frame prediction. After determining whether the inter-frame prediction of the first prediction has been performed, it is then determined whether the intra-frame prediction of the second prediction has been performed. The bitstream includes second prediction usage information. The second prediction uses information to indicate whether intra-frame prediction of the second prediction has been performed. The reconstruction block is generated based on a weighted sum of the first and second prediction blocks, assuming that intra-frame prediction of the second prediction is performed.

25. A method for transmitting a bit stream, the method comprising: Send bit stream, The bitstream includes prediction mode information and second prediction usage information for the current block. The prediction mode information is used to determine the prediction mode for the current block. The prediction mode indicates that a first prediction for the current block is performed to generate a first prediction block. The prediction mode indicates that a second prediction for the current block is performed to generate a second prediction block. The first and second prediction blocks are used to generate the reconstruction blocks. Filtering is performed on the reconstructed block. The first prediction is inter-frame prediction. The second prediction is intra-frame prediction. Specifically, after determining that the inter-frame prediction of the first prediction has been performed, it is then determined whether the intra-frame prediction of the second prediction has been performed. The second prediction uses information to indicate whether intra-frame prediction of the second prediction has been performed. The reconstruction block is generated based on a weighted sum of the first and second prediction blocks, provided that an intra-frame prediction of the second prediction is performed.

Images