Image encoding methods and image decoding methods, as well as computer-readable recording media
By acquiring the transform coefficient distribution information of neighboring blocks, the intra-frame prediction mode is adaptively determined and finely divided, which solves the problems of large information content and high computational complexity of intra-frame prediction mode in the prior art, and realizes more efficient image compression and simplified decoding process.
Patent Information
- Application Number
- CN202311570502.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2017-07-28
- Filing Date
- 2018-07-27
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2038-07-27
AI Technical Summary
In existing technologies, intra-frame prediction modes contain a large amount of information, resulting in low compression efficiency and high computational complexity. Decoding devices also have parsing dependencies, which increases the difficulty of implementing such devices.
By acquiring the transform coefficient distribution information of neighboring blocks, the intra-prediction mode of the current block is adaptively determined in the direction. The intra-prediction mode is finely divided using similarity flag information, which increases the number of candidate modes in a specific direction, reduces the amount of information, and optimizes the coding process.
It improves image compression efficiency and quality, reduces computational complexity, decreases dependence on decoding devices, and simplifies device configuration.
Smart Images

Figure CN117395409B_ABST
Abstract
Description
[0001] This application is a divisional application of the invention patent application filed on July 27, 2018, with application number "201880050380.6" and titled "Image Processing Method and Image Encoding / Decoding Method and Apparatus Using Image Processing Method and Image Encoding / Decoding Method". Technical Field
[0002] This invention relates to an image encoding / decoding method, an image encoding / decoding apparatus, and a recording medium for storing bitstreams. More specifically, this invention relates to an image encoding / decoding method and apparatus using intra-frame prediction, and a recording medium for storing bitstreams generated by said image encoding / decoding method and apparatus. Background Technology
[0003] Digital video technology can be found in a variety of digital video devices, such as digital television, digital direct broadcasting systems, wireless broadcasting systems, personal digital assistants (PDAs), laptops or desktop computers, digital cameras, digital recording devices, video game devices, and video game consoles. Digital video devices efficiently transmit and receive digital video information by using video compression technologies such as MPEG-2, MPEG-4, ITU-T H.264 / MPEG-4, Part 10, Advanced Video Coding (AVC), and H.265 / HEVC (High-Efficiency Video Coding). Video compression technologies involve spatial and temporal prediction to remove or reduce inherent redundancy in video sequences.
[0004] Various image compression techniques exist, such as inter-frame prediction for predicting pixel values within the current frame from previous or subsequent frames, intra-frame prediction for predicting pixel values within a region of the current frame from another region of the current frame, and entropy coding for assigning short codes to frequently occurring pixel values and long codes to less frequently occurring pixel values. These image compression techniques can be used to efficiently compress, transmit, and store video data.
[0005] In order to economically and efficiently process images of various resolutions and frame rates required in these applications, a video decoding device is needed that can easily process images according to the performance and functionality required in the application.
[0006] For example, when compressing an image, the image is first divided into multiple blocks, each with a predetermined size, and then encoding is performed based on each block. To improve compression efficiency, inter-frame prediction and intra-frame prediction techniques are used to eliminate redundancy in the image.
[0007] In this scenario, residual signals can be generated through intra-frame or inter-frame prediction. Encoding the residual signal reduces the amount of data, thus increasing the data compression ratio. Because prediction is performed with higher efficiency, residual signals with smaller values are obtained. Therefore, this residual signal can be used to perform more efficient predictions.
[0008] Intra-frame prediction uses pixels surrounding the current block to predict the data for that block. The difference between the actual value and the predicted value of each pixel in the current block is called the residual signal block. For intra-frame prediction, HEVC supports a greater number of prediction modes (35 modes) compared to the existing H.264 / AVC, which supports 9 prediction modes.
[0009] For inter-frame prediction, the current block is compared with blocks in neighboring frames to find the most similar block. The location information (Vx, Vy) of the located block is called the motion vector. The difference between each pixel in the current block and each pixel in the prediction block created based on the motion vector is called the residual signal or motion-compensated residual block.
[0010] In this way, the accuracy of intra-frame and inter-frame prediction is improved. As a result, while the amount of residual signal data is reduced, the computational cost for processing moving images is greatly increased.
[0011] Specifically, for more efficient image compression, 67 intra-frame prediction modes can be used in existing technologies. These modes include planar mode, DC mode, and 65 intra-frame orientation modes, and can process the orientation of edges in natural images more accurately and precisely.
[0012] Because of these subdivided intra-prediction modes, the number of bits required to encode information about the optimal intra-prediction mode is greatly increased, which leads to a reduction in compression efficiency.
[0013] To address this issue, i.e. to reduce the amount of information, the optimal intra-frame mode for the current block can be derived from information about other blocks that have already been decoded.
[0014] However, while the technique of deriving the intra-frame mode used to predict the current block can reduce the compression ratio to some extent, it has the problem of greatly increasing the computational complexity for encoding and decoding.
[0015] Specifically, the following problems exist: During the derivation of the intra-frame mode of the current block, it is first necessary to parse the decoded values or intra-frame modes of other blocks. Furthermore, when referencing other blocks to encode or decode the current block, the decoding device has a parsing dependency between the decoding processing of previous blocks and the decoding processing of the current block. This not only increases computational complexity but also causes difficulties in implementing device modules and configuring data. Summary of the Invention
[0016] Technical issues
[0017] The present invention aims to provide an image encoding / decoding method and device that can improve image quality and compression efficiency.
[0018] Furthermore, the present invention aims to provide an image encoding / decoding method and apparatus that performs image encoding / decoding through intra-frame prediction, which can improve compression efficiency.
[0019] Furthermore, the present invention aims to provide a recording medium wherein the recording medium stores a bitstream generated by an image encoding / decoding method or device.
[0020] Technical solution
[0021] A method for decoding an image according to the present invention may include: obtaining transform coefficient distribution information of neighboring blocks from a bitstream; adaptively determining the intra-prediction mode of the current block to be decoded based on the transform coefficient distribution information of neighboring blocks; and reconstructing the current block based on the determined intra-prediction mode, wherein the step of adaptively determining the intra-prediction mode of the current block to be decoded includes: determining similarity flag information corresponding to neighboring blocks based on the transform coefficient distribution information of neighboring blocks.
[0022] In the method for decoding an image according to the present invention, when a neighboring block is located to the left of the current block, the similarity marker information includes horizontal similarity marker information.
[0023] In the method for decoding an image according to the present invention, when a neighboring block is located above the current block, the similarity marker information includes vertical similarity marker information.
[0024] In the method for decoding an image according to the present invention, the step of adaptively determining the intra-prediction mode of the current block to be decoded includes: finely dividing the intra-prediction modes in a specific direction according to similarity flag information for the intra-prediction mode of the current block to increase the number of candidate modes in the specific direction.
[0025] In the method for decoding an image according to the present invention, the step of finely dividing the intra-prediction modes in a specific direction includes: dividing the intra-prediction modes that can be used in the current block in a predetermined horizontal or vertical direction range according to similarity flag information.
[0026] In the method for decoding an image according to the present invention, the step of finely dividing the intra-prediction mode in a specific direction includes: finely dividing the mode corresponding to the horizontal similarity mark or the vertical similarity mark in the most probable mode (MPM) candidate list for the current block according to similarity mark information; and adding the divided mode to the MPM candidate list.
[0027] The method for decoding an image according to the present invention further includes: obtaining intra-prediction mode information of neighboring blocks, wherein the step of adaptively determining the intra-prediction mode of the current block to be decoded includes: determining horizontal similarity marker information or vertical similarity marker information of neighboring blocks based on the intra-prediction mode information and transform coefficient distribution information of neighboring blocks.
[0028] In the method for decoding an image according to the present invention, the step of finely dividing the intra-prediction modes in a specific direction includes: restricting the intra-prediction modes available in the current block to the horizontal or vertical direction based on similarity flag information.
[0029] An image decoding apparatus according to the present invention may include: a neighboring block information processing unit configured to obtain transform coefficient distribution information of neighboring blocks from a bitstream; a mode determination unit configured to adaptively determine an intra-prediction mode of the current block to be decoded based on the transform coefficient distribution information of neighboring blocks; and an intra-prediction unit configured to reconstruct the current block based on the determined intra-prediction mode, wherein the mode determination unit determines similarity flag information corresponding to neighboring blocks based on the transform coefficient distribution information of neighboring blocks.
[0030] In the device for decoding an image according to the present invention, when a neighboring block is located to the left of the current block, the similarity flag information includes horizontal similarity flag information.
[0031] In the device for decoding an image according to the present invention, when a neighboring block is located above the current block, the similarity marker information includes vertical similarity marker information.
[0032] In the image decoding apparatus according to the present invention, the mode determination unit performs fine division of candidate modes for intra-frame prediction modes for the current block based on similarity flag information.
[0033] In the image decoding apparatus according to the present invention, the mode determination unit performs fine division of intra-frame prediction modes that can be used in the current block within a predetermined range in the horizontal or vertical direction based on similarity flag information.
[0034] In the image decoding apparatus according to the present invention, the mode determination unit performs fine division of the modes corresponding to the horizontal similarity markers or the vertical similarity markers within the most probable mode (MPM) candidate list of the current block based on similarity marker information.
[0035] The device for decoding an image according to the present invention further includes: a neighbor pattern recognition unit configured to obtain intra-frame prediction mode information of neighboring blocks, wherein the pattern determination unit determines horizontal similarity marker information or vertical similarity marker information of neighboring blocks based on the intra-frame prediction mode information and transform coefficient distribution information of neighboring blocks.
[0036] In the image decoding apparatus according to the present invention, the mode determination unit restricts the intra-frame prediction mode available in the current block to either the horizontal or vertical direction based on similarity flag information.
[0037] An image encoding method according to the present invention includes: acquiring information about neighboring blocks of a current block to be encoded; adaptively determining an intra-prediction mode of the current block to be encoded based on the information about the neighboring blocks; and encoding the current block according to the determined intra-prediction mode.
[0038] In the method for encoding an image according to the present invention, the step of adaptively determining the intra-prediction mode of the current block to be encoded includes: determining similarity flag information corresponding to the neighboring blocks based on the transform coefficient distribution information of the neighboring blocks.
[0039] In the method for encoding an image according to the present invention, a lookup table for entropy encoding is determined based on directional information of the current block determined based on information about neighboring blocks.
[0040] A computer-readable recording medium according to the present invention can store a bitstream generated by the image encoding method according to the present invention.
[0041] It will be understood that the foregoing generalized features are exemplary aspects of the invention as described in the following detailed description, and do not limit the scope of the invention.
[0042] Beneficial effects
[0043] According to the present invention, an image encoding / decoding method and apparatus capable of improving image quality and compression efficiency can be provided.
[0044] Furthermore, according to the present invention, an image encoding / decoding method and apparatus for performing image encoding / decoding via intra-frame prediction can be provided, the method and apparatus being able to improve compression efficiency.
[0045] Furthermore, according to the present invention, a recording medium may be provided, wherein the recording medium stores a bit stream generated by an image encoding / decoding method or device. Attached Figure Description
[0046] Figure 1 This is a block diagram illustrating the configuration of an encoding device applying an embodiment of the present invention;
[0047] Figure 2 This is a block diagram illustrating the configuration of a decoding device applying an embodiment of the present invention;
[0048] Figure 3 This is a schematic diagram showing the partition structure of an image used for image encoding / decoding;
[0049] Figure 4 This is a diagram illustrating one embodiment of intra-frame prediction processing;
[0050] Figure 5 This is a diagram illustrating one embodiment of inter-frame prediction processing;
[0051] Figure 6 This is a diagram illustrating the transformation and quantization processes;
[0052] Figure 7 This is a block diagram illustrating the configuration of an image encoding apparatus according to an embodiment of the present disclosure;
[0053] Figures 8 to 11 This is a diagram illustrating a method for partitioning an image into blocks and performing image processing based on each block, according to an embodiment of the present disclosure;
[0054] Figure 12 This is a block diagram illustrating an inter-frame prediction method performed in an image coding apparatus according to an embodiment of the present disclosure;
[0055] Figure 13 This is a block diagram illustrating the configuration of an image decoding device according to an embodiment of the present disclosure;
[0056] Figure 14 This is a block diagram illustrating inter-frame prediction processing performed in an image decoding device according to an embodiment of the present disclosure;
[0057] Figure 15 This is a diagram illustrating a method for partitioning an image into blocks and performing image processing based on each block, according to another embodiment of the present disclosure;
[0058] Figure 16This is a diagram illustrating a method for partitioning an image into blocks and performing image processing based on each block, according to another embodiment of the present disclosure;
[0059] Figure 17 This is a diagram illustrating an embodiment of a method for forming a transform unit by partitioning coding units using a binary tree structure, according to an embodiment of the present disclosure;
[0060] Figure 18 This is an illustration of another embodiment of a method for partitioning an image into blocks and performing image processing based on each block, according to one embodiment of the present disclosure;
[0061] Figure 19 and Figure 20 These are illustrations of other embodiments of a method for partitioning an image into blocks and performing image processing based on each block, according to one embodiment of the present disclosure;
[0062] Figure 21 and Figure 22 This is a diagram illustrating a method for determining the partitioning structure of a transform unit by performing rate distortion optimization (RDO);
[0063] Figure 23 This is a block diagram illustrating the configuration of an intra-prediction unit for determining an intra-prediction mode according to an embodiment of the present disclosure.
[0064] Figure 24 and Figure 25 This is a flowchart illustrating a method for performing intra-predictive decoding processing using a direction-adaptive intra-predictive mode according to an embodiment of the present disclosure;
[0065] Figure 26 and Figure 27 This is a flowchart illustrating horizontal and vertical similarity markers for orientation-adaptive intra-frame mode according to an embodiment of the present disclosure;
[0066] Figures 28 to 31 This is a diagram illustrating a direction-adaptive intra-frame mode for certain situations according to an embodiment of the present disclosure; and
[0067] Figures 32 to 34 This is a diagram illustrating a method for improving entropy coding efficiency using information about neighboring blocks according to an embodiment of the present disclosure. Detailed Implementation
[0068] Various modifications can be made to this invention, and various embodiments of the invention exist, wherein examples of these various embodiments will now be provided with reference to the accompanying drawings, and examples of these various embodiments will be described in detail. However, the invention is not limited thereto, although exemplary embodiments may be interpreted as including all modifications, equivalents, or substitutions within the technical concept and scope of the invention. Similar reference numerals indicate the same or similar functions in various respects. In the drawings, the shapes and sizes of elements may be exaggerated for clarity. In the following detailed description of the invention, reference is made to the accompanying drawings, which illustrate specific embodiments in which the invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice this disclosure. It should be understood that the various embodiments of this disclosure, though different, are not necessarily mutually exclusive. For example, specific features, structures, and characteristics described herein in connection with one embodiment may be implemented in other embodiments without departing from the spirit and scope of this disclosure. Furthermore, it should be understood that the position or arrangement of various elements within each disclosed embodiment may be modified without departing from the spirit and scope of this disclosure. Therefore, the following detailed description should not be construed in a limiting sense, and the scope of this disclosure is defined only by the appended claims (and, where appropriate, the full scope of the equivalents claimed in the claims).
[0069] The terms "first," "second," etc., used in this specification may be used to describe various components, but these components are not to be construed as being limited to these terms. These terms are only used to distinguish one component from others. For example, without departing from the scope of the invention, a "first" component may be referred to as a "second" component, and a "second" component may similarly be referred to as a "first" component. The term "and / or" includes a combination of multiple items or any one of multiple items.
[0070] It will be understood that, in this specification, when an element is referred to only as "connected to" or "joined to" another element rather than "directly connected to" or "directly joined to" another element, the element may be "directly connected to" or "directly joined to" the other element, or connected to or joined to the other element if there are other elements between the element and the other element. Conversely, it should be understood that when an element is referred to as "directly joined" or "directly connected" to another element, there are no intermediate elements.
[0071] Furthermore, the components shown in the embodiments of the present invention are illustrated independently to present distinct functionalities. Therefore, this does not imply that each component is composed as a separate hardware or software unit. In other words, for convenience, each component includes every one of the enumerated components. Thus, at least two components in each component may be combined to form a single component, or a single component may be divided into multiple components for performing each function. Embodiments where each component is combined and embodiments where a component is divided are also included within the scope of the invention without departing from its spirit.
[0072] The terminology used in this specification is for describing particular embodiments only and is not intended to limit the invention. Expressions used in the singular include plural expressions unless they have a distinct meaning in the context. In this specification, it will be understood that terms such as “comprising,” “having,” etc., are intended to indicate the presence of the features, quantities, steps, actions, elements, components, or combinations thereof disclosed in the specification, and are not intended to exclude the possibility that one or more other features, quantities, steps, actions, elements, components, or combinations thereof may be present or added. In other words, when a particular element is referred to as “comprising,” elements other than the corresponding element are not excluded; rather, additional elements may be included in embodiments of the invention or within the scope of the invention.
[0073] Furthermore, some components may not be essential for performing the necessary functions of the invention, but rather optional components that merely enhance its performance. The invention can be implemented by including only the essential components necessary for carrying out the invention, excluding components used to enhance performance. Structures that include only the essential components and exclude optional components used solely for enhancing performance are also included within the scope of the invention.
[0074] In the following, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In describing exemplary embodiments of the invention, well-known functions or structures will not be described in detail, as they would unnecessarily obscure the understanding of the invention. The same constituent elements in the drawings are denoted by the same reference numerals, and repeated descriptions of the same elements will be omitted.
[0075] In the following text, an image may refer to a frame that constitutes a video, or it may refer to the video itself. For example, "encoding or decoding an image or both" may refer to "encoding or decoding a moving image or both," and may also refer to "encoding or decoding an image within an image of a moving image or both."
[0076] In the following text, the terms "moving images" and "video" may be used to mean the same thing and may be used interchangeably.
[0077] In the following text, the target image can be an encoded target image that serves as an encoding target and / or a decoded target image that serves as a decoding target. Furthermore, the target image can be an input image input to an encoding device and an input image input to a decoding device. Here, the target image may have the same meaning as the current image.
[0078] In the following text, the terms “image,” “picture,” “frame,” and “screen” may be used as having the same meaning and interchangeably with each other.
[0079] In the following text, a target block can be an encoded target block that serves as the encoding target and / or a decoded target block that serves as the decoding target. Furthermore, a target block can be the current block that serves as the target of the current encoding and / or decoding. For example, the terms "target block" and "current block" can be used to mean the same thing and are interchangeable.
[0080] In the following text, the terms “block” and “unit” may be used to mean the same thing and are interchangeable. Alternatively, “block” may refer to a specific unit.
[0081] In the following text, the terms “region” and “fragment” are used interchangeably.
[0082] In the following text, a specific signal can be a signal representing a specific block. For example, the original signal can be a signal representing the target block. The prediction signal can be a signal representing the prediction block. The residual signal can be a signal representing the residual block.
[0083] In this embodiment, each of the specific information, data, flags, indexes, elements, and attributes may have a value. A value equal to "0" for information, data, flags, indexes, elements, and attributes may represent logical false or a first predefined value. In other words, the values "0", false, logical false, and the first predefined value can be interchanged with each other. A value equal to "1" for information, data, flags, indexes, elements, and attributes may represent logical true or a second predefined value. In other words, the values "1", true, logical true, and the second predefined value can be interchanged with each other.
[0084] When variables i or j are used to represent columns, rows, or indices, the value of i can be an integer equal to or greater than 0, or an integer equal to or greater than 1. That is, columns, rows, indices, etc., can be counted starting from 0, or they can be counted starting from 1.
[0085] Terminology Description
[0086] Encoder: Represents the device that performs encoding. In other words, it refers to the encoding device.
[0087] Decoder: Represents the device that performs decoding. In other words, it refers to the decoding device.
[0088] A block is an M×N sample array. Here, M and N can represent positive integers, and a block can represent a two-dimensional sample array. A block can refer to a unit. The current block can represent a coding target block that becomes the target during encoding, or a decoding target block that becomes the target during decoding. Furthermore, the current block can be at least one of a coding block, a prediction block, a residual block, and a transform block.
[0089] Samples are the basic units that make up a block. Based on the bit depth (Bd), samples can be represented as numbers from 0 to 2. Bd The value is -1. In this invention, a sample point can be used to represent a pixel. That is, a sample point, a pel, and a pixel can have the same meaning.
[0090] Unit: Refers to an encoding and decoding unit. When encoding and decoding an image, a unit can be a region generated by partitioning a single image. Furthermore, a unit can represent a sub-partitioning unit when a single image is partitioned into sub-partitioning units during encoding or decoding. That is, an image can be partitioned into multiple units. When encoding and decoding an image, predetermined processing can be performed for each unit. A single unit can be partitioned into sub-units smaller than the unit's size. Depending on the function, a unit can represent a block, macroblock, coding tree unit, coding tree block, coding unit, coding block, prediction unit, prediction block, residual unit, residual block, transform unit, transform block, etc. Furthermore, to distinguish a unit from a block, a unit can include a luma component block, a chroma component block associated with the luma component block, and syntax elements for each chroma component block. Units can have various sizes and shapes; specifically, the shape of a unit can be a two-dimensional geometric figure, such as a square, rectangle, trapezoid, triangle, pentagon, etc. In addition, the cell information may include at least one of the following: cell type indicating coding cell, prediction cell, transform cell, etc., cell size, cell depth, and the order of encoding and decoding of the cell.
[0091] A coding tree unit is a single coding tree block configured with the luminance component Y and two coding tree blocks associated with the chrominance components Cb and Cr. Furthermore, a coding tree unit can represent a block and the syntax elements of each block. Each coding tree unit can be partitioned using at least one of quadtree partitioning, binary tree partitioning, and ternary tree partitioning methods to configure lower-level units such as coding units, prediction units, transform units, etc. A coding tree unit can be used as a term to specify a sample block that becomes a processing unit when encoding / decoding an image as an input image. Here, a quadtree can represent a quaternion tree.
[0092] Encoding block: Can be used as a term to specify any one of the Y encoding block, Cb encoding block, and Cr encoding block.
[0093] Neighboring blocks: These can represent blocks adjacent to the current block. A block adjacent to the current block can be a block that touches the boundary of the current block, or a block located within a predetermined distance from the current block. A neighboring block can also represent a block adjacent to a vertex of the current block. Here, a block adjacent to a vertex of the current block can be a block that is vertically adjacent to a block horizontally adjacent to the current block, or a block that is horizontally adjacent to a block vertically adjacent to the current block.
[0094] Reconstructed neighboring blocks: These can represent neighboring blocks that are adjacent to the current block and have already been spatially / temporally encoded or decoded. Here, reconstructed neighboring blocks can represent reconstructed neighboring units. Reconstructed spatial neighboring blocks can be blocks within the current frame that have already been reconstructed through encoding or decoding, or both. Reconstructed temporally neighboring blocks are blocks within a reference image that are located at the position corresponding to the current block in the current frame, or neighboring blocks of said block.
[0095] Cell depth: Represents the degree of cell partitioning. In a tree structure, the highest node (root node) corresponds to the first cell that is not partitioned. Furthermore, the highest node can have the minimum depth value. In this case, the depth of the highest node can be level 0. A node with a depth of level 1 represents a cell generated by partitioning the first cell once. A node with a depth of level 2 represents a cell generated by partitioning the first cell twice. A node with a depth of level n represents a cell generated by partitioning the first cell n times. Leaf nodes can be the lowest-level nodes and cannot be further partitioned. The depth of a leaf node can be the highest-level. For example, the predefined value for the highest-level can be 3. The root node can have the lowest depth, and the leaf nodes can have the deepest depth. Additionally, when cells are represented as a tree structure, the level at which the cell exists can represent the cell depth.
[0096] Bitstream: A bitstream that can represent encoded image information.
[0097] Parameter set: Corresponds to the header information in the configuration within the bitstream. At least one of the video parameter set, sequence parameter set, frame parameter set, and adaptive parameter set may be included in the parameter set. In addition, the parameter set may include slice header and tile header information.
[0098] Explanation: This could mean determining the value of a syntax element by performing entropy decoding, or it could mean entropy decoding itself.
[0099] Symbols: can represent at least one of the syntax elements, encoding parameters, and transform coefficient values of the encoding / decoding target unit. Additionally, symbols can represent entropy encoding targets or entropy decoding results.
[0100] Prediction mode: This can be information indicating the mode that is encoded / decoded using intra-frame prediction or the mode that is encoded / decoded using inter-frame prediction.
[0101] Prediction Unit: A basic unit that can be represented when performing prediction (such as inter-frame prediction, intra-frame prediction, inter-frame compensation, intra-frame compensation, and motion compensation). A single prediction unit can be partitioned into multiple partitions with smaller sizes, or it can be partitioned into multiple lower-level prediction units. Multiple partitions can be the basic units when performing prediction or compensation. Partitions generated by dividing prediction units can also be prediction units.
[0102] Prediction cell partitioning: can represent the shape obtained by partitioning prediction cells.
[0103] A reference frame list can refer to a list of one or more reference frames used for inter-frame prediction or motion compensation. Several types of available reference frame lists exist, including LC (list combination), L0 (list 0), L1 (list 1), L2 (list 2), and L3 (list 3).
[0104] The inter-frame prediction indicator can indicate the direction of inter-frame prediction (one-way prediction, two-way prediction, etc.) for the current block. Optionally, the inter-frame prediction indicator can indicate the number of reference frames used to generate the prediction blocks for the current block. Optionally, the inter-frame prediction indicator can indicate the number of prediction blocks used when performing inter-frame prediction or motion compensation on the current block.
[0105] The prediction list utilization flag indicates whether at least one reference frame from a specific reference frame list is used to generate the prediction block. The prediction list utilization flag can be used to derive the inter-frame prediction indicator, and conversely, the inter-frame prediction indicator can be used to derive the prediction list utilization flag. For example, when the prediction list utilization flag has a first value of zero (0), it indicates that reference frames from the reference frame list are not used to generate the prediction block. On the other hand, when the prediction list utilization flag has a second value of one (1), it indicates that the reference frame list is used to generate the prediction block.
[0106] The reference screen index can refer to the index of a specific reference screen in the reference screen list.
[0107] A reference frame can refer to a frame referenced by a specific block for the purpose of inter-frame prediction or motion compensation of that specific block. Alternatively, a reference frame can be a frame that includes a reference block referenced by the current block for inter-frame prediction or motion compensation. In the following text, the terms "reference frame" and "reference image" have the same meaning and are interchangeable.
[0108] Motion vectors can be two-dimensional vectors used for inter-frame prediction or motion compensation. A motion vector can represent the offset between the encoded / decoded target block and the reference block. For example, (mvX, mvY) can represent a motion vector. Here, mvX can represent the horizontal component, and mvY can represent the vertical component.
[0109] The search range can be a two-dimensional region searched during inter-frame prediction to retrieve motion vectors. For example, the size of the search range can be M×N. Here, M and N are both integers.
[0110] Motion vector candidates can refer to a block of prediction candidates or the motion vectors within a block of prediction candidates when making predictions about motion vectors. Furthermore, motion vector candidates can be included in a list of motion vector candidates.
[0111] A motion vector candidate list can represent a list consisting of one or more motion vector candidates.
[0112] A motion vector candidate index can represent an indicator that points to a motion vector candidate in the motion vector candidate list. Alternatively, a motion vector candidate index can be an index of a motion vector predictor.
[0113] Motion information may represent information including at least one of the following: motion vector, reference frame index, inter-frame prediction indicator, prediction list utilization flag, reference frame list information, reference frame, motion vector candidate, motion vector candidate index, merge candidate, and merge index.
[0114] A merge candidate list can represent a list consisting of one or more merge candidates.
[0115] Merge candidates can be spatial merge candidates, temporal merge candidates, combined merge candidates, combined double prediction merge candidates, or zero merge candidates. Merge candidates may include motion information such as inter-frame prediction indicators, reference frame indices for each list, motion vectors, prediction list utilization flags, and inter-frame prediction indicators.
[0116] The merge index can represent an indicator that points to a merge candidate in the merge candidate list. Optionally, the merge index can indicate a block from which a merge candidate has been derived among the reconstructed blocks that are spatially / temporally adjacent to the current block. Optionally, the merge index can indicate at least one piece of motion information for the merge candidate.
[0117] Transform unit: This can represent the basic unit used when performing encoding / decoding (such as transform, inverse transform, quantization, dequantization, transform coefficient encoding / decoding) on the residual signal. A single transform unit can be partitioned into multiple lower-level transform units with smaller sizes. Here, the transform / inverse transform may include at least one of a first transform / first inverse transform and a second transform / second inverse transform.
[0118] Scaling: This refers to the process of multiplying the quantization level by a factor. Transform coefficients can be generated by scaling the quantization level. Scaling can also be called inverse quantization.
[0119] Quantization parameters: These represent values used when transform coefficients are used to generate quantization levels during quantization. Quantization parameters can also represent values used when transform coefficients are generated during dequantization by scaling the quantization levels. Quantization parameters can be values mapped to the quantization step size.
[0120] Incremental quantization parameter: can represent the difference between the predicted quantization parameter and the quantization parameter of the encoding / decoding target unit.
[0121] Scan: This can refer to a method of sorting coefficients within a cell, block, or matrix. For example, changing a two-dimensional matrix of coefficients into a one-dimensional matrix can be called a scan, and changing a one-dimensional matrix of coefficients into a two-dimensional matrix can be called a scan or inverse scan.
[0122] Transform coefficients: These represent the coefficient values generated after a transform is performed in the encoder. Transform coefficients can also represent the coefficient values generated after at least one of entropy decoding and dequantization is performed in the decoder. The quantization level obtained by quantizing the transform coefficients or residual signal, or the quantized transform coefficient level, can also fall within the meaning of transform coefficients.
[0123] Quantization level: This can represent the value generated in the encoder by quantizing the transform coefficients or residual signal. Optionally, the quantization level can represent the value of the dequantization target that will undergo dequantization in the decoder. Similarly, the transform coefficient level, as a result of transform and quantization, can also fall within the meaning of quantization level.
[0124] Non-zero transform coefficients: can represent transform coefficients with values other than zero, or transform coefficient levels or quantization levels with values other than zero.
[0125] Quantization matrix: A matrix used in quantization or dequantization processes to improve subjective or objective image quality. The quantization matrix can also be referred to as a scaling list.
[0126] Quantization matrix coefficients: These represent each element within the quantization matrix. Quantization matrix coefficients can also be called matrix coefficients.
[0127] Default matrix: can represent a predefined quantization matrix in the encoder or decoder.
[0128] Non-default matrix: can represent a quantization matrix that is not predefined in the encoder or decoder but is sent by the user via signal.
[0129] Statistical value: For at least one of the following variables, coding parameters, constant values, etc., that have a computable specific value, a statistical value can be one or more of the following: mean, weighted average, weighted sum, minimum, maximum, most frequent value, median, interpolation value.
[0130] Figure 1 This is a block diagram illustrating the configuration of an encoding device according to an embodiment of the present invention.
[0131] Encoding device 100 may be an encoder, a video encoding device, or an image encoding device. The video may include at least one image. Encoding device 100 may encode at least one image sequentially.
[0132] Reference Figure 1 The encoding device 100 may include a motion prediction unit 111, a motion compensation unit 112, an intra-frame prediction unit 120, a switcher 115, a subtractor 125, a transform unit 130, a quantization unit 140, an entropy coding unit 150, an inverse quantization unit 160, an inverse transform unit 170, an adder 175, a filter unit 180, and a reference frame buffer 190.
[0133] Encoding device 100 can encode the input image using intra-frame mode, inter-frame mode, or both. Furthermore, encoding device 100 can generate a bitstream including encoding information by encoding the input image and output the generated bitstream. The generated bitstream can be stored in a computer-readable recording medium or streamed via a wired / wireless transmission medium. When intra-frame mode is used as the prediction mode, switcher 115 can switch to intra-frame mode. Optionally, when inter-frame mode is used as the prediction mode, switcher 115 can switch to inter-frame mode. Here, intra-frame mode can refer to intra-frame prediction mode, and inter-frame mode can refer to inter-frame prediction mode. Encoding device 100 can generate prediction blocks for input blocks of the input image. Furthermore, encoding device 100 can encode residual blocks using the residual between the input block and the prediction block after generating the prediction blocks. The input image can be referred to as the current image as the current encoding target. The input block can be referred to as the current block as the current encoding target, or as the encoding target block.
[0134] When the prediction mode is intra-frame mode, the intra-frame prediction unit 120 can use samples from blocks that have been encoded / decoded and are adjacent to the current block as reference samples. The intra-frame prediction unit 120 can perform spatial prediction on the current block using the reference samples, or generate prediction samples for the input block by performing spatial prediction. Here, intra-frame prediction can refer to prediction within a frame.
[0135] When the prediction mode is inter-frame mode, the motion prediction unit 111 can retrieve the region that best matches the input block from the reference image during motion prediction and derive the motion vector using the retrieved region. In this case, the search region can be used as the region. The reference image can be stored in the reference frame buffer 190. Here, the reference image can be stored in the reference frame buffer 190 when encoding / decoding the reference image is performed.
[0136] The motion compensation unit 112 can generate a prediction block by performing motion compensation on the current block using motion vectors. Here, inter-frame prediction can refer to prediction or motion compensation between frames.
[0137] When the value of the motion vector is not an integer, the motion prediction unit 111 and the motion compensation unit 112 can generate prediction blocks by applying an interpolation filter to a portion of the reference frame. To perform inter-frame prediction or motion compensation on the coding unit, it can be determined which mode—skip mode, merge mode, advanced motion vector prediction (AMVP) mode, or current frame reference mode—will be used for motion prediction and motion compensation of the prediction unit included in the corresponding coding unit. Then, depending on the determined mode, inter-frame prediction or motion compensation can be performed differently.
[0138] Subtractor 125 can generate a residual block by using the residual between the input block and the prediction block. The residual block can be referred to as a residual signal. The residual signal can represent the difference between the original signal and the prediction signal. Furthermore, the residual signal can be a signal generated by transforming or quantizing, or transforming and quantizing, the difference between the original signal and the prediction signal. The residual block can be the residual signal of a block cell.
[0139] Transform unit 130 can generate transform coefficients by performing a transform on the residual block and output the generated transform coefficients. Here, the transform coefficients can be coefficient values generated by performing a transform on the residual block. When a transform skip mode is applied, transform unit 130 can skip the transform on the residual block.
[0140] The level of quantization can be generated by applying quantization to the transform coefficients or to the residual signal. In the following examples, the level of quantization may also be referred to as the transform coefficients.
[0141] The quantization unit 140 can generate a quantization level by quantizing the transform coefficients or residual signal according to parameters, and output the generated quantization level. Here, the quantization unit 140 can quantize the transform coefficients using a quantization matrix.
[0142] Entropy coding unit 150 can generate a bitstream by performing entropy coding on the values calculated by quantization unit 140 according to a probability distribution or on the coding parameter values calculated during encoding, and output the generated bitstream. Entropy coding unit 150 can perform entropy coding on sample information of the image and information used for decoding the image. For example, the information used for decoding the image may include syntax elements.
[0143] When entropy coding is applied, symbols are represented such that fewer bits are allocated to symbols with high generation probability and more bits are allocated to symbols with low generation probability, thus reducing the size of the bitstream used to encode the symbols. The entropy coding unit 150 can use coding methods for entropy coding such as Exponential Golomb, Context Adaptive Variable Length Coding (CAVLC), and Context Adaptive Binary Arithmetic Coding (CABAC). For example, the entropy coding unit 150 can perform entropy coding by using a variable-length code (VLC) table. Furthermore, the entropy coding unit 150 can derive a binarization method for the target symbol and a probability model for the target symbol / bits, and perform arithmetic coding by using the derived binarization method and context model.
[0144] In order to encode the transform coefficient levels (quantization levels), the entropy coding unit 150 can change the coefficients in two-dimensional block form into one-dimensional vector form by using a transform coefficient scanning method.
[0145] Encoding parameters may include information such as syntax elements (flags, indexes, etc.) encoded in the encoder and signaled to the decoder, as well as information derived during encoding or decoding. Encoding parameters can represent the information required when encoding or decoding an image. For example, at least one value or combination of the following may be included in the encoding parameters: cell / block size, cell / block depth, cell / block partitioning information, cell / block shape, cell / block partitioning structure, whether quadtree partitioning is performed, whether binary tree partitioning is performed, binary tree partitioning direction (horizontal or vertical), binary tree partitioning type (symmetric or asymmetric), whether the current encoding unit is partitioned via ternary tree partitioning, the direction of ternary tree partitioning (horizontal or vertical), the type of ternary tree partitioning (symmetric or asymmetric), whether the current encoding unit is partitioned via multi-type tree partitioning, and the direction of multi-type tree partitioning. This includes the following: orientation (horizontal or vertical), type of multi-type tree partitions (symmetric or asymmetric), tree structure of multi-type tree partitions (binary or ternary), prediction mode (intra-frame prediction or inter-frame prediction), intra-frame prediction mode / direction for luma, intra-frame prediction mode / direction for chroma, intra-frame partition information, inter-frame partition information, coded block partition flag, prediction block partition flag, transform block partition flag, reference sample filtering method, reference sample filter taps, reference sample filter coefficients, prediction block filtering method, prediction block filter taps, prediction block filter coefficients, prediction block boundary filtering method, prediction block boundary filter taps, prediction block boundary filter coefficients, and intra-frame prediction mode. Inter-frame prediction mode, motion information, motion vector, motion vector difference, reference frame index, inter-frame prediction angle, inter-frame prediction indicator, prediction list utilization flag, reference frame list, reference frame, motion vector predictor index, motion vector predictor candidate, motion vector candidate list, whether to use merge mode, merge index, merge candidate, merge candidate list, whether to use skip mode, interpolation filter type, interpolation filter taps, interpolation filter coefficients, motion vector magnitude, motion vector representation accuracy, transform type, transform size, information on whether the first (first) transform is used, information on whether the second transform is used, first transform index, second transform index Information on the presence of residual signals, code block style, code block flag (CBF), quantization parameters, quantization parameter residuals, quantization matrix, whether an intra-loop filter is applied, intra-loop filter coefficients, intra-loop filter taps, intra-loop filter shape / form, whether a deblocking filter is applied, deblocking filter coefficients, deblocking filter taps, deblocking filter strength, deblocking filter shape / form, whether adaptive sample offset is applied, adaptive sample offset value, adaptive sample offset category, adaptive sample offset type, whether an adaptive loop filter is applied, adaptive loop filter coefficients, adaptive loop filter taps, adaptive loop filter shape / form.Binarization / debinarization method, context model determination method, context model update method, whether to execute normal mode, whether to execute bypass mode, context binary bits, bypass binary bits, valid coefficient flag, last valid coefficient flag, encoding flag for the unit of the coefficient group, position of the last valid coefficient, flag indicating whether the coefficient value is greater than 1, flag indicating whether the coefficient value is greater than 2, flag indicating whether the coefficient value is greater than 3, information about the remaining coefficient values, symbol information, reconstructed luminance samples, reconstructed chrominance samples, residual luminance samples, residual chrominance samples, luminance transformation coefficient, chrominance transformation coefficient, quantized luminance level, quantized chrominance level, transformation coefficient level scanning method. Information includes: the size and shape of the motion vector search region on the decoder side; the number of motion vector searches on the decoder side; information about the CTU size; information about the minimum block size; information about the maximum block size; information about the maximum block depth; information about the minimum block depth; image display / output order; strip identification information; strip type; strip partition information; parallel block identification information; parallel block type; parallel block partition information; picture type; bit depth of input samples; bit depth of reconstructed samples; bit depth of residual samples; bit depth of transform coefficients; bit depth of quantization levels; and information about the luminance signal or the chrominance signal.
[0146] Here, sending a flag or index with a signal can represent the encoder entropy encoding the corresponding flag or index and including it in the bitstream, and can also represent the decoder entropy decoding the corresponding flag or index from the bitstream.
[0147] When the encoding device 100 performs encoding via inter-frame prediction, the encoded current image can be used as a reference image for another image to be processed subsequently. Therefore, the encoding device 100 can reconstruct or decode the encoded current image, or store the reconstructed or decoded image as a reference image in the reference frame buffer 190.
[0148] The quantization level can be dequantized in dequantization unit 160 or inverse transformed in inverse transform unit 170. The coefficients that have undergone dequantization or inverse transform, or both, can be added to the prediction block by adder 175. A reconstructed block can be generated by adding the coefficients that have undergone dequantization or inverse transform, or both, to the prediction block. Here, the coefficients that have undergone dequantization or inverse transform, or both, can represent coefficients for which at least one of dequantization and inverse transform has been performed, and can represent the reconstructed residual block.
[0149] The reconstructed block can be passed through filter unit 180. Filter unit 180 can apply at least one of deblocking filter, sample adaptive offset (SAO), and adaptive loop filter (ALF) to the reconstructed sample, reconstructed block, or reconstructed image. Filter unit 180 may be referred to as an in-loop filter.
[0150] Deblocking filters remove block distortion generated at the boundaries between blocks. To determine whether to apply a deblocking filter, the number of samples included in several rows or columns within the block can be used. When a deblocking filter is applied to a block, another filter can be applied based on the desired deblocking intensity.
[0151] To compensate for coding errors, a suitable offset value can be added to the sample value using a sample-adaptive offset. The sample-adaptive offset corrects the offset between the deblocked image and the original image on a sample-by-sample basis. This can be achieved by considering edge information about each sample point when applying the offset, or by dividing the image's samples into a predetermined number of regions, determining the regions where the offset will be applied, and then applying the offset to those regions.
[0152] Adaptive loop filters (ALFs) can perform filtering based on a comparison between the filtered reconstructed image and the original image. Samples included in the image can be partitioned into predetermined groups, the filter to be applied to each group can be determined, and differential filtering can be performed on each group. Information regarding whether to apply an ALF can be transmitted via a coding unit (CU), and the form and coefficients of the ALF to be applied to each block can vary.
[0153] The reconstructed blocks or reconstructed image that have passed through filter unit 180 can be stored in reference frame buffer 190. The reconstructed blocks processed by filter unit 180 can be a portion of the reference image. That is, the reference image is a reconstructed image composed of the reconstructed blocks processed by filter unit 180. The stored reference image can be used later for inter-frame prediction or motion compensation.
[0154] Figure 2 This is a block diagram illustrating the configuration of a decoding device according to an embodiment of the present invention.
[0155] Decoding device 200 can be a decoder, video decoding device, or image decoding device.
[0156] Reference Figure 2 The decoding device 200 may include an entropy decoding unit 210, an inverse quantization unit 220, an inverse transform unit 230, an intra-frame prediction unit 240, a motion compensation unit 250, an adder 225, a filter unit 260, and a reference frame buffer 270.
[0157] Decoding device 200 can receive a bitstream output from encoding device 100. Decoding device 200 can receive a bitstream stored in a computer-readable recording medium, or a bitstream streamed via a wired / wireless transmission medium. Decoding device 200 can decode the bitstream using intra-frame mode or inter-frame mode. Furthermore, decoding device 200 can generate a reconstructed image or a decoded image generated through decoding, and output the reconstructed image or decoded image. When the prediction mode used during decoding is intra-frame mode, the switcher can be switched to intra-frame mode. Optionally, when the prediction mode used during decoding is inter-frame mode, the switcher can be switched to inter-frame mode.
[0158] Decoding device 200 obtains a reconstructed residual block and generates a prediction block by decoding the input bitstream. Once the reconstructed residual block and prediction block are obtained, decoding device 200 generates a reconstructed block that becomes the decoding target by adding the reconstructed residual block and the prediction block. The decoding target block can be referred to as the current block.
[0159] The entropy decoding unit 210 can generate symbols by performing entropy decoding on the bitstream according to a probability distribution. The generated symbols may include symbols in quantized hierarchical form. Here, the entropy decoding method can be the inverse process of the entropy encoding method described above.
[0160] In order to decode the transform coefficient levels (quantization levels), the entropy decoding unit 210 can change the coefficients in unidirectional vector form into two-dimensional block form by using a transform coefficient scanning method.
[0161] The quantization level can be dequantized in the dequantization unit 220, or the quantization level can be inversely transformed in the inverse transform unit 230. The quantization level can be the result of dequantization, inverse transform, or both, and can be generated as a reconstructed residual block. Here, the dequantization unit 220 can apply the quantization matrix to the quantization level.
[0162] When using intra-frame mode, intra-frame prediction unit 240 can generate a prediction block by performing spatial prediction on the current block, wherein the spatial prediction uses sample values of blocks that are adjacent to the target block and have already been decoded.
[0163] When using inter-frame mode, motion compensation unit 250 can generate a prediction block by performing motion compensation on the current block, wherein the motion compensation uses motion vectors and a reference image stored in reference frame buffer 270.
[0164] Adder 225 generates a reconstructed block by adding the reconstructed residual block to the prediction block. Filter unit 260 can apply at least one of a deblocking filter, a sample adaptive offset, and an adaptive loop filter to the reconstructed block or reconstructed image. Filter unit 260 can output a reconstructed image. The reconstructed block or reconstructed image can be stored in a reference frame buffer 270 and used when performing inter-frame prediction. The reconstructed block processed by filter unit 260 can be a portion of the reference image. That is, the reference image is a reconstructed image composed of the reconstructed blocks processed by filter unit 260. The stored reference image can be used later during inter-frame prediction or motion compensation.
[0165] Figure 3 It is a schematic diagram illustrating the partitioning structure of an image when it is encoded and decoded. Figure 3 An example of partitioning a single cell into multiple lower-level cells is illustrated.
[0166] To effectively partition an image, coding units (CUs) can be used during encoding and decoding. A coding unit can serve as the basic unit when encoding / decoding an image. Furthermore, a coding unit can be used to distinguish between intra-frame prediction modes and inter-frame prediction modes during image encoding / decoding. A coding unit can be the basic unit used for prediction, transform, quantization, inverse transform, inverse quantization, or encoding / decoding processing of transform coefficients.
[0167] Reference Figure 3 Image 300 is partitioned sequentially according to the Largest Coding Unit (LCU), and the LCU unit is determined as the partitioning structure. Here, LCU can be used with the same meaning as Coding Tree Unit (CTU). Unit partitioning can represent partitioning of the block associated with that unit. The block partitioning information may include information about the unit depth. The depth information may represent the number or degree to which the unit is partitioned, or both the number and degree to which the unit is partitioned. A single unit can be partitioned into multiple lower-level units hierarchically associated with the depth information based on a tree structure. In other words, the unit and the lower-level units generated by partitioning that unit may correspond to a node and the child nodes of that node, respectively. Each of the partitioned lower-level units may have depth information. The depth information may be information representing the size of the CU and may be stored in each CU. The unit depth represents the number and / or degree associated with partitioning the unit. Therefore, the partitioning information of the lower-level units may include information about the size of the lower-level units.
[0168] The partitioning structure represents the distribution of coding units (CUs) within the LCU 310. This distribution can be determined by whether a single CU is partitioned into multiple CUs (positive integers equal to or greater than 2, including 2, 4, 8, 16, etc.). The horizontal and vertical dimensions of the CUs generated by partitioning can be half the horizontal and vertical dimensions of the CUs before partitioning, respectively, or they can have dimensions smaller than the horizontal and vertical dimensions before partitioning, depending on the number of partitions. CUs can be recursively partitioned into multiple CUs. Through recursive partitioning, at least one of the height and width of the CU after partitioning can be reduced compared to at least one of the height and width of the CU before partitioning. CU partitioning can be performed recursively until a predefined depth or a predefined size is reached. For example, the depth of the LCU can be 0, and the depth of the minimum coding unit (SCU) can be a predefined maximum depth. Here, as mentioned above, the LCU can be the coding unit with the maximum coding unit size, and the SCU can be the coding unit with the minimum coding unit size. Partitioning begins at the LCU 310, and the CU depth increases by 1 when the horizontal or vertical dimension of the CU, or both the horizontal and vertical dimensions, decrease through partitioning. For example, for each depth, the size of an unpartitioned CU can be 2N×2N. Furthermore, in the case of partitioned CUs, a CU of size 2N×2N can be partitioned into four CUs of size N×N. As the depth increases by 1, the size of N can be halved.
[0169] Furthermore, partition information of a CU can be used to indicate whether a CU is partitioned. Partition information can be 1 bit. All CUs except SCUs can include partition information. For example, when the partition information value is 1, the CU may not be partitioned; when the partition information value is 2, the CU may be partitioned.
[0170] Reference Figure 3 An LCU with depth 0 can be a 64×64 block. 0 can be the minimum depth. An SCU with depth 3 can be an 8×8 block. 3 can be the maximum depth. CUs with 32×32 blocks and 16×16 blocks can be represented as depth 1 and depth 2, respectively.
[0171] For example, when a single coding unit is partitioned into four coding units, the horizontal and vertical dimensions of the four partitioned coding units can be half the size of the CU before partitioning. In one embodiment, when a 32×32 coding unit is partitioned into four coding units, the size of each of the four partitioned coding units can be 16×16. When a single coding unit is partitioned into four coding units, the coding unit can be said to be partitioned into a quadtree form.
[0172] For example, when a coding unit is partitioned into two sub-coding units, the horizontal or vertical dimension (width or height) of each of the two sub-coding units can be half the horizontal or vertical dimension of the original coding unit. For example, when a coding unit of size 32×32 is vertically partitioned into two sub-coding units, each of the two sub-coding units can have a size of 16×32. For example, when a coding unit of size 8×32 is horizontally partitioned into two sub-coding units, each of the two sub-coding units can have a size of 8×16. When a coding unit is partitioned into two sub-coding units, it can be said that the coding unit is binary partitioned, or partitioned according to a binary tree partitioning structure.
[0173] For example, when a coding unit is divided into three sub-coding units, the horizontal or vertical dimensions of the coding unit can be divided in a 1:2:1 ratio, resulting in three sub-coding units with a horizontal or vertical dimension ratio of 1:2:1. For instance, when a 16×32 coding unit is horizontally divided into three sub-coding units, these three sub-coding units, in order from the top to the bottom, can have dimensions of 16×8, 16×16, and 16×8, respectively. Similarly, when a 32×32 coding unit is vertically divided into three sub-coding units, these three sub-coding units, in order from the left to the right, can have dimensions of 8×32, 16×32, and 8×32, respectively. When a coding unit is divided into three sub-coding units, it can be said that the coding unit is tri-partitioned or partitioned according to a ternary tree partitioning structure.
[0174] exist Figure 3 In the example, the coding tree unit (CTU) 320 is an example of a CTU in which quadtree partitioning, binary tree partitioning, and ternary tree partitioning structures are all applied.
[0175] As described above, to partition the CTU, at least one of a quadtree partitioning structure, a binary tree partitioning structure, and a ternary tree partitioning structure can be applied. Various tree partitioning structures can be applied sequentially to the CTU according to a predetermined priority order. For example, a quadtree partitioning structure can be preferentially applied to the CTU. Encoding units that cannot be further partitioned using a quadtree partitioning structure can correspond to leaf nodes of a quadtree. Encoding units corresponding to leaf nodes of a quadtree can be used as root nodes of binary and / or ternary tree partitioning structures. That is, encoding units corresponding to leaf nodes of a quadtree can be further partitioned according to a binary or ternary tree partitioning structure, or they can be left unpartitioned. Therefore, by preventing the encoded blocks obtained from binary or ternary tree partitioning of encoding units corresponding to leaf nodes of a quadtree from undergoing further quadtree partitioning, block partitioning operations and / or the operation of signaling partitioning information can be effectively performed.
[0176] The fact that a coding unit corresponding to a node in a quadtree is partitioned can be signaled using four-partition information. Four-partition information with a first value (e.g., "1") indicates that the current coding unit is partitioned according to the quadtree partitioning structure. Four-partition information with a second value (e.g., "0") indicates that the current coding unit is not partitioned according to the quadtree partitioning structure. The four-partition information can be a flag with a predetermined length (e.g., one bit).
[0177] There may be no priority between binary tree partitions and ternary tree partitions. That is, the coding unit corresponding to the leaf node of the quadtree can further undergo any partition in either binary tree or ternary tree partitions. Furthermore, the coding unit generated by binary tree partitions or ternary tree partitions may undergo further binary tree partitions or further ternary tree partitions, or it may not be further partitioned.
[0178] A tree structure in which there is no priority between binary tree partitions and ternary tree partitions is called a multi-type tree structure. The coding unit corresponding to the leaf node of a quadtree can be used as the root node of a multi-type tree. At least one of multi-type tree partition indication information, partition direction information, and partition tree information can be used to signal whether to partition the coding unit corresponding to a node in the multi-type tree. To partition the coding unit corresponding to a node in the multi-type tree, the multi-type tree partition indication information, partition direction information, and partition tree information can be signaled sequentially.
[0179] A multi-type tree partitioning indication with a first value (e.g., "1") indicates that the current coding unit will undergo a multi-type tree partition. A multi-type tree partitioning indication with a second value (e.g., "0") indicates that the current coding unit will not undergo a multi-type tree partition.
[0180] When the coding unit corresponding to a node of a multi-type tree is further partitioned according to the multi-type tree partitioning structure, the coding unit may include partitioning direction information. The partitioning direction information may indicate in which direction the current coding unit will be partitioned according to the multi-type tree partitioning. Partitioning direction information with a first value (e.g., "1") may indicate that the current coding unit will be vertically partitioned. Partitioning direction information with a second value (e.g., "0") may indicate that the current coding unit will be horizontally partitioned.
[0181] When the coding unit corresponding to a node of a multi-type tree is further partitioned according to the multi-type tree partitioning structure, the current coding unit may include partitioning tree information. The partitioning tree information may indicate the tree partitioning structure that will be used to partition the nodes of the multi-type tree. Partitioning tree information with a first value (e.g., "1") may indicate that the current coding unit will be partitioned according to a binary tree partitioning structure. Partitioning tree information with a second value (e.g., "0") may indicate that the current coding unit will be partitioned according to a ternary tree partitioning structure.
[0182] Partition indication information, partition tree information, and partition direction information can all be flags with a predetermined length (e.g., one bit).
[0183] At least one of the following—quadtree partitioning indication information, multi-type tree partitioning indication information, partitioning direction information, and partitioning tree information—can be entropy encoded / decoded. To entropy encode / decode those types of information, information about neighboring coding units adjacent to the current coding unit can be used. For example, there is a high probability that the partitioning type (partitioned or unpartitioned, partitioning tree, and / or partitioning direction) of the left-hand neighboring coding unit and / or the upper-hand neighboring coding unit of the current coding unit is similar to the partitioning type of the current coding unit. Therefore, contextual information for entropy encoding / decoding of information about the current coding unit can be derived from the information about neighboring coding units. Information about neighboring coding units may include at least one of the following: quadtree partitioning information, multi-type tree partitioning indication information, partitioning direction information, and partitioning tree information.
[0184] As another example, in binary tree partitioning and ternary tree partitioning, binary tree partitioning can be performed first. That is, the current coding unit can first undergo binary tree partitioning, and then the coding unit corresponding to the leaf node of the binary tree can be set as the root node for ternary tree partitioning. In this case, for the coding unit corresponding to the node of the ternary tree, neither quadtree partitioning nor binary tree partitioning can be performed.
[0185] Encoding units that cannot be partitioned according to quadtree, binary tree, and / or ternary tree partitioning structures become the basic units for encoding, prediction, and / or transformation. In other words, these encoding units cannot be further partitioned for prediction and / or transformation. Therefore, partitioning structure information and partitioning information for dividing encoding units into prediction and / or transformation units may not exist in the bitstream.
[0186] However, when the size of the coding unit (i.e., the basic unit used for partitioning) is larger than the size of the maximum transform block, the coding unit can be partitioned recursively until the size of the coding unit is reduced to be equal to or smaller than the size of the maximum transform block. For example, when the size of the coding unit is 64×64 and the size of the maximum transform block is 32×32, the coding unit can be partitioned into four 32×32 blocks for transformation. For example, when the size of the coding unit is 32×64 and the size of the maximum transform block is 32×32, the coding unit can be partitioned into two 32×32 blocks for transformation. In this case, the partitions of the coding unit for transformation are not sent separately by signal, and the partitions of the coding unit for transformation can be determined by comparing the horizontal or vertical dimensions of the coding unit with the horizontal or vertical dimensions of the maximum transform block. For example, when the horizontal dimension (width) of the coding unit is greater than the horizontal dimension (width) of the maximum transform block, the coding unit can be vertically bisected. For example, when the vertical dimension (length) of the coding unit is greater than the vertical dimension (length) of the maximum transform block, the coding unit can be horizontally bisected.
[0187] Information regarding the maximum and / or minimum size of the coding unit and the maximum and / or minimum size of the transform block can be transmitted or determined at a higher level than the coding unit. This higher level can be, for example, the sequence level, the frame level, the strip level, etc. For example, the minimum size of the coding unit can be determined to be 4×4. For example, the maximum size of the transform block can be determined to be 64×64. For example, the minimum size of the transform block can be determined to be 4×4.
[0188] Information regarding the minimum size of the coding unit corresponding to the leaf node of the quadtree (minimum size of the quadtree) and / or the maximum depth of the multi-type tree from the root node to the leaf node (maximum depth of the multi-type tree) can be transmitted or determined at a higher level of the coding unit. For example, the higher level could be the sequence level, the frame level, the strip level, etc. Information regarding the minimum size of the quadtree and / or the maximum depth of the multi-type tree can be transmitted or determined for each of the intra-frame stripes and inter-frame stripes.
[0189] The difference information between the size of the CTU and the maximum size of the transform block can be signaled or determined at a higher level of the coding unit. For example, the higher level could be the sequence level, the frame level, the stripe level, etc. The maximum size of the coding unit corresponding to each node of the binary tree (hereinafter referred to as the maximum size of the binary tree) can be determined based on the size of the coding tree unit and the difference information. The maximum size of the coding unit corresponding to each node of the ternary tree (hereinafter referred to as the maximum size of the ternary tree) can vary depending on the type of stripe. For example, for an intra-frame stripe, the maximum size of the ternary tree could be 32×32. For example, for an inter-frame stripe, the maximum size of the ternary tree could be 128×128. For example, the minimum size of the coding unit corresponding to each node of the binary tree (hereinafter referred to as the minimum size of the binary tree) and / or the minimum size of the coding unit corresponding to each node of the ternary tree (hereinafter referred to as the minimum size of the ternary tree) can be set as the minimum size of the coding block.
[0190] As another example, the maximum size of a binary tree and / or the maximum size of a ternary tree can be signaled or determined at the stripe level. Alternatively, the minimum size of a binary tree and / or the minimum size of a ternary tree can be signaled or determined at the stripe level.
[0191] Based on the size and depth information of the various blocks mentioned above, the four-partition information, multi-type tree partition indication information, partition tree information and / or partition direction information may or may not be included in the bitstream.
[0192] For example, when the size of the coding unit is no greater than the minimum size of the quadtree, the coding unit does not contain four-partition information. Therefore, the four-partition information can be derived from the second value.
[0193] For example, when the size (horizontal and vertical dimensions) of the coding unit corresponding to a node of a multi-type tree is greater than the maximum size (horizontal and vertical dimensions) of a binary tree and / or the maximum size (horizontal and vertical dimensions) of a ternary tree, the coding unit may not be divided into two or three partitions. Therefore, multi-type tree partitioning indication information can be derived from a second value instead of being sent by signal.
[0194] Optionally, when the size (horizontal and vertical dimensions) of the coding unit corresponding to a node of a multi-type tree is the same as the maximum size (horizontal and vertical dimensions) of a binary tree, and / or twice the maximum size (horizontal and vertical dimensions) of a ternary tree, the coding unit may not be further divided into two or three partitions. Therefore, multi-type tree partitioning indication information can be derived from a second value without signal transmission. This is because when the coding unit is partitioned according to the binary tree partitioning structure and / or the ternary tree partitioning structure, coding units smaller than the minimum size of the binary tree and / or the minimum size of the ternary tree are generated.
[0195] Optionally, when the depth of the coding unit corresponding to a node in the multi-type tree is equal to the maximum depth of the multi-type tree, further binary and / or ternary partitioning of the coding unit is not required. Therefore, multi-type tree partitioning indication information can be derived from the second value without sending a signal.
[0196] Optionally, multi-type tree partitioning indication information may be signaled only if at least one of the vertical binary tree partitioning, horizontal binary tree partitioning, vertical ternary tree partitioning, and horizontal ternary tree partitioning is feasible for the coding unit corresponding to the node of the multi-type tree. Otherwise, it may not be possible to partition the coding unit into two and / or three partitions. Therefore, multi-type tree partitioning indication information may not be signaled, but can be derived from a second value.
[0197] Optionally, partition direction information may be signaled only if both vertical binary tree partitioning and horizontal binary tree partitioning, or both vertical ternary tree partitioning and horizontal ternary tree partitioning, are feasible for the coding units corresponding to the nodes of the multi-type tree. Otherwise, partition direction information may not be signaled; instead, it may be derived from values indicating possible partition directions.
[0198] Optionally, partition tree information may be signaled only if both vertical binary tree partitions and vertical ternary tree partitions, or both horizontal binary tree partitions and horizontal ternary tree partitions, are feasible for the encoded tree corresponding to the nodes of the multi-type tree. Otherwise, partition tree information may not be signaled; instead, it may be derived from values indicating possible partition tree structures.
[0199] Figure 4 This is a diagram illustrating intra-frame prediction processing.
[0200] Figure 4 The arrows from the center to the outside in the image can indicate the prediction direction of the intra-frame prediction mode.
[0201] Intra-frame coding and / or decoding can be performed using reference samples from neighboring blocks of the current block. A neighboring block can be a reconstructed neighboring block. For example, intra-frame coding and / or decoding can be performed using coding parameters or values of reference samples included in the reconstructed neighboring block.
[0202] A prediction block can represent a block generated by performing intra-frame prediction. A prediction block can correspond to at least one of CU, PU, and TU. The cells of a prediction block can have the size of one of CU, PU, and TU. A prediction block can be a square block with dimensions such as 2×2, 4×4, 16×16, 32×32, or 64×64, or a rectangular block with dimensions such as 2×8, 4×8, 2×16, 4×16, and 8×16.
[0203] Intra-prediction can be performed based on the intra-prediction mode for the current block. The number of intra-prediction modes that the current block can have can be a fixed value, or it can be a value determined differently depending on the attributes of the predicted block. For example, the attributes of the predicted block can include the size and shape of the predicted block.
[0204] Regardless of the block size, the number of intra-prediction modes can be fixed at N. Alternatively, the number of intra-prediction modes can be 3, 5, 9, 17, 34, 35, 36, 65, or 67, etc. Optionally, the number of intra-prediction modes can vary depending on the block size or the color component type, or both. For example, the number of intra-prediction modes can vary depending on whether the color component is a luma signal or a chrominance signal. For example, the number of intra-prediction modes can increase as the block size increases. Optionally, the number of intra-prediction modes for the luma component block can be greater than the number of intra-prediction modes for the chrominance component block.
[0205] Intra-frame prediction modes can be non-angular or angular. Non-angular modes can be DC or planar modes, and angular modes can be prediction modes with a specific direction or angle. Intra-frame prediction modes can be represented by at least one of mode number, mode value, mode number, mode angle, and mode direction. The number of intra-frame prediction modes can be greater than or equal to 1, M, including non-angular and angular modes.
[0206] To perform intra-frame prediction on the current block, a step can be performed to determine whether a sample included in a reconstructed neighboring block can be used as a reference sample for the current block. When there are samples that cannot be used as reference samples for the current block, a value obtained by copying or interpolating at least one sample value included in a reconstructed neighboring block, or by performing both copying and interpolation, can be used to replace the unavailable sample value of the sample. Thus, the replaced sample value is used as a reference sample for the current block.
[0207] When performing intra-frame prediction, filters can be applied to at least one of the reference samples and the prediction samples based on the intra-frame prediction mode and the size of the current block.
[0208] In planar mode, when generating the prediction block for the current block, the sample value of the target sample is generated by using a weighted sum of the upper and left reference samples of the current sample, and the upper right and lower left reference samples of the current block, based on the position of the target sample within the prediction block. Furthermore, in DC mode, the average value of the upper and left reference samples of the current block can be used when generating the prediction block. Additionally, in angle mode, the prediction block can be generated using the upper, left, upper right, and / or lower left reference samples of the current block. Interpolation can be performed on real cells to generate the prediction sample values.
[0209] The intra-prediction mode of the current block can be entropy-coded / decoded by predicting the intra-prediction modes of adjacent blocks. When the intra-prediction modes of the current block and its neighboring blocks are the same, information indicating that the intra-prediction modes of the current block and its neighboring blocks are the same can be signaled using predetermined flag information. Furthermore, an indicator of the intra-prediction mode among multiple neighboring blocks that is the same as the intra-prediction mode of the current block can be signaled. When the intra-prediction modes of the current block and its neighboring blocks are different, the intra-prediction mode information of the current block can be entropy-coded / decoded by performing entropy coding / decoding based on the intra-prediction modes of neighboring blocks.
[0210] Figure 5 This is a diagram illustrating an embodiment of inter-screen prediction processing.
[0211] exist Figure 5 In this context, rectangles can represent the image. Figure 5 In the image, the arrow indicates the prediction direction. Based on the encoding type of the frame, frames can be classified into intra-frame frames (I-frames), predictive frames (P-frames), and dual-predictive frames (B-frames).
[0212] I-frames can be encoded via intra-frame prediction without requiring inter-frame prediction. P-frames can be encoded via inter-frame prediction using a reference frame present in one direction (i.e., forward or backward) relative to the current block. B-frames can be encoded via inter-frame prediction using reference frames present in both directions (i.e., forward and backward) relative to the current block. When using inter-frame prediction, the encoder can perform inter-frame prediction or motion compensation, and the decoder can perform the corresponding motion compensation.
[0213] The following section will describe in detail an embodiment of inter-screen prediction.
[0214] Reference frames and motion information can be used to perform inter-frame prediction or motion compensation.
[0215] Motion information of the current block can be derived by each of the encoding device 100 and the decoding device 200 during inter-frame prediction. The motion information of the current block can be derived using motion information of reconstructed neighboring blocks, motion information of co-located blocks (also called col blocks or co-position blocks), and / or motion information of blocks adjacent to the co-position block. A co-position block can represent a block within a previously reconstructed co-located frame (also called a col frame or co-position frame) that is spatially located at the same position as the current block. A co-position frame can be one of one or more reference frames included in a list of reference frames.
[0216] The method for deriving motion information of the current block can be based on changes in the prediction mode of the current block. For example, prediction modes used for inter-frame prediction may include AMVP mode, merge mode, skip mode, and current frame reference mode. The merge mode can be called the motion merge mode.
[0217] For example, when AMVP is used as a prediction mode, at least one of the motion vectors of reconstructed neighboring blocks, co-located blocks, blocks adjacent to co-located blocks, and (0,0) motion vectors can be identified as motion vector candidates for the current block, and a motion vector candidate list is generated using these motion vector candidates. Motion vector candidates for the current block can be derived using the generated motion vector candidate list. Motion information for the current block can be determined based on the derived motion vector candidates. The motion vectors of co-located blocks or blocks adjacent to co-located blocks can be referred to as temporal motion vector candidates, and the motion vectors of reconstructed neighboring blocks can be referred to as spatial motion vector candidates.
[0218] Encoding device 100 can calculate the motion vector difference (MVD) between the motion vector of the current block and motion vector candidates, and can perform entropy encoding on the motion vector difference (MVD). Furthermore, encoding device 100 can perform entropy encoding on the motion vector candidate index and generate a bitstream. The motion vector candidate index indicates the best motion vector candidate among the motion vector candidates included in the motion vector candidate list. Decoding device 200 can perform entropy decoding on the motion vector candidate index included in the bitstream, and can select motion vector candidates for the target block to be decoded from the motion vector candidates included in the motion vector candidate list by using the entropy-decoded motion vector candidate index. Furthermore, decoding device 200 can add the entropy-decoded MVD to the motion vector candidate extracted by entropy decoding to derive the motion vector of the target block.
[0219] The bitstream may include a reference frame index indicating a reference frame. The reference frame index may be entropy encoded by the encoding device 100 and subsequently transmitted as a bitstream to the decoding device 200. The decoding device 200 may generate a predicted block for the decoded target block based on the derived motion vectors and the reference frame index information.
[0220] Another example of a method for deriving motion information for the current block could be a merge pattern. A merge pattern can represent a method for merging the motion of multiple blocks. A merge pattern can represent a pattern for deriving motion information for the current block from motion information of neighboring blocks. When a merge pattern is applied, the motion information of reconstructed neighboring blocks and / or co-located blocks can be used to generate a list of merge candidates. Motion information may include at least one of motion vectors, reference frame indices, and inter-frame prediction indicators. Prediction indicators may indicate unidirectional prediction (L0 prediction or L1 prediction) or bidirectional prediction (L0 prediction and L1 prediction).
[0221] The merge candidate list can be a list of stored motion information. The motion information included in the merge candidate list can be at least one of zero merge candidates and new motion information, wherein the new motion information is a combination of motion information of a neighboring block adjacent to the current block (spatial merge candidate), motion information of a co-located block of the current block included in the reference frame (temporal merge candidate), and motion information existing in the merge candidate list.
[0222] Encoding device 100 can generate a bitstream by performing entropy encoding on at least one of a merge flag and a merge index, and can transmit the bitstream as a signal to decoding device 200. The merge flag may be information indicating whether a merge mode is performed for each block, and the merge index may be information indicating which neighboring block among the current block's neighboring blocks is the target block for merging. For example, neighboring blocks of the current block may include a left-hand neighboring block to the left of the current block, an upper-hand neighboring block positioned above the current block, and a time-adjacent neighboring block.
[0223] Skip mode can be a mode in which motion information of neighboring blocks is applied to the current block as is. When skip mode is applied, encoding device 100 can perform entropy encoding on information about which block's motion information will be used as the current block's motion information to generate a bitstream, and can send the bitstream to decoding device 200 as a signal. Encoding device 100 may not send syntax elements regarding at least one of motion vector difference information, coded block flags, and transform coefficient levels to decoding device 200 as a signal.
[0224] The current frame reference mode can represent the prediction mode in which the previously reconstructed region within the current frame to which the current block belongs was used for prediction. Here, a vector can be used to specify the previously reconstructed region. Information indicating whether the current block will be encoded in the current frame reference mode can be encoded using the current block's reference frame index. A flag or index indicating whether the current block is a block encoded in the current frame reference mode can be signaled, and the flag or index can be derived based on the current block's reference frame index. When the current block is encoded in the current frame reference mode, the current frame can be added to the reference frame list for the current block so that the current frame is located at a fixed position or an arbitrary position in the reference frame list. The fixed position can be, for example, the position indicated by reference frame index 0, or the last position in the list. When the current frame is added to the reference frame list so that the current frame is located at an arbitrary position, a reference frame index indicating the arbitrary position can be signaled.
[0225] Figure 6 This is a diagram illustrating the transformation and quantization processes.
[0226] like Figure 6 As shown, transform and / or quantization are performed on the residual signal to generate a quantized level signal. The residual signal is the difference between the original block and the predicted block (i.e., an intra-frame predicted block or an inter-frame predicted block). The predicted block is generated through intra-frame prediction or inter-frame prediction. The transform can be a first transform, a second transform, or both. The first transform of the residual signal produces transform coefficients, and the second transform of the transform coefficients produces second transform coefficients.
[0227] At least one scheme selected from a variety of predefined transform schemes is used to perform the initial transform. Examples of the predefined transform schemes include Discrete Cosine Transform (DCT), Discrete Sine Transform (DST), and Karhunen-Loève Transform (KLT). The transform coefficients generated by the initial transform may undergo a secondary transform. The transform scheme used for the initial and / or secondary transforms can be determined based on the coding parameters of the current block and / or its neighboring blocks. Alternatively, the transform scheme can be determined via signaling of the transform information.
[0228] Since the residual signal is quantized through the first and second transforms, a quantized level signal (quantization coefficients) is generated. Depending on the intra-frame prediction mode or block size / shape, the quantized level signal can be scanned using at least one of diagonal top-right scan, vertical scan, and horizontal scan. For example, when scanning coefficients according to the diagonal top-right scan, the block-form coefficients become a one-dimensional vector form. In addition to the diagonal top-right scan, horizontal scans of two-dimensional block-form coefficients and vertical scans of two-dimensional block-form coefficients can be used depending on the intra-frame prediction mode and / or size of the transform block. The scanned quantized level coefficients can be entropy-encoded for insertion into the bitstream.
[0229] The decoder performs entropy decoding on the bitstream to obtain quantization level coefficients. These coefficients can be arranged in a two-dimensional block format via inverse scanning. For inverse scanning, at least one of the following methods can be used: diagonal top-right scan, vertical scan, and horizontal scan.
[0230] Then, the quantization level coefficients can be dequantized, followed by a second inverse transform as needed, and finally a first inverse transform as needed to generate the reconstructed residual signal.
[0231] Figure 7 This is a block diagram illustrating the configuration of an image encoding device according to an embodiment of the present disclosure. Figure 7 Show Figure 1 One embodiment.
[0232] The image encoding device 700 includes a screen partitioning unit 710, a transformation unit 720, a quantization unit 730, a scanning unit 731, an entropy coding unit 740, an intra-frame prediction unit 750, an inter-frame prediction unit 760, an inverse quantization unit 735, an inverse transformation unit 725, a post-processing unit 770, a screen storage unit 780, a subtractor 790, and an adder 795.
[0233] Reference Figure 7 The screen partitioning unit 710 analyzes the input video signal, partitions the screen into coding units, determines the prediction mode, and determines the size of the prediction unit for each coding unit.
[0234] The frame partitioning unit 710 feeds the prediction units to be encoded to the intra-frame prediction unit 750 or the inter-frame prediction unit 760 according to the selected prediction mode (or prediction method). The frame partitioning unit 710 feeds the prediction units to be encoded to the subtractor 790.
[0235] An image (i.e., video) consists of multiple stripes, and each stripe can be partitioned into multiple coding tree units (CTUs), where each coding tree unit is the basic unit used for partitioning the image.
[0236] Each coding tree unit can be partitioned into one, two or more coding units (CUs), where each coding unit is a basic unit used for inter-frame prediction or intra-frame prediction.
[0237] The maximum size of the coding tree unit may differ from the maximum size of the coding unit, and information about them may be sent to the decoding device 1300.
[0238] Each coding unit can be partitioned into one or more prediction units (PUs), where each prediction unit is a basic unit used for prediction processing.
[0239] In this case, the coding device 700 selects a prediction method for each coding unit between intra-frame prediction and inter-frame prediction, and can generate prediction blocks differently for each prediction unit.
[0240] Each coding unit (CU) can be partitioned into one, two, or more transform units (TUs), where each transform unit is the basic unit for transforming residual blocks.
[0241] In this case, the image partitioning unit 710 feeds image data to the subtractor 790 based on each block (e.g., based on each prediction unit or based on each transform unit).
[0242] Figures 8 to 11 This is an illustration of an embodiment of the present disclosure for describing a method of partitioning an image into blocks and performing image processing based on each block. Figures 8 to 11 Show Figure 3 One embodiment.
[0243] Reference Figure 8 When a 256×256 coding tree unit (CTU) is partitioned into four square coding units (CUs) of equal size using a quadtree structure.
[0244] Each of the four square coding units (CUs) can be further partitioned using a quadtree structure. Each coding unit (CU) partitioned by the quadtree structure can have a depth of any value in the range of 0 to 3.
[0245] Furthermore, as described below, binary tree structures can be used to further partition coding units that have already been partitioned using quadtree structures. That is, a quadtree block can be partitioned into two binary tree sub-blocks. First, the CTU is partitioned using a quadtree structure. A node can be partitioned up to the minimum quadtree node size minQTSize. When the size of a node in the quadtree structure is not greater than the maximum allowable binary tree node size MaxBTSize, the nodes in the quadtree structure can be further partitioned using a binary tree structure. Additionally, the nodes in the binary tree structure can be partitioned up to the minimum allowable binary tree node size or the maximum allowable binary tree partition depth.
[0246] In other words, the CTU is first partitioned using a quadtree structure, and each node generated by the partitioning can be further recursively partitioned using either a quadtree or a binary tree structure. The binary tree nodes generated by the final partitioning are not further partitioned and are determined as the coding units used as the basic units for prediction and transformation. Here, when the CU is a block generated by binary tree partitioning, asymmetric binary partitioning can be performed. When performing asymmetric binary partitioning, a CU can be divided into two sub-CUs with sizes of 1 / 4 and 3 / 4 of the original coding unit CU, respectively.
[0247] The coding unit (CU) can be partitioned into two or more prediction units based on the prediction mode to be used.
[0248] For intra-frame prediction mode, when a CU has a size of 2N×2N, a PU can have the following characteristics: Figure 9 The 2N×2N dimensions shown in (a), or as in Figure 9 The N×N dimensions are shown in (b).
[0249] On the other hand, in inter-frame prediction mode, when a coding unit (CU) has a size of 2N×2N, a prediction unit (PU) can have any of the following sizes: Figure 10 The 2N×2N dimensions shown in (a) Figure 10 The 2N×N dimensions shown in (b) Figure 10 The N×2N dimensions shown in (c) Figure 10 The N×N dimensions shown in (d) Figure 10 The dimensions of 2N×nU shown in (e) Figure 10 The dimensions of 2N×nD shown in (f) Figure 10 The dimensions of nL×2N shown in (g) and Figure 10 The dimensions of nR×2N are shown in (h).
[0250] Reference Figure 11A quadtree structure is used to partition the coding unit (CU) to produce four square transform units (TU).
[0251] Each of the four square transformation units TU can be further partitioned using a quadtree structure. Each of the four transformation units can have a partition depth of any value from 0 to 3.
[0252] When using inter-frame prediction mode to predict coding units (CUs), prediction units (PUs) and transform units (TUs) can independently have different partitioning structures.
[0253] When using intra-frame prediction mode to predict coding units (CUs), the size of the transform unit (TU) obtained by partitioning the coding unit (CU) will not be greater than the size of the prediction unit (PU).
[0254] The transformation unit (TU) can have a maximum size of 64×64 pixels.
[0255] Transformation unit 720 can transform residual blocks, wherein the residual blocks are residual signals representing the difference between the original signal input to prediction unit PU and the prediction signal generated by intra-prediction unit 750 or inter-prediction unit 760. Transformation can be performed on a per-TU basis.
[0256] The transformation matrix used in the transform process can be determined based on whether the mode is intra-frame or inter-frame. Since the residual signal generated by intra-frame prediction has directionality according to the intra-frame prediction mode, the transformation matrix can be adaptively determined.
[0257] A transformation unit can be transformed into two (horizontal and vertical) one-dimensional transformation matrices. For example, a predetermined transformation matrix can be used for inter-frame prediction.
[0258] On the other hand, in the case of intra-prediction mode, when the horizontal mode is selected as the intra-prediction mode, since the residual block is likely to have vertical orientation, a DCT-based integer matrix is used for the vertical direction and a KLT-based integer matrix is used for the horizontal direction. When the intra-prediction mode is the vertical orientation mode, both the DST-based and KLT-based integer matrices are used for the vertical direction, and the DCT-based integer matrix is used for the horizontal direction.
[0259] In DC mode, DCT-based integer matrices are applied to both the horizontal and vertical directions.
[0260] In the case of intra-frame prediction, the transformation matrix can be adaptively determined based on the size of the transform unit (TU).
[0261] On the other hand, the transform unit 720 can select the transform kernel type according to coding efficiency. The transform unit 720 can define various types of basis functions for inter-frame prediction mode and intra-frame prediction mode, and selectively use one of the basis functions.
[0262] Transform unit 720 can define various types of basis functions for inter-frame prediction modes and intra-frame prediction modes, and selectively use one of the basis functions. Transform unit 720 may be equipped with a mapping table for supplementary transformations, thereby enabling the performance of supplementary transformations. For example, transform unit 720 can perform mode-dependent inseparable quadratic transform (MDNSST) processing.
[0263] Quantization unit 730 determines a quantization step size, wherein the quantization step size is used to quantize the coefficients obtained by transforming the residual block using a transformation matrix, wherein the quantization step size can be determined for each quantization unit having a predetermined size or a larger size.
[0264] The size of the quantization unit can be 8×8 or 16×16. The quantization unit 730 can quantize the coefficients of the transform block using a quantization matrix determined based on the quantization step size and the prediction mode.
[0265] The quantization unit 730 can use the quantization step size of the quantization unit adjacent to the current quantization unit as a quantization step size prediction factor.
[0266] Quantization unit 730 can scan the neighboring quantization units of the current quantization unit in the order of left quantization unit, top quantization unit, and top left quantization unit, and use one or two effective quantization steps to generate the quantization step prediction factor of the current quantization unit.
[0267] For example, quantization unit 730 can determine the first quantization step that appears first in the scan order among the found effective quantization steps, or the average of two effective quantization steps found through scanning, as the quantization step predictor. When only one quantization step is effective, this effective quantization step can be determined as the quantization step predictor.
[0268] When the quantization step size predictor is determined, the quantization unit 730 feeds the difference between the quantization step size of the current quantization unit and the quantization step size predictor to the entropy coding unit 740.
[0269] On the other hand, there may be cases where the left-hand coding unit, the top coding unit, and the top-left coding unit of the current coding unit are all absent, or there may be cases where the previous coding units in the coding order are absent in the largest coding unit.
[0270] Therefore, the quantization step size of the previous quantization unit immediately preceding the current quantization unit in the coding order within the largest coding unit, or the quantization step size of the neighboring coding unit adjacent to the current quantization unit, can be used as a quantization step size candidate.
[0271] In this case, the priority for selecting the quantization step size predictor can be in the following order: 1) the coding unit to the left of the current coding unit, 2) the coding unit above the current coding unit, 3) the coding unit to the upper left of the current coding unit, and 4) the coding units preceding the current coding unit in the coding order. This order can be changed. Optionally, the coding unit to the upper left can be ignored.
[0272] The quantized transform block can be fed to the dequantization unit 735 and the scanning unit 731.
[0273] Scanning unit 731 scans the quantized coefficients of the transform block and transforms the coefficients into a one-dimensional quantized coefficient set. In this case, since the distribution of the coefficients of the transform block can depend on the intra-prediction mode, the scanning method can be selected according to the intra-prediction mode.
[0274] The coefficient scanning method can vary depending on the size of the transform unit. The scanning mode can vary depending on the intra-frame prediction mode. In this case, the scanning order of the coefficients can be reversed.
[0275] When quantized coefficients are divided into multiple subsets, the same scan pattern can be applied to the quantized coefficients within each subset. Alternatively, a scan pattern, a zigzag scan pattern, or a diagonal scan pattern can be used to scan subsets.
[0276] The scanning pattern can preferably be determined in such a way that the remaining subsets are scanned sequentially from the principal subset including DC. However, the reverse order is also possible.
[0277] The scanning mode used to scan subsets can be the same as the scanning mode used to scan the quantized coefficients within each subset. The scanning mode used to scan subsets can be determined based on the intra-frame prediction mode.
[0278] On the other hand, the encoding device 700 can insert information indicating the position of the last non-zero quantization coefficient in the transform unit TU and the position of the last non-zero quantization coefficient in each subset into the bit stream to send to the decoding device 1300.
[0279] As described above, the dequantization unit 735 can dequantize the quantized coefficients. The inverse transform unit 725 performs an inverse transform based on each transform unit (TU), thereby restoring the dequantized transform coefficients to residual blocks in the spatial domain.
[0280] Adder 795 can generate a reconstructed block by adding the residual block reconstructed by inverse transform unit 725 to the prediction block received from intra-prediction unit 750 or inter-prediction unit 760.
[0281] In addition, the post-processing unit 770 can perform deblocking filtering to eliminate blockiness in the reconstructed image, pixel adaptive offset (SAO) application processing to compensate for the difference between the original image and the reconstructed image based on each pixel, and adaptive loop filtering (ALF) to compensate for the difference between the original image and the reconstructed image based on each coding unit.
[0282] Deblocking filtering can be applied to the boundaries of prediction units (PUs) or transformation units (TUs) with a size equal to or greater than a predetermined size.
[0283] For example, deblocking filtering may include: determining the boundary to be filtered; determining the boundary filter intensity to be applied to the boundary; determining whether to use a deblocking filter; and when it is determined that a deblocking filter should be used, selecting the filter to be applied to the boundary.
[0284] Whether to apply deblocking filtering depends on: i) whether the boundary filtering strength is greater than zero, and ii) whether the value indicating the change in pixel value at the boundary between two blocks (P block and Q block) adjacent to the filtered boundary is less than a first reference value determined by the quantization parameter.
[0285] Preferably, two or more filters can be used. If the absolute value of the difference between the values of two pixels located at the block boundary is greater than or equal to the second reference value, then a filter that performs relatively weak filtering is selected.
[0286] The second reference value is determined by the quantization parameter and the boundary filter strength.
[0287] Sample Adaptive Offset (SAO) is a process that reduces the distortion between pixels in the deblocked image and pixels in the original image. The decision to apply SAO is made on a per-frame or per-strip basis.
[0288] The image or strip can be divided into multiple offset regions, and an offset type can be determined for each offset region. The offset types can include a predetermined number of edge offset types (e.g., four edge offset types) and two strip offset types.
[0289] For example, when the offset type is one of the edge offset types, the edge type is determined for each pixel, and the offset corresponding to the edge type is applied to each pixel. The edge type is determined by comparing the current pixel with its two neighboring pixels.
[0290] Adaptive loop filtering can be performed based on values obtained from comparing the original image with the reconstructed image to which deblocking filtering or adaptive offset has been applied.
[0291] The image storage unit 780 receives post-processed image data from the post-processing unit 770, reconstructs an image based on each image, and stores the reconstructed image. An image is an image corresponding to a frame or a field.
[0292] The inter-frame prediction unit 760 uses one or more frames stored in the frame storage unit 780 to perform motion estimation and determines one or more reference frame indices and one or more motion vectors that specify one or more reference frames.
[0293] In this case, a prediction block (hereinafter referred to as a prediction block) corresponding to the prediction unit to be encoded can be extracted from the reference frames for motion estimation stored in the frame storage unit 780 based on one or more determined reference frame indices and one or more determined motion vectors.
[0294] Intra-prediction unit 750 uses the reconstructed pixel values within the frame, including the current prediction unit, to perform intra-prediction coding.
[0295] Intra-prediction unit 750 receives current prediction unit (PU) for prediction coding, selects an intra-prediction mode from a predetermined number of intra-prediction modes according to the size of the current block, and performs intra-prediction on the current PU using the selected intra-prediction mode.
[0296] The intra-prediction unit 750 can adaptively filter reference pixels to generate intra-prediction blocks. When reference pixels are unavailable, the intra-prediction unit 750 can use available pixels to generate reference pixels.
[0297] The entropy coding unit 740 performs entropy coding on the quantization coefficients quantized by the quantization unit 730, the intra-prediction information received from the intra-prediction unit 750, and the motion information received from the inter-prediction unit 760.
[0298] Figure 12 This is a block diagram illustrating a method for performing inter-frame prediction in an image coding apparatus according to an embodiment of the present disclosure. The illustrated inter-frame prediction encoder includes a motion information determination unit 1261, a motion information coding mode determination unit 1262, a motion information coding unit 1263, a prediction block generation unit 1264, a residual block generation unit 1265, a residual block coding unit 1266, and a multiplexer 1267.
[0299] Reference Figure 12The motion information determination unit 1261 determines the motion information of the current block. The motion information includes a reference frame index and a motion vector. The reference frame index indicates the previously encoded and reconstructed frame.
[0300] When the current block is encoded using one-way inter-frame prediction, the reference frame index of a reference frame in reference frame 0 (L0) is included. When the current block is encoded using two-way prediction, the reference frame index of a reference frame in reference frame 0 (L0) and the reference frame index of a reference frame in reference frame 1 (L1) may be included.
[0301] Furthermore, when encoding the current block via bidirectional prediction, the index of one or both reference frames in the composite list LC generated by combining list 0 and list 1 may be included.
[0302] Motion vectors indicate the position of the predicted block within the frame, as indicated by each reference frame index, and can be represented on a per-pixel basis (based on each integer pixel) or on a per-subpixel basis.
[0303] For example, motion vectors can have sub-pixel resolution, such as half a pixel, a quarter pixel, an eighth pixel, or a sixteenth pixel. When the motion vector does not have integer pixel resolution, prediction blocks can be generated based on each integer pixel.
[0304] The motion information encoding mode determination unit 1262 determines one of the skip mode, merge mode and AMVP mode as the encoding mode for the motion information of the current block.
[0305] Skip mode is used when there is a skip candidate with the same motion information as the current block and the residual signal is zero. Skip mode is also used when the current block, which is the prediction unit (PU), has the same size as the coding unit (CU).
[0306] The merge mode is used when a merge candidate with the same motion information as the current block exists. The merge mode is also used when the size of the current block differs from the size of the coding unit (CU). Furthermore, the merge candidate and the skip candidate can be the same.
[0307] When skip and merge modes are not applicable, the AMVP mode is used. The AMVP candidate with the motion vector most similar to the current block's motion vector is selected as the AMVP predictor.
[0308] The motion information encoding unit 1263 encodes the motion information according to the method determined by the motion information encoding mode determination unit 1262.
[0309] For example, the motion information coding unit 1263 performs merged motion vector coding processing when the motion information coding mode is skip mode or merge mode, and performs AMVP coding processing when the motion information coding mode is AMVP mode.
[0310] The prediction block generation unit 1264 uses the motion information of the current block to generate a prediction block. When the motion vector has an integer pixel resolution, the prediction block generation unit 1264 copies the block corresponding to the position indicated by the motion vector in the frame indicated by the reference frame index as the prediction block of the current block.
[0311] On the other hand, when the motion vector does not have an integer pixel resolution, the prediction block generation unit 1264 generates pixels for the prediction block from integer pixels in the frame indicated by the reference frame index.
[0312] In this case, an 8-tap interpolation filter for luminance pixels is used to generate the predicted pixels, and a 4-tap interpolation filter for chrominance pixels is used to generate the predicted pixels.
[0313] The residual block generation unit 1265 generates residual blocks from the current block and the prediction block of the current block. When the current block has a size of 2N×2N, the residual block generation unit 1265 uses the current block and the corresponding 2N×2N prediction block to generate residual blocks.
[0314] On the other hand, when the current block used for prediction has a size of 2N×N or N×2N, two prediction blocks are obtained for each of the 2N×N blocks that constitute a 2N×2N block, and the two 2N×N prediction blocks are used to generate a final 2N×2N prediction block.
[0315] Furthermore, 2N×2N prediction blocks can be used to generate 2N×2N residual blocks. Overlap smoothing can be applied to boundary pixels to eliminate discontinuities at the boundaries between two 2N×N prediction blocks.
[0316] The residual block coding unit 1266 can divide the residual block into one or more transform units (TUs), and can perform transform coding, quantization and entropy coding on each transform unit (TU).
[0317] The residual block coding unit 1266 can transform the residual blocks generated by inter-frame prediction using an integer-based transformation matrix, and the transformation matrix can be an integer-based DCT matrix.
[0318] On the other hand, the residual block coding unit 1266 uses a quantization matrix to quantize the coefficients of the residual block transformed by a transformation matrix. The quantization matrix can be determined based on the quantization parameters.
[0319] Quantization parameters are determined for each coding unit (CU) with a size equal to or greater than a predetermined size. When the current coding unit (CU) is smaller than the predetermined size, only the quantization parameters of the first coding unit that appears first in the coding order among the coding units with a size equal to or greater than the predetermined size are encoded, and since the quantization parameters of the remaining coding units (CU) are the same as those of the first coding unit (CU), the quantization parameters of the remaining coding units (CU) are not encoded.
[0320] In addition, the coefficients of the transform block can be quantized using a quantization matrix determined based on quantization parameters and prediction mode.
[0321] Predictive coding can be performed on the quantization parameters determined for each coding unit (CU) with a size equal to or greater than a predetermined size using the quantization parameters of the coding unit (CU) adjacent to the current coding unit (CU).
[0322] One or two valid quantization parameters are retrieved by searching the left and top coding units (CUs) in the order of left and top coding units (CUs), and the retrieved one or two valid quantization parameters are used to generate a quantization parameter predictor for the current coding unit (CU).
[0323] For example, the first valid quantization parameter retrieved in the order described above can be determined as the quantization parameter predictor for the current block. Alternatively, the first valid quantization parameter can be determined as the quantization parameter predictor by searching the left-hand coding unit (CU) and the previous coding unit (CU) of the current coding unit in the order of coding order.
[0324] The coefficients of the quantized transform block are scanned and transformed into a one-dimensional quantized coefficient set, and the scanning mode can be set differently according to the entropy coding mode.
[0325] For example, when encoding via CABAC, the quantization coefficients of the inter-frame predictive coding can be scanned in a predetermined manner (i.e., using zigzag or diagonal raster scanning). When encoding via CAVLC, the scanning can be performed in a different manner than described above.
[0326] For example, a zigzag scan can be used when performing a scan for inter-frame predictive coding. On the other hand, when performing a scan for intra-frame predictive coding, the scan method can be determined based on the intra-frame predictive mode to be used. The coefficient scan method can be determined based on the size of the transform unit.
[0327] The scanning mode can vary depending on the selected mode in the direction prediction mode, and the quantization coefficients can be scanned in the opposite direction.
[0328] The multiplexer 1267 multiplexes the motion information encoded by the motion information encoding unit 1263 and the residual signal encoded by the residual block encoding unit 1266.
[0329] For example, in skip or merge modes, motion information includes only indices indicating predictors. In AMVP mode, motion information may include the reference frame index of the current block, the difference motion vector, and the AMVP index.
[0330] The following text will describe it in detail. Figure 7 An embodiment of the operation of the intra-frame prediction unit 750 is shown in the figure.
[0331] The intra-prediction unit 750 receives the size and prediction mode information of the prediction unit PU from the frame partitioning unit 710, and reads reference pixels from the frame storage unit 780 to determine the intra-prediction mode of the prediction unit (PU).
[0332] Intra-prediction unit 750 determines whether to create a reference pixel by checking for the presence of unavailable pixels, and the reference pixel can be used to determine one of multiple intra-prediction modes for the current block.
[0333] When the current block is located at the top edge of the current screen, the pixels adjacent to the top edge of the current block are undefined. When the current block is located at the left edge of the current screen, the pixels adjacent to the left edge of the current block are also undefined. These undefined pixels are considered unusable pixels.
[0334] Furthermore, pixels that are not pre-encoded and reconstructed are also considered unusable when the current block is located at a strip boundary and therefore the pixels adjacent to the upper or left boundary of the strip are not pre-encoded and reconstructed pixels.
[0335] As described above, when there are no pixels to the left or above the current block, or when there are no previously encoded and reconstructed pixels, the intra-prediction mode for the current block is determined based solely on the available pixels.
[0336] On the other hand, these reference pixels at unavailable pixel locations can be generated by copying pixel values from available reference pixels in the current block. For example, when pixels in the block above the current block are unavailable, some or all of the pixels in the block to the left of the current block can be used to generate pixels in the block above, and vice versa.
[0337] In other words, reference pixels can be generated by copying the pixel value from the nearest available reference pixel to an unavailable pixel location in a predetermined direction. Conversely, when there is no available reference pixel adjacent to the current block in the predetermined direction, the pixel value of the nearest available reference pixel in the opposite direction can be copied to generate a reference pixel.
[0338] Although a pixel exists in the block above or to the left of the current block, it is possible that, depending on the encoding mode of the block above or to the left, the pixel in the block above or to the left cannot be used as a usable reference pixel.
[0339] For example, when a reference pixel adjacent to the upper boundary of the current block belongs to a block that was previously predicted and coded by inter-frame prediction and subsequently reconstructed, the reference pixel is determined to be an unusable pixel.
[0340] In this case, usable reference pixels can be generated using pixels belonging to neighboring blocks that were previously intra-predictively coded and reconstructed adjacent to the current block. The encoding device 700 can send information indicating the fact that usable reference pixels have been determined according to the encoding mode to the decoding device 1300.
[0341] The intra-prediction unit 750 determines the intra-prediction mode for the current block using reference pixels. The number of intra-prediction modes that can be applied to the current block is determined based on the size of the current block.
[0342] For example, when the current block has a size of 8×8, 16×16, or 32×32, 34 intra-prediction modes are available. On the other hand, when the current block has a size of 4×4, 17 intra-prediction modes are available. Furthermore, the 34 intra-prediction modes can be further subdivided into 67 intra-prediction modes based on the block size.
[0343] The 67, 34, or 17 intra-frame prediction modes include at least one non-directional mode and multiple directional modes.
[0344] The at least one non-directional mode includes a DC mode and / or a planar mode. When the DC mode and the planar mode are classified as non-directional modes, there may be 67 or 35 intra-prediction modes for intra-prediction of the current block, regardless of the size of the current block.
[0345] In this case, two non-directional modes (DC mode and planar mode) and 64 or 33 directional modes can be used for prediction of the current block.
[0346] When using planar mode, the value of at least one pixel (or the predicted value of a pixel, referred to below as the first reference value) and a reference pixel are used to generate a predicted block for the current block, wherein the at least one pixel is located at the lower right corner of the current block.
[0347] Specifically, in order to utilize an increased number of directional patterns (i.e., 65 directional patterns), the intra-prediction unit 750 can use a Most Probable Mode (MPM) mechanism as the intra-pattern encoding / decoding method. In this case, a predetermined number (e.g., N) of the Most Probable Modes (MPMs) can be selected, and the optimal intra-directional pattern can be determined from the MPMs. Here, N can be an integer of 1 or greater (i.e., any one of 1, 2, 3, 4, 5, ...). The description given below relates to an embodiment of the invention in which six MPMs are extracted.
[0348] Intra-prediction unit 750 can construct an MPM list by obtaining the intra-mode of neighboring blocks corresponding to the candidate position, deriving the intra-mode of the current block, or using the basic intra-mode.
[0349] The initial MPM list may include five intra-modes from neighboring blocks, a planar mode, and a DC mode. The intra-prediction unit 750 constructs an MPM list containing unique modes by eliminating redundancy in the modes according to sequential processing operations. Therefore, the initial intra-modes are arranged in the order of left, top, planar, DC, bottom left, top right, and top left. When there is any space in the 6-mode MPM list, derived modes can be added to fill that space. For example, a mode to be added to the MPM list can be derived by adding -1 or +1 to the mode number of the directional modes included in the MPM list.
[0350] If the MPM list still has space even after performing the above processing, basic patterns can be added to the list in the following order: Basic patterns: Vertical, Horizontal, Pattern 2, and Diagonal Pattern. Through all these processes, a list with six unique patterns can be constructed.
[0351] As can be seen from the above... Figures 7 to 12 The configuration derivation of the image encoding device 700 described herein is based on the configuration of an image decoding device according to an embodiment of the present invention. This can be achieved by performing the above references in reverse order. Figures 7 to 12 The image encoding method described is used to decode the image.
[0352] Figure 13 This is a block diagram illustrating the configuration of a motion picture decoding device according to an embodiment of the present invention. The decoding device 1300 includes an entropy decoding unit 1310, an inverse quantization / inverse transform unit 1320, an adder 1370, a deblocking filter 1350, a picture storage unit 1360, an intra-frame prediction unit 1330, a motion compensation prediction unit 1340, and an intra-frame / inter-frame switching switch 1380. Figure 13 Show Figure 2 One embodiment.
[0353] The entropy decoding unit 1310 receives the encoded bitstream from the image encoding device 80, decodes the bitstream and parses it into intra-frame prediction mode index, motion information, quantization coefficient sequence, etc., and feeds the decoded motion information to the motion compensation prediction unit 1340.
[0354] The entropy decoding unit 1310 transmits the intra-prediction mode index to the intra-prediction unit 1330 and the dequantization / inverse transform unit 1320, so that the dequantization coefficient sequence can be transmitted to the dequantization / inverse transform unit 1320.
[0355] The dequantization / inverse transform unit 1320 uses any one of a plurality of scan modes to transform the quantization coefficient sequence into a two-dimensional set of dequantized coefficients. For example, the selection of the scan mode is performed based on the prediction mode of the current block (intra-prediction or inter-prediction) and the intra-prediction mode when the prediction mode is intra-prediction.
[0356] The dequantization / inverse transform unit 1320 reconstructs quantization coefficients by applying a quantization matrix selected from multiple quantization matrices to a two-dimensional set of dequantization coefficients.
[0357] Different quantization matrices can be applied depending on the size of the current block to be reconstructed. For blocks of the same size, different quantization matrices can be used based on at least one of the current block's prediction mode and its intra-frame prediction mode.
[0358] The dequantization / inverse transform unit 1320 can perform dequantization on the reconstructed quantization coefficients to generate residual blocks. Dequantization is performed on a per-transform unit (TU) basis.
[0359] Adder 1370 reconstructs image blocks by adding the residual blocks reconstructed by inverse quantization / inverse transform unit 1320 to the prediction blocks generated by motion compensation prediction unit 1340.
[0360] The deblocking filter 1350 performs deblocking filtering on the reconstructed image generated by the adder 1370 to reduce deblocking artifacts caused by image loss that occurs during quantization processing.
[0361] The image storage unit 1360 is a frame memory for storing partially decoded images, wherein the partially decoded images are obtained by performing deblocking filtering processing by a deblocking filter.
[0362] Intra-prediction unit 1330 reconstructs the intra-prediction mode of the current block based on the intra-prediction mode index received from entropy decoding unit 1310, and generates a prediction block based on the reconstructed intra-prediction mode.
[0363] The motion compensation prediction unit 1340 generates a prediction block corresponding to the current block from the image stored in the image storage unit 1360. The motion compensation prediction unit 1340 can generate the prediction block by using a selected interpolation filter when applying fractional precision motion compensation.
[0364] The intra / inter frame switching switch 1380 can provide the adder 1370 with prediction blocks generated by the intra prediction unit 1330 or the motion compensation prediction unit 1340 according to the coding mode.
[0365] Figure 14 This is a block diagram for describing inter-frame prediction processing performed in an image decoding device according to an embodiment of the present disclosure.
[0366] The inter-frame prediction decoder includes a demultiplexer 1441, a motion information coding mode determination unit 1442, a merged mode motion information decoding unit 1443, an AMVP mode motion information decoding unit 1444, a prediction block generation unit 1445, a residual block decoding unit 1446, and a reconstruction block generation unit 1447. Here, the merged mode motion information decoding unit 1443 and the AMVP mode motion information decoding unit 1444 may be included in the motion information decoding unit (not shown).
[0367] Reference Figure 14 The demultiplexer 1441 demultiplexes the received bitstream to recover the currently encoded motion information and encoded residual signal included in the received bitstream. The demultiplexer 1441 sends the recovered motion information to the motion information encoding mode determination unit 1442 and sends the recovered residual signal to the residual block decoding unit 1446.
[0368] The motion information encoding mode determination unit 1442 determines the motion information encoding mode of the current block. When the skip flag skip_flag in the received bit stream has a value of 1, the motion information encoding mode determination unit 1442 can determine that the motion information encoding mode of the current block is encoded in skip mode.
[0369] When the skip flag in the received bitstream has a value of 0 and the motion information received from the demultiplexer 1441 only has a merge index, the motion information encoding mode determination unit 1442 can determine that the motion information encoding mode of the current block is encoded in merge mode.
[0370] Furthermore, when the skip flag in the received bitstream has a value of 0 and the motion information received from the demultiplexer 1441 has a reference frame index, a differential motion vector, and an AMVP index, the motion information encoding mode determination unit 1442 can determine that the motion information encoding mode of the current block is encoded in AMVP mode.
[0371] When the motion information encoding mode determination unit 1442 determines that the motion information encoding mode of the current block is skip mode or merge mode, the merge mode motion information decoding unit 1443 is activated. When the motion information encoding mode determination unit 1442 determines that the motion information encoding mode of the current block is AMVP mode, the AMVP mode motion information decoding unit 1444 can be activated.
[0372] The prediction block generation unit 1345 uses the motion information reconstructed by the merged mode motion information decoding unit 1443 or the AMVP mode motion information decoding unit 1444 to generate the prediction block for the current block.
[0373] When the motion vector has an integer pixel resolution, the block corresponding to the position indicated by the motion vector in the picture indicated by the reference picture index is copied to generate the prediction block of the current block.
[0374] On the other hand, when the motion vector does not have an integer pixel resolution, pixels for the prediction block (hereinafter referred to as prediction pixels) are generated from integer pixels in the frame indicated by the reference frame index. In this case, an eight-tap interpolation filter for luminance pixels and a four-tap interpolation filter for chrominance pixels are used to generate the prediction pixels.
[0375] The residual block decoding unit 1446 generates a two-dimensional quantization coefficient block by performing entropy decoding on the residual signal and inverse scanning the coefficients after entropy decoding. The inverse scanning method can be based on variations of the entropy decoding method.
[0376] For example, the diagonal raster inverse scanning method can be used for CABAC-based decoding, and the zigzag inverse scanning method can be used for CAVLC-based decoding. Furthermore, the inverse scanning method can be determined based on the size of the prediction block.
[0377] The residual block decoding unit 1446 dequantizes the generated coefficient block using the dequantization matrix. To derive the quantization matrix, the quantization parameters are reconstructed. The quantization step size is reconstructed for each coding unit with a predetermined size or larger.
[0378] The residual block decoding unit 1446 reconstructs the residual block by performing an inverse transform on the dequantized coefficient block.
[0379] The reconstruction block generation unit 1447 generates a reconstruction block by adding the prediction block generated by the prediction block generation unit 1445 to the residual block generated by the residual block decoding unit 1446.
[0380] In the following text, reference will be made to Figure 13 The process of reconstructing the current block by intra-frame prediction according to an embodiment is described.
[0381] First, the intra-prediction mode of the current block is obtained by decoding the received bitstream. To do this, the entropy decoding unit 1310 refers to one of a plurality of intra-prediction mode tables to reconstruct the first intra-prediction mode index of the current block.
[0382] Multiple intra-prediction mode tables are shared by the encoding device 700 and the decoding device 1300. One of the intra-prediction mode tables can be selected from the intra-prediction mode tables based on the distribution of intra-prediction modes of multiple blocks adjacent to the current block.
[0383] For example, when the intra prediction mode of the block to the left of the current block is the same as the intra prediction mode of the block above the current block, the first intra prediction mode table is used to reconstruct the first intra prediction mode index of the current block. Conversely, when the intra prediction mode of the block to the left of the current block is different from the intra prediction mode of the block above the current block, the first intra prediction mode index of the current block can be reconstructed using the second intra prediction mode table.
[0384] As another example, when the intra-prediction modes of the upper and left blocks of the current block are both directional intra-prediction modes, and when the angle between the orientations of the intra-prediction modes of the upper and left blocks is within a predetermined angle range, a first intra-prediction mode table can be used to reconstruct the first intra-prediction mode index of the current block. On the other hand, when the angle is outside the predetermined angle range, a second intra-prediction mode table can be used to reconstruct the first intra-prediction mode index of the current block.
[0385] Entropy decoding unit 1310 sends the first intra-prediction mode index of the reconstructed current block to intra-prediction unit 1330.
[0386] When the index has a minimum value (i.e., when the index is 0), the intra-prediction unit 1330 that receives the first intra-prediction mode index can determine the maximum permissible mode of the current block as the intra-prediction mode of the current block.
[0387] On the other hand, when the index has a non-zero value, the intra-prediction unit 1330 compares the index indicated by the maximum allowable mode of the current block with the first intra-prediction mode index. If the first intra-prediction mode index is not less than the index indicated by the maximum allowable mode of the current block, the intra-prediction mode corresponding to the second intra-prediction mode index obtained by adding 1 to the first intra-prediction mode index is determined as the intra-prediction mode of the current block. If the first intra-prediction mode index is less than the index indicated by the maximum allowable mode of the current block, the intra-prediction unit 1330 may determine the intra-prediction mode corresponding to the first intra-prediction mode index as the intra-prediction mode of the current block.
[0388] The set of permissible intra-prediction modes for the current block may include at least one non-directional mode and multiple directional modes.
[0389] Examples of the at least one non-directional mode can be a DC mode and / or a planar mode. Furthermore, the DC mode or planar mode can be adaptively included in the set of permissible intra-frame prediction modes.
[0390] Therefore, information specifying non-directional modes included in the set of permissible intra-frame prediction modes can be included in the frame header or strip header.
[0391] Additionally, when an MPM flag is present, the intra-prediction unit 1330 can derive MPM candidate modes from neighboring blocks adjacent to the target block and generate an MPM list using the derived MPM candidate modes. In this case, the intra-prediction unit 1330 can add the MPM candidates derived from the neighboring blocks to the MPM list. The intra-prediction unit 1330 uses the generated MPM list to encode the intra-prediction mode.
[0392] Specifically, in embodiments of the present invention, the intra-prediction unit 1330 may restrict the permissible intra-prediction modes of the current block based on at least one of the coefficient distribution of the inverse-transformed block, the intra-prediction modes of neighboring blocks, and the size / shape of the current block, and may restrict prediction mode candidates when generating the MPM candidate list. Here, the restriction process may mean that only a predetermined intra-prediction mode is allowed or the predetermined intra-prediction mode is not allowed.
[0393] For example, as an allowable intra-prediction mode for the current block, only predetermined intra-prediction modes may be allowed based on information obtained from at least one of the coefficient distribution of the inverse-transformed block, the intra-prediction modes of neighboring blocks, the size of the current block, and the shape of the current block. Alternatively, for example, only predetermined intra-prediction modes may be allowed to generate an MPM candidate mode list based on information obtained from at least one of the coefficient distribution of the inverse-transformed block, the intra-prediction modes of neighboring blocks, the size of the current block, and the shape of the current block.
[0394] Furthermore, in embodiments of the present invention, the intra-prediction unit 1330 may adaptively perform processing to restrict the permissible intra-prediction modes of the current block based on at least one of the coefficient distribution of the inversely transformed block, the intra-prediction modes of neighboring blocks, and the size and shape of the current block, and may process the prediction mode candidates in an adaptive direction when generating the MPM candidate mode list. Here, the adaptive direction processing may mean performing processing based on information about a predetermined direction.
[0395] In other words, permissible intra-prediction modes for the current block can be determined based on information about a predetermined direction obtained from at least one of the coefficient distribution of the inverse-transformed block, the intra-prediction modes of neighboring blocks, the size of the current block, and the shape of the current block. Optionally, to generate a list of MPM candidate modes, MPM candidates can be determined based on information about a predetermined direction obtained from at least one of the coefficient distribution of the inverse-transformed block, the intra-prediction modes of neighboring blocks, the size of the current block, and the shape of the current block. Optionally, a predetermined mode among non-MPM modes can be selected based on predetermined direction information obtained from at least one of the coefficient distribution of the inverse-transformed block, the intra-prediction modes of neighboring blocks, the size of the current block, and the shape of the current block. Compared to non-MPM modes, the predetermined modes can be encoded / decoded using relatively fewer bits.
[0396] Adaptively processing the permissible intra-prediction modes for the current block can mean determining the permissible intra-prediction modes based on the size and / or shape of the current block. For example, when the current block is rectangular, a predetermined directional mode can be limited based on the width-to-height ratio of the current block. Alternatively, the predetermined directional mode can be limited based on the size and / or orientation of the diagonal components. The predetermined directional mode can be a mode with horizontal or vertical orientation.
[0397] In this scenario, by changing the reference pixel used to predict the restricted directional mode, the mode corresponding to the changed reference pixel can be determined as an allowable intra-prediction mode instead of a restricted directional mode. For example, when the current block is a rectangular 2N×N block, the allowable intra-prediction mode can be restricted to an intra-prediction mode used to perform prediction using reference pixels in the lower left block. An intra-prediction mode used to perform prediction using reference pixels in the upper right block can be determined as an allowable intra-prediction mode, rather than a restricted mode. Furthermore, for example, when the current block is a rectangular N×2N block, the allowable intra-prediction mode is restricted to an intra-prediction mode used to perform prediction using reference pixels in the upper right block. Optionally, an intra-prediction mode used to perform prediction using reference pixels in the lower left block can be determined as an allowable intra-prediction mode, rather than a restricted mode.
[0398] Furthermore, the number of restricted intra-prediction modes and / or the number of intra-prediction modes used to replace restricted intra-prediction modes may vary depending on the size and / or shape of the current block. For example, the number may vary based on the width-to-height ratio of the current block or the size and / or orientation of the diagonal components. Additionally, the number of restricted intra-prediction modes may differ from the number of intra-prediction modes used to replace restricted intra-prediction modes.
[0399] Next, in order to generate an intra-prediction block (a prediction block generated by intra-prediction), the intra-prediction unit 1330 reads reference pixels from the frame storage unit 1360 and determines whether there are unusable reference pixels.
[0400] By applying the decoded intra-prediction mode of the current block, a determination operation can be performed based on whether there are reference pixels for generating the intra-prediction block.
[0401] Next, when it is necessary to generate reference pixels, the intra-frame prediction unit 1330 can generate reference pixels arranged at the locations of unavailable pixels by using the previously reconstructed available reference pixels.
[0402] The definition and method for generating unavailable reference pixels can be compared with... Figure 7 The same intra-prediction unit 750 shown is used. However, the reference pixels used to generate the intra-prediction block can be selectively reconstructed based on the decoded intra-prediction mode of the current block.
[0403] Furthermore, the intra-prediction unit 1330 determines whether to apply a filter to a reference pixel to generate a prediction block. That is, the intra-prediction unit 1330 can determine whether to apply a filter to a reference pixel to generate an intra-prediction block for the current block based on the decoded intra-prediction mode and size of the current prediction block.
[0404] The larger the block size, the more severe the block artifact problem becomes. As the block size increases, the number of prediction patterns used to filter the reference pixel also increases. When the block size is larger than a predetermined size, the block is considered a flat region. Therefore, to reduce computational complexity, filtering of the reference pixel may be omitted.
[0405] When it is determined that filtering needs to be applied to the reference pixel, the intra-prediction unit 1330 uses the filter to filter the reference pixel.
[0406] Two or more filters can be used adaptively based on the step difference between reference pixels. The filter coefficients are preferably symmetrical.
[0407] Two or more filters can be applied adaptively based on the size of the current block. When applying filters, narrowband filters can be applied to smaller blocks, and wideband filters can be applied to larger blocks.
[0408] When using DC mode for prediction, a prediction block is generated with the average pixel values of the reference pixel, thus eliminating the need for filtering. When the image has a vertical pattern associated with the vertical direction or a horizontal pattern associated with the horizontal direction, filtering of the reference pixel is not required.
[0409] In this way, since the application of filtering is associated with the intra-prediction mode of the current block, the reference pixels can be filtered adaptively based on the intra-prediction mode of the current block and the size of the prediction block.
[0410] Next, the intra-prediction unit 1330 generates prediction blocks using reference pixels or filtered reference pixels based on the reconstructed intra-prediction mode. The generation of prediction blocks can be performed in the same manner as in the encoding device 700. Therefore, the details of the generation of prediction blocks for decoding are not described.
[0411] Intra-prediction unit 1330 determines whether to filter the generated prediction block. This determination can be based on information included in the stripe header or coding unit header, or on the intra-prediction mode of the current block.
[0412] When it is determined that the generated prediction block will be filtered, the intra-prediction unit 1330 can generate new pixels by filtering the pixels at a specific location in the prediction block using available reference pixels adjacent to the current block.
[0413] For example, when using DC mode, some predicted pixels adjacent to the reference pixels can be filtered using the reference pixels adjacent to the predicted pixels.
[0414] Therefore, depending on the position of the predicted pixel, one or two reference pixels are used to filter the predicted pixel, and the filtering of the predicted pixel in DC mode can be applied to prediction blocks of all sizes.
[0415] On the other hand, in the case of vertical mode, in the predicted pixels of the predicted block, the predicted pixels adjacent to the left reference pixel can be modified using the reference pixels used to generate the predicted block, excluding the upper reference pixel.
[0416] In a similar manner, in the horizontal mode, the predicted pixels generated can be modified using reference pixels other than the left reference pixel used to generate the predicted block, which are adjacent to the upper reference pixel.
[0417] Filtering of predicted pixels can be represented by performing filtering using values predicted from a predetermined intra-frame prediction mode and one or more reference pixels. For example, filtering can be performed by weighted summing of predicted values generated using directional prediction and one or more values of one or more reference pixels adjacent to the current block.
[0418] In this way, the current block can be reconstructed using the predicted block of the current block and the residual block of the current block.
[0419] Figure 15This is a diagram illustrating a method for partitioning an image into blocks and performing image processing based on each block, according to another embodiment of the present disclosure.
[0420] Reference Figure 15 The coding tree unit (CTU) with a maximum size of 256×256 is partitioned using a quadtree structure to generate four square coding units (CUs).
[0421] In the CUs obtained from partitioning a quadtree structure, at least one CU can be partitioned using a binary tree structure to produce two CUs with rectangular shapes. At least one CU in the CUs can be further partitioned using a ternary tree structure.
[0422] In the CUs obtained from the quadtree structure partitioning, at least one CU can be further partitioned using the quadtree structure to produce four square CUs.
[0423] In the CUs obtained from the binary tree structure partitioning, at least one CU can be further partitioned by the binary tree structure partitioning to produce two smaller CUs with rectangular shapes.
[0424] In the CUs obtained from quadtree partitioning, at least one CU can be further partitioned into smaller CUs with square or rectangular shapes by quadtree structure partitioning or binary tree structure partitioning.
[0425] The coded blocks (CBs) obtained from the binary tree structure partitioning can be used for prediction and transformation as is, without further partitioning. Figure 15 The prediction unit (PU) and transform unit (TU) in the coded block (CB) shown may have the same size as the corresponding CB.
[0426] It can be used as already referenced Figures 9 to 10 The partitioning method described will partition each CU in the resulting CU through a quadtree structure into one PU or two or more PUs.
[0427] It can be used as already referenced Figure 11 The partitioning method described divides each coding unit in the quadtree structure partitioned into one TU or two or more TUs, and each TU can have a maximum size of 64×64 pixels.
[0428] The syntax for partitioning an image into blocks and processing the image based on each block can use flags to indicate partitioning information. For example, the flag `split_cu_flag` can be used to indicate whether a CU (Cu unit) is partitioned, and the flag `binary_depth` can be used to indicate the depth of the CU generated by partitioning through a binary tree structure. Additionally, the flag `binary_split_flag` can be used to indicate whether the CU is partitioned using a binary tree structure.
[0429] It can be applied and referenced Figures 7 to 14 The described method is based on the references already referenced Figure 15 The blocks obtained from the described partitioning operations (e.g., CU, PU, and TU) are used for image encoding and decoding.
[0430] In the following text, refer to Figures 16 to 21 The following describes a method for partitioning a CU into one TU or two or more TUs according to another embodiment.
[0431] According to an embodiment of the present invention, the CU can be partitioned into TUs generated by partitioning through a binary tree structure, wherein each TU in the TU is a basic unit for transforming the residual block.
[0432] Figure 16 This is an illustration of a method for dividing an image into blocks and performing image processing based on each block, according to another embodiment of the present disclosure.
[0433] Reference Figure 16 At least one of the rectangular coding blocks CB0 and CB1 generated by the binary tree structure partitioning can be further partitioned into square transformation units TU0 and TU1 through the binary tree structure partitioning, wherein each of the rectangular coding blocks CB0 and CB1 has a size of N×2N or 2N×N, and the square transformation units TU0 and TU1 have a size of N×N.
[0434] As described above, block-based image coding methods can perform prediction, transformation, quantization, and entropy coding processes.
[0435] In the prediction process, a prediction signal is generated by referencing the current block to be encoded, a previously encoded image, or a previously encoded neighboring image, and a difference signal is calculated as the difference between the prediction signal and the current block.
[0436] In transform processing, various transform functions can be used to transform the differential signal (input signal). The transformed signal is classified into DC coefficients and AC coefficients, and energy compression is performed to improve coding efficiency.
[0437] In the quantization process, the input transform coefficients are quantized, and then the quantized signal is entropy encoded to generate an encoded image.
[0438] Image decoding methods can be performed in the reverse order of encoding, and image quality distortion may occur during quantization.
[0439] As a method to improve coding efficiency and reduce image quality distortion, the size or shape of the TU and the transform function used can vary depending on the distribution of the differential signal (input signal) in the transform process and the properties of the image.
[0440] For example, when using cost measurement methods (such as sum of absolute differences (SAD) or mean squared error (MSE)) to find blocks similar to the current block in the prediction process via block-based motion estimation, the distribution of the difference signal can appear in a variety of forms depending on the nature of the image.
[0441] Therefore, efficient coding can be performed by selecting the size or shape of the transform unit (TU) through various distribution forms based on differential signals.
[0442] For example, when a differential signal appears in any coded block (CBx), an effective transformation can be performed by partitioning the coded block (CBx) into two transform units (TUs) via a binary tree structure. The DC value is generally considered to represent the average value of the input signal. Therefore, when a differential signal is input in the transformation process, the DC value can be effectively represented because each of the coded blocks (CBx) is partitioned into two transform units (TUs).
[0443] Reference Figure 17 The square 2N×2N encoding unit CU0 can be partitioned into two rectangular transformation units TU0 and UT1 with an N×2N or 2N×N size by using a binary tree structure partitioning.
[0444] According to another embodiment of the present invention, as described above, multiple transform units (TUs) are generated by performing two or more binary tree structure partitions on the coding unit (CU).
[0445] Reference Figure 18 First, the N×2N rectangular coding block CB1 is partitioned into two N×N blocks using a binary tree partitioning structure. Then, each of the two N×N blocks is further partitioned into either an N×N / 2 or an N / 2×N rectangular block using a binary tree partitioning structure. Next, the N / 2×N or N×N / 2 blocks are further partitioned into N / 2×N / 2 square transform units TU1, TU2, TU4, and TU5 using a binary tree partitioning structure.
[0446] Reference Figure 19A 2N×2N square coding block CB0 can be partitioned through a binary tree structure to generate an N×2N block. This N×2N block can be further partitioned through a binary tree structure to generate an N×N square block. This N×N square block can be further partitioned through a binary tree structure to generate N / 2×N rectangular transformation units TU1 and TU2.
[0447] Reference Figure 20 A 2N×N rectangular coding block CB0 can be partitioned using a binary tree structure to generate an N×N block, and each of the N×N blocks can be partitioned using a quadtree structure to generate N / 2×N / 2 square transformation units TU1, TU2, TU3 and TU4.
[0448] Regarding the reference Figures 16 to 20 The blocks obtained by the described partitioning method (e.g., coding units (CUs), prediction units (PUs), and transform units (TUs)) can be obtained using already referenced... Figures 7 to 14 The described method performs image encoding and decoding.
[0449] In the following text, an embodiment of the method by which the encoding device 700 determines the block partition structure will be described.
[0450] The image encoding device 700's image partitioning unit 110 can determine the divisible coding unit (CU), prediction unit PU, and transform unit TU by performing rate-distortion optimization (RDO) according to a preset order.
[0451] For example, in order to determine the block partitioning structure, the picture partitioning unit 710 can determine the optimal block partitioning structure in terms of bit rate and distortion by performing Rate Distortion Optimized Quantization (RDO-Q).
[0452] Reference Figure 21 When the encoding unit CU has a size of 2N×2N pixels, the optimal partitioning structure of the transform unit TU can be determined by performing RDO in the following order: (a) 2N×2N pixel size, (b) N×N pixel size, (c) N×2N pixel size, and (d) 2N×N pixel size.
[0453] Reference Figure 22 When the encoding unit CU has an N×2N pixel size or a 2N×N pixel size, the optimal partitioning structure of the transform unit TU can be determined by performing RDO in the following order: (a) N×2N pixel size (or 2N×N pixel size), (b) N×N pixel size, (c) N / 2×N (or N×N / 2) pixel size and N×N pixel size, (d) N / 2×N / 2 pixel size, N / 2×N pixel size and N×N pixel size, and (e) N / 2×N pixel size.
[0454] In the foregoing, although the block partitioning method according to the present invention has been described with reference to the example of performing rate distortion optimization (RDO) to determine the block partitioning structure, the screen partitioning unit 710 may use a sum of absolute difference (DAD) algorithm or mean square error (MSE) algorithm to determine the block partitioning structure, thereby providing a good trade-off between improving efficiency and reducing complexity.
[0455] The image processing method according to an embodiment of the present invention and the image encoding and decoding method using the image processing method will be described in more detail below.
[0456] Figure 23 This is a block diagram according to an embodiment of the present disclosure for describing the configuration for determining the intra-prediction mode of an intra-prediction unit.
[0457] Reference Figure 23 According to an embodiment of the present invention, the intra-frame prediction unit 1330 includes: a neighboring block information processing unit, including a coefficient distribution identification unit 2331 and a neighboring pattern identification unit 2332; a similarity flag determination unit 2333; a similarity flag storage unit 2335; and a pattern determination unit 2334.
[0458] Intra-prediction unit 1330 can determine the intra-prediction mode of the current block. For example, mode determination unit 2334 can perform restriction processing or direction adaptation processing on the intra-prediction mode corresponding to the current block based on information about previously decoded neighboring blocks located near the current block. Neighboring block information processing unit can obtain information about previously decoded neighboring blocks located near the current block to be intra-predicted and decoded. Mode determination unit 2334 can use neighboring block information processing unit to parse bitstream to obtain information. In addition, when similarity flag information of previously processed neighboring blocks exists, mode determination unit 2334 can obtain similarity flag information from similarity flag storage unit 2335. Mode determination unit 2334 can use the obtained similarity flag information to perform restriction processing or direction adaptation processing on candidate modes corresponding to the current block. Here, restriction processing can mean that only a predetermined intra-prediction mode is allowed or a predetermined intra-prediction mode is not allowed. For example, based on information about neighboring blocks and / or similarity flag information of neighboring blocks, only a predetermined intra-prediction mode corresponding to the current block is allowed. Here, direction adaptation processing can mean that processing is performed based on information about a predetermined direction. For example, the intra-prediction mode corresponding to the current block can be determined based on predetermined direction information obtained from information about neighboring blocks and / or similarity flag information of neighboring blocks.
[0459] Furthermore, the intra-frame prediction unit 1330 can determine the similarity flag IS_SIMILAR_FLAG corresponding to the current block from information about neighboring blocks using the information determined by the similarity flag determination unit 2333. The determined similarity flag is stored and updated in association with the current flag in the similarity flag storage unit 2335 and is used to determine the pattern of the next block.
[0460] Here, information about neighboring blocks may include at least one of inverse transform coefficient distribution information, pixel value change information, pixel value information, and pixel difference information. For example, pixel value change information can be calculated using the dispersion of neighboring pixels included in the neighboring blocks, and examples of measures of dispersion include statistical indicators such as variance, standard deviation, mean error, and interquartile range.
[0461] The neighboring block information processing unit may include a coefficient distribution identification unit 2331 and a neighboring pattern identification unit 2332. The neighboring block information processing unit can provide information for determining the pattern or similarity markers of the current block.
[0462] The similarity flag determination unit 2333 can acquire the inverse transform coefficient distribution information identified by the coefficient distribution recognition unit 2331. Furthermore, the similarity flag determination unit 2330 can acquire intra-frame mode information of neighboring blocks identified by the neighboring pattern recognition unit 2332. The similarity flag determination unit 2333 can determine the similarity flag by using the acquired inverse transform coefficient distribution information and / or the intra-frame mode information of neighboring blocks. Here, the similarity flag may include a horizontal similarity flag or a vertical similarity flag.
[0463] The horizontal similarity flag indicates that pixels in previously decoded neighboring blocks have vertical orientation. Having vertical orientation indicates similarity to the block horizontally adjacent to the current block to the left. For example, the horizontal similarity flag indicating that a previously decoded left block horizontally adjacent to the current block has vertical orientation indicates similarity to the current block. Therefore, this horizontal similarity flag can be represented by the left similarity flag IS-SIMILAR_LEFT.
[0464] The vertical similarity flag indicates that previously decoded neighboring blocks have horizontal similarity. Having horizontal similarity means that the block is similar to the block directly above it and perpendicular to the current block. For example, the vertical similarity flag indicates that the previously decoded block above the current block, which is perpendicular to the current block, has vertical similarity to the current block. This vertical similarity flag can be represented by the upper similarity flag IS_SIMILAR_UP.
[0465] Horizontal or vertical similarity flags will be calculated from neighboring blocks and stored in the similarity flag storage unit 2335. On the other hand, the decoding device 1300 continuously updates the similarity flag information in response to changes in neighboring blocks. This processing can be performed without additional signaling information.
[0466] The similarity flag determination unit 2333 can obtain information about neighboring blocks from the coefficient distribution recognition unit 2331, such as inverse transform coefficient distribution information. The similarity flag determination unit 2333 can determine the horizontal or vertical orientation of neighboring blocks based on the neighboring block information. The similarity flag determination unit 2333 can use the coefficient distribution information and / or intra-frame mode information of neighboring blocks to more accurately determine the orientation. Furthermore, the determined orientation can be used to determine the similarity flag.
[0467] The mode determination unit 2334 can perform restriction processing or orientation adaptation processing on intra-frame mode determination based on the similarity flag information of neighboring blocks processed previously.
[0468] Here, the restriction processing can mean allowing only predetermined intra-prediction modes or disallowing predetermined intra-prediction modes. For example, the allowed intra-prediction modes for the current block can be restricted based on similarity flag information. Optionally, the prediction mode candidates in the MPM candidate mode list can be restricted. More specifically, as allowed intra-prediction modes for the current block, only predetermined intra-prediction modes can be allowed based on information obtained from at least one of the coefficient distribution of the inverse-transformed block, the intra-prediction modes of neighboring blocks, the size of the current block, and the shape of the current block. Optionally, when generating the MPM candidate mode list, only predetermined intra-prediction mode candidates can be allowed based on information obtained from at least one of the coefficient distribution of the inverse-transformed block, the intra-prediction modes of neighboring blocks, the size of the current block, and the shape of the current block.
[0469] Here, orientation-adaptive processing can mean performing corresponding processing based on predetermined orientation information. For example, the permissible intra-prediction modes of the current block can be processed in a restricted manner based on similarity flag information. Optionally, the prediction mode candidates in the MPM candidate mode list can be processed in an orientation-adaptive manner. More specifically, the permissible intra-prediction modes of the current block can be determined based on predetermined orientation information obtained from at least one of the coefficient distribution of the inverse-transformed block, the intra-prediction modes of neighboring blocks, the size of the current block, and the shape of the current block. Optionally, when generating the MPM candidate mode list, MPM candidates can be determined based on predetermined orientation information obtained from at least one of the coefficient distribution of the inverse-transformed block, the intra-prediction modes of neighboring blocks, the size of the current block, and the shape of the current block.
[0470] On the other hand, when similarity markers are determined, the coefficient distribution identification unit 2331 can obtain and use only transformed coefficient information by parsing the bitstream. The coefficient distribution identification unit 2331 may not use pixel information of previously predicted decoded blocks or reconstructed residual signals. Therefore, constraint processing or orientation adaptive processing can eliminate parsing dependence.
[0471] The mode determination unit 2334 can perform constraint processing without additional signaling information sent from the encoding device 700. Therefore, additional signaling information regarding predicted mode candidates with unnecessary directionality can be reduced, and compression efficiency can be improved without causing inter-frame processing resolution dependencies in decoding.
[0472] However, one embodiment of the present invention does not exclude the case where similarity flag information corresponding to neighboring blocks is transmitted by signal. The determination and storage processing of similarity flag information can be performed as needed in the same manner as in the encoding device 700. That is, the encoding device 700 can generate similarity flags based on neighboring block information and use the generated similarity flags to determine the intra-frame mode in a direction-adaptive manner. In this case, the generated similarity flag information is included in the bitstream and is therefore transmitted as a bitstream to the decoding device 1300. The mode determination unit 2334 can determine the mode of the current block using the similarity flag information of neighboring blocks transmitted by signal, even if it does not obtain information about neighboring blocks separately or does not process information about neighboring blocks. Therefore, in this case, computational complexity can be reduced and the performance of the decoding device can be improved.
[0473] Specifically, according to one embodiment of the invention, orientation adaptive processing can reduce the number of intra-prediction modes within an angle for a specific direction not indicated by a similarity marker, thereby simplifying the processing. Furthermore, orientation adaptive processing can increase the number of intra-prediction modes within an angle by finer segmenting the angle for a direction indicated by a similarity marker. That is, although the total number of intra-prediction mode candidates remains unchanged, more accurate and precise prediction modes can be determined.
[0474] For example, when the intra-frame mode of a neighboring block is a horizontal mode, a relatively larger number of directional modes can be assigned to the horizontal direction because the best prediction mode of the current block is highly likely to be similar to the horizontal mode. On the other hand, in this case, a relatively small number of directional modes are assigned to the vertical direction.
[0475] Figure 24 and Figure 25 This is a flowchart illustrating a method for performing intra-predictive decoding processing using a direction-adaptive intra-predictive mode according to an embodiment of the present disclosure.
[0476] Figure 26 and Figure 27 This is a flowchart illustrating horizontal and vertical similarity markers determined according to an embodiment of the present disclosure in order to obtain an orientation-adaptive intra-frame mode.
[0477] Reference Figure 24 The decoding device 1300 can obtain information about neighboring blocks (S2410).
[0478] For example, the decoding device 1300 may use the coefficient distribution recognition unit 2331 and / or the neighboring pattern recognition unit 2332 to obtain coefficient distribution information and / or intra-frame pattern information.
[0479] The similarity flag determination unit 2333 can determine the horizontal similarity flag or the vertical similarity flag by using information about neighboring blocks (e.g., transform coefficient distribution information and / or intra-frame mode information) (S2420). Furthermore, the horizontal similarity flag or the vertical similarity flag determined in step S2420 can be stored in the similarity flag storage unit 2335 (S2430).
[0480] Here, as Figure 26 and Figure 27 As shown, horizontal similarity markers and / or vertical similarity markers can be determined based on information about neighboring blocks. For example, horizontal and / or vertical similarity markers can be determined from transform coefficient information of neighboring blocks. Specifically, the information about neighboring blocks can be transform coefficient distribution information. Similarity markers can be used to determine the intra-frame mode of a block.
[0481] Specifically, the similarity flag determination unit 2333 can use the coefficient distribution information of neighboring blocks and / or intra-frame mode information to determine whether the distribution of the residual values of each pixel to be inversely transformed has a vertical or horizontal orientation.
[0482] More specifically, refer to Figure 26 and Figure 27 The similarity flag determination unit 2333 can determine whether the reconstructed pixels in the bottom row or rightmost column of the reconstructed block have directional similarity values based on the coefficient distribution information in the transform domain obtained from the coefficient distribution recognition unit 2331 and the intra-frame mode information of neighboring blocks. For example, the aforementioned determination operation can be performed in the parsing step.
[0483] Here, the coefficient distribution information can be classified according to different transform kernels. The similarity flag determination unit 2333 can perform direction mapping corresponding to at least one of the coefficient distributions classified according to the transform kernel and intra-frame mode according to a preset mapping table.
[0484] For example, such as Figure 26 As shown, when the DCT transform coefficients exist only in the first row or are all zero, it can be confirmed that the reconstruction residuals in the rightmost column have the same value.
[0485] Therefore, the similarity flag determination unit 2333 can determine whether the DCT coefficients exist only in the first row of neighboring blocks or are all zero. After determination, the similarity flag determination unit 2333 can determine whether the intra-frame mode of the neighboring blocks is DC mode or vertical direction mode.
[0486] As described above, when the DCT transform coefficients exist only in the first row of neighboring blocks or are all zero, and when the intra-frame mode is DC mode or vertical orientation mode, it can be assumed that the rightmost reconstructed pixel to be reconstructed has the same value. Therefore, when the DCT transform coefficients of neighboring blocks exist only in the first row of neighboring blocks or are all zero, and when the intra-frame mode is DC mode or vertical orientation mode, the similarity flag determination unit 2333 can determine that the residual signal of the reconstructed neighboring blocks has verticality. The similarity flag determination unit 2333 sets the horizontal similarity flag IS_SIMILAR_LEFT for the current block to the value 1. On the other hand, when the above conditions are not met, the horizontal similarity flag IS_SIMILAR_LEFT can be set to the value 0.
[0487] On the other hand, refer to Figure 27 When the DCT transform coefficients exist only in the first column or are all zero, it can be confirmed that the reconstruction residuals in the bottom row have the same value.
[0488] Therefore, when the DCT transform coefficients exist only in the first column of the neighboring blocks or are all zero, and when the intra-frame mode is DC mode or horizontal direction mode, the similarity flag determination unit 2333 determines that the reconstructed residual signal in the neighboring blocks has horizontality. The similarity flag determination unit 2333 can set the vertical similarity flag IS_SIMILAR_UP for the current block to a value of 1. On the other hand, when the above conditions are not met, the vertical similarity flag IS_SIMILAR_UP can be set to a value of 0.
[0489] Therefore, the horizontal similarity flag indicates whether the pixels to the left of the current block are similar, and the vertical similarity flag indicates whether the pixels above the current block are similar.
[0490] Furthermore, the directionality of the current block can be indicated by using either a horizontal similarity flag or a vertical similarity flag. For example, when the DCT transform coefficients of a neighboring block exist only in the first row of the neighboring block or are all zero, and when the intra-frame mode is DC mode or vertical orientation mode, the similarity flag determination unit 2333 determines that the reconstructed residual signal in the neighboring block has verticality. Based on this determination operation, the horizontal similarity flag for the current block can be determined to have a value of 1. When the DCT transform coefficients exist only in the first column of the neighboring block or are all zero, and when the intra-frame mode is DC mode or horizontal orientation mode, the similarity flag determination unit 2333 determines that the reconstructed residual signal of the neighboring block has horizontality. Then, based on this determination operation, the horizontal similarity flag for the current block can be determined to have a value of 0.
[0491] As described above, the coefficient distribution identification unit 2331 parses the bitstream to identify the coefficient distribution, the neighboring pattern identification unit 2332 identifies the intra-frame pattern of neighboring blocks, and the pattern determination unit 2334 determines the intra-frame pattern of the current block. The decoding device 1300 can preprocess this through parsing. For example, when the scanning direction of the transform coefficients is determined according to the intra-frame pattern, the intra-frame pattern determination process according to an embodiment of the present invention can be performed in the parsing process.
[0492] The operation of the similarity marker determination unit 2333 is described with reference to the DCT transform. When using a transform method different from DCT, the similarity marker determination unit 2333 can use feature information from different transforms. For example, the similarity marker determination unit 2333 can identify coefficient distribution information corresponding to the feature information, and can determine horizontal or vertical similarity markers based on the coefficient distribution information. Horizontal or vertical similarity markers can be determined based on whether the reconstructed pixels in the lower row or right column of the reconstructed block have similar values within a predetermined range. Furthermore, in the above embodiment, the orientation of the bottommost or rightmost reconstructed pixel in the reconstructed block is predicted, but this is not a limitation. The orientation of reconstructed pixels at any position in the reconstructed block can be predicted using DCT or another transform method.
[0493] Although embodiments of the present invention provide an example of the similarity flag determination unit 2333 using both coefficient distribution information and intra-frame mode information, it is also possible to use only one of the coefficient distribution information and intra-frame mode information.
[0494] On the other hand, when the similarity flag determination unit 2333 determines a horizontal similarity flag or a vertical similarity flag, the flag can be stored as an internal variable during the parsing process. This flag can be used to determine the intra-frame mode of the next block or lower-layer block. Therefore, the similarity flag is not included in the bitstream to be transmitted as a signal, but it can be processed and / or generated in the decoding device 1300. However, as described above, depending on efficiency, the similarity flag processed by the encoding device 700 can be transmitted as a signal to the decoding device 1300.
[0495] Reference Figure 25 When there are neighboring blocks for which similar flags have been determined, the neighboring blocks of the current block can be identified (S2510).
[0496] Furthermore, horizontal or vertical similarity markers corresponding to neighboring blocks can be obtained from the similarity marker determination unit 2333 (S2520). Horizontal or vertical similarity markers can be predetermined.
[0497] Furthermore, based on the acquired flags, a candidate list of orientation-adaptive intra-prediction modes can be generated (S2530). For example, a restricted candidate list of intra-prediction modes can be created based on the acquired flags.
[0498] Then, the intra-mode of the current block can be selected from the orientation-adaptive intra-mode candidate list (S2540). For example, the intra-mode of the current block can be selected from a restricted intra-mode candidate list.
[0499] Figures 28 to 31 This is a diagram illustrating a direction-adaptive intra-frame mode for certain situations according to an embodiment of the present disclosure.
[0500] According to an embodiment of the present invention, the intra-prediction unit 1330 can adaptively determine the permissible intra-prediction modes of the current block based on at least one similarity flag determined from the coefficient distribution of the inverse-transformed block, the intra-prediction modes of neighboring blocks, the size of the current block, and the shape of the current block. For example, the number of permissible intra-prediction modes of the current block can be increased or decreased based on the orientation information of the similarity flag determined from at least one of the coefficient distribution of the inverse-transformed block, the intra-prediction modes of neighboring blocks, the size of the current block, and the shape of the current block.
[0501] Furthermore, the intra-prediction unit 1330 can determine the prediction mode candidates of the MPM candidate mode list for the current block based on at least one determined similarity flag from the coefficient distribution of the inverse-transformed block, the intra-prediction modes of neighboring blocks, the size of the current block, and the shape of the current block. For example, the number of prediction mode candidates in the MPM candidate mode list for the current block can be increased or decreased based on the orientation information of at least one determined similarity flag from the coefficient distribution of the inverse-transformed block, the intra-prediction modes of neighboring blocks, the size of the current block, and the shape of the current block.
[0502] For example, when a left-neighboring pixel is similar to a pixel in the current block, the number of intra-mode candidates in the horizontal direction can be reduced to simplify intra-mode candidate selection in the horizontal or leftward direction. Furthermore, to subdivide (i.e., finer-divide) the intra-mode candidate directions in the upward or vertical direction, the number of intra-mode candidates in the vertical direction can be increased. In this case, any mode in the horizontal direction will produce almost identical predictions. Therefore, mode subdivision (i.e., finer-divide) in the vertical direction allows for finer adjustment of intra-mode prediction in the vertical direction, thereby improving prediction accuracy and image quality. Here, subdividing (i.e., finer-divide) the intra-mode candidate direction means increasing the number of intra-prediction modes in a predetermined direction.
[0503] When each of the pixels above a current block is similar to a pixel in the current block, the number of intra-mode candidates in the vertical direction can be reduced to simplify intra-mode candidate selection in the vertical or upward direction. Conversely, the number of intra-mode candidates in the horizontal direction can be increased to subdivide (i.e., finer divide) the intra-mode candidate direction in the horizontal direction. In this case, any mode in the vertical direction will produce almost the same prediction. Therefore, mode subdivision (finer division) in the horizontal direction allows for finer adjustment of intra-mode prediction in the horizontal direction, thereby improving prediction accuracy and image quality.
[0504] Therefore, a list can be constructed that includes an increased number of intra-prediction mode candidates in one direction (i.e., the horizontal or vertical direction) and a decreased number of intra-prediction mode candidates in other directions.
[0505] For example, the intra-prediction unit 1330 can reduce the number of intra-modes corresponding to a predetermined threshold or greater in the horizontal direction based on a horizontal similarity flag. Conversely, in contrast to the reduction operation described above, the intra-modes corresponding to the predetermined threshold or greater are subdivided, and an increased number of intra-modes can be assigned. The predetermined threshold can be represented by a mode number, mode value, mode digit, mode angle, mode direction, or one or more combinations thereof. However, the predetermined threshold is not limited to these and can be any condition that can be used to increase or decrease intra-modes in the vertical direction.
[0506] For example, the intra-prediction unit 1330 can reduce the number of intra-modes corresponding to a predetermined threshold or greater in the vertical direction based on the vertical similarity flag. On the other hand, in contrast to the reduction operation described above, the intra-modes corresponding to the predetermined threshold or greater in the horizontal direction can be subdivided, and an increased number of intra-modes can be assigned to the horizontal direction.
[0507] In other words, while enabling more precise intra-frame prediction processing for specific necessary directions, the total number of directional patterns is maintained.
[0508] When decreasing or increasing the number of intra-prediction modes, the decrease or increase operation can be adaptively processed based on at least one of the intra-prediction modes of the current block, the size of the current block, and the shape of the current block. For example, when the current block is a 2N×N block, at least one intra-prediction mode in the horizontal mode with a value other than a predetermined threshold can be decreased. In this case, instead of decreasing the horizontal intra-prediction mode, at least one vertical intra-prediction mode can be added. Similarly, when the current block is an N×2N block, at least one intra-prediction mode with a value other than a predetermined threshold can be decreased from the vertical mode. In this case, instead of decreasing the vertical intra-prediction mode, at least one horizontal intra-prediction mode can be added. In other words, the number of intra-prediction modes in the horizontal direction may differ from the number of intra-prediction modes in the vertical direction, depending on the size and / or shape of the current block. Furthermore, the number of intra-prediction modes removed may differ from the number of intra-prediction modes added.
[0509] The predetermined threshold can be determined by one or more of the following: a predetermined intra-frame prediction mode number, a mode value, a mode number, a mode angle, a mode direction, the width of the current block, and its height.
[0510] The removed intra-prediction modes and / or the additional intra-prediction modes assigned in place of the removed intra-prediction modes can vary depending on the size and / or shape of the current block. For example, they can vary based on the width-to-height ratio of the current block or the size and / or orientation of the diagonal components. The reduced intra-prediction modes can be modes using reference pixels arranged outside (width) × N and N × (height) of the current block. Here, N is a positive integer greater than zero; for example, N could be 2. The number of intra-prediction modes can be reduced or increased by shifting the intra-prediction mode numbers. For example, a predetermined offset can be applied to the intra-prediction mode numbers.
[0511] As an example of this disclosure, when the left-neighboring pixel is similar to a pixel in the current block, the number of intra-mode candidates in the horizontal or leftward direction can be limited. In this case, any mode in the horizontal direction will produce almost the same prediction result. Therefore, the limitation process reduces computational complexity and computational cost. Furthermore, the number of bits allocated to intra-mode can be reduced. Therefore, compression efficiency can be improved.
[0512] When a neighboring pixel above is similar to a pixel in the current block, the number of intra-mode candidates in the vertical or upward direction can be limited. In this case, any mode in the vertical direction will produce almost the same prediction result. Therefore, the limitation process reduces computational complexity and computational cost, and reduces the number of bits allocated to intra-mode, thereby improving compression efficiency.
[0513] Therefore, a list can be constructed that includes an increased number of intra-prediction mode candidates in one direction (i.e., the horizontal or vertical direction) determined by similarity flags and a decreased number of intra-prediction mode candidates in other directions.
[0514] Therefore, a list can be constructed that includes restricted intra-prediction mode candidates in one of the horizontal and vertical directions, selected based on a similarity flag. For example, the intra-prediction unit 1330 can restrict intra-modes in the horizontal direction by removing all horizontal intra-modes corresponding to a predetermined threshold or a larger value, thereby retaining only specific directions.
[0515] For example, the intra-prediction unit 1330 can limit intra-frame modes in the vertical direction by removing all vertical intra-frame modes corresponding to a predetermined threshold or a larger value, thereby retaining only specific directions.
[0516] Figure 28 This shows the case where all horizontal or vertical similarity flags are zero, and the intra-frame mode is determined or the MPM candidate list is constructed without performing constraint processing or orientation adaptation processing.
[0517] Figure 29 This illustrates the case where the horizontal similarity flag has a value of 1 and the vertical similarity flag has a value of 0. Figure 29 In case (a), the intra-frame modes subdivided in the vertical direction are included in the target determination list or MPM candidate list, rather than the intra-frame modes with a horizontal component having a predetermined threshold or greater. Furthermore, for example, in Figure 29 In case (b), intra-frame patterns containing a predetermined threshold or larger value can be removed in advance so that they are not included in the target list or MPM candidate list.
[0518] Figure 30 This illustrates the case where the horizontal similarity flag has a value of 0 and the vertical similarity flag has a value of 1. Figure 30 In case (a), intra-frame modes subdivided in the horizontal direction are included in the target determination list or MPM candidate list, rather than intra-frame modes with a vertical component having a predetermined threshold or greater. Furthermore, for example, in Figure 30 In case (b), intra-frame patterns containing vertical components with a predetermined threshold or greater can be removed in advance so that they are not included in the target list or MPM candidate list.
[0519] Figure 31 This shows the case where both the horizontal similarity flag and the vertical similarity flag have a value of 1. Figure 31In case (a), intra-frame patterns densely concentrated in the vertical and horizontal directions can be more finely segmented, and the resulting additional intra-frame patterns can be included in the target list or MPM candidate list. Furthermore, for example, in Figure 31 In case (b), all directional intra-frame modes other than vertical and horizontal modes are excluded from the target list or MPM candidate list.
[0520] Therefore, the amount of computation required to derive the intra-frame mode in the decoding device 1300 can be reduced, and the number of bits allocated to the intra-frame mode can also be reduced.
[0521] On the other hand, the number of modes to be removed or added in the vertical or horizontal direction is given only for illustrative purposes, and the number of intra-modes in each direction can be determined differently depending on the total number of intra-modes. However, the number of intra-modes can be adaptively increased or decreased in a particular direction based on similar flags.
[0522] Furthermore, according to embodiments of the present invention, the intra-prediction unit 1330 can determine the intra-mode angle range of the current block based on neighboring block information. Additionally, the intra-prediction unit 1330 can construct an intra-mode candidate list using only the intra-modes included within the angle range. For example, the intra-prediction unit 1330 can use directional information derived from neighboring blocks to determine the angle range table. Furthermore, the intra-prediction unit 1330 can adaptively determine the intra-mode based on the angle range table.
[0523] Furthermore, the number of intra-frame modes included within the stated angular range can be the same as or reduced compared to the number of intra-frame modes set before performing orientation adaptation processing. When the number is the same, the intra-frame modes corresponding to the intra-frame mode recognition information can be more finely divided according to equal angular intervals. In this case, image quality is improved.
[0524] On the other hand, when the number of intra-frame modes within the angular range is reduced compared to the number set before performing direction adaptive processing, intra-frame modes outside the angular range can be excluded, resulting in a slight degradation in image quality and an increase in computational load and transmission efficiency. In this case, coding efficiency can be significantly improved.
[0525] Therefore, the intra-prediction unit 1330 can select a partitioning mode in which intra-modes within a specific directional angle range are more finely divided, or an exclusion mode in which intra-modes outside the directional angle range are excluded. In this case, additional processing to improve both image quality and coding efficiency can be performed.
[0526] Figures 32 to 34 This is a diagram illustrating a method for improving entropy coding efficiency using neighboring block information according to an embodiment of the present disclosure.
[0527] Figure 32 This is a flowchart illustrating an image processing method according to an embodiment of the present invention.
[0528] The entropy coding unit 740 (or entropy decoding unit 1310) can perform entropy coding or entropy decoding on the intra-frame pattern information of the current block using the directional information of neighboring blocks. The directional information of neighboring blocks can be obtained from the neighboring block information processing unit. For example, the directional information can be obtained from the coefficient distribution recognition unit 2331 and / or the neighboring pattern recognition unit 2332, and the directional information may include the aforementioned similarity flag information.
[0529] Reference Figure 32 The directionality of the current block can be determined from the neighboring block information (S3210). Based on the directionality determined in step S3210, a lookup table for entropy coding of intra-frame mode information can be determined (S3220).
[0530] For example, when a horizontal similarity flag and / or a vertical similarity flag are determined, or when decoded intra-frame mode information is obtained, the entropy coding unit 740 may determine an entropy coding lookup table corresponding to the determined horizontal similarity flag and / or vertical similarity flag, or corresponding to the intra-frame mode information of neighboring blocks.
[0531] The directional information of neighboring blocks can be obtained from information about previously decoded neighboring blocks. Here, the directional information may include horizontal directional information and / or vertical directional information. Furthermore, an entropy-encoded lookup table corresponding to information about a specific directionality (i.e., specific directional information) can be determined.
[0532] The entropy coding lookup table may include a coding table in which intra-frame modes have different variable-length codes based on specific directional information. For example, compared to other intra-frame modes, shorter variable-length codes can be assigned to intra-frame modes that correspond to or do not correspond to specific directional information. Therefore, entropy coding and entropy decoding with higher probabilities applied to specific directions can be performed, thereby improving entropy coding efficiency.
[0533] Reference Figure 33 and Figure 34 SET 0 is a set of intra-frame modes that includes intra-frame modes in the horizontal direction, and SET 1 is a set of intra-frame modes that includes intra-frame modes in the vertical direction. When a pixel in the upper block is similar to a pixel in the current block and therefore the vertical similarity flag has a value of 1, the intra-frame modes in SET 0 are assigned shorter codewords (BIT or BIN) compared to the intra-frame modes in SET 1. On the other hand, the intra-frame modes in SET 1 can be assigned relatively longer codewords compared to the intra-frame modes in SET 0. A lookup table with the above characteristics can be constructed.
[0534] If the decoded intra-mode of the block to the left of the current block is horizontal, the directionality information of the current block can be determined to be horizontal. In this case, the intra-mode included in SET 0 is assigned a shorter codeword (BIT or BIN) compared to the intra-mode included in SET 1. On the other hand, the intra-mode in SET 1 can be assigned a relatively longer codeword compared to the intra-mode in SET 0. A lookup table with the above characteristics can be constructed.
[0535] If the decoded intra-mode of the block above the current block is vertical, the orientation information of the current block can be determined as vertical. In this case, intra-modes included in SET 1 are assigned shorter codewords (BIT or BIN). On the other hand, intra-modes included in SET 0 can be assigned relatively longer codewords compared to intra-modes in SET 1. A lookup table with the above characteristics can be constructed.
[0536] Entropy coding unit 740 or entropy decoding unit 1310 can use the directional information of neighboring blocks to determine a lookup table for intra-mode coding or intra-mode decoding of the current block. Based on the directionality of the current block predicted from the neighboring block information, intra-modes can be classified into a set of intra-modes that assign relatively short variable-length codes to directions with higher probabilities and a set of intra-modes that assign relatively long variable-length codes to directions with lower probabilities. Therefore, this process can improve the efficiency of entropy coding.
[0537] Furthermore, the entropy coding unit 740 or the entropy decoding unit 1310 can perform arithmetic entropy coding or arithmetic entropy decoding by using the directional information of neighboring blocks to assign codes with relatively high probabilities to directional information with relatively high probabilities. Additionally, the entropy coding unit 740 or the entropy decoding unit 1310 can perform arithmetic entropy coding or arithmetic entropy decoding by assigning codes with relatively low probabilities to directional information with relatively low probabilities. Therefore, this process can improve the efficiency of entropy coding.
[0538] The above embodiments can be performed in the same way in both the encoder and decoder.
[0539] The order in which the above embodiments are applied may differ between the encoder and the decoder, or the order in which the above embodiments are applied may be the same in the encoder and the decoder.
[0540] The above embodiments can be performed on each luminance signal and chrominance signal, or the above embodiments can be performed on both luminance and chrominance signals in the same way.
[0541] The block shape of the above embodiments of the present invention can be square or non-square.
[0542] The above embodiments of the present invention can be applied based on the size of at least one of the coding block, prediction block, transform block, block, current block, coding unit, prediction unit, transform unit, unit, and current unit. Here, size can be defined as the minimum size or maximum size, or both, for which the above embodiments are applied, or it can be defined as a fixed size to which the above embodiments are applied. Furthermore, in the above embodiments, a first embodiment can be applied to a first size, and a second embodiment can be applied to a second size. In other words, the above embodiments can be applied in combination based on sizes. Furthermore, the above embodiments can be applied when the size is equal to or greater than the minimum size and equal to or less than the maximum size. In other words, the above embodiments can be applied when the block size is included within a specific range.
[0543] For example, the above embodiments can be applied when the size of the current block is 8×8 or larger. For example, the above embodiments can be applied when the size of the current block is 4×4 or larger. For example, the above embodiments can be applied when the size of the current block is 16×16 or larger. For example, the above embodiments can be applied when the size of the current block is equal to or greater than 16×16 and equal to or less than 64×64.
[0544] The above embodiments of the present invention can be applied according to time layers. To identify the time layer to which the above embodiments can be applied, an additional identifier can be signaled, and the above embodiments can be applied to a specified time layer identified by the corresponding identifier. Here, the identifier can be defined as the lowest or highest layer to which the above embodiments can be applied, or both, or it can be defined as indicating a specific layer to which the embodiment is applied. Furthermore, a fixed time layer to which the embodiments are applied can be defined.
[0545] For example, the above embodiments can be applied when the time layer of the current image is the lowest layer. For example, the above embodiments can be applied when the time layer identifier of the current image is 1. For example, the above embodiments can be applied when the time layer of the current image is the highest layer.
[0546] The strip types to which the above embodiments of the present invention are applied can be defined, and the above embodiments can be applied according to the corresponding strip types.
[0547] In the above embodiments, the method is described based on a flowchart having a series of steps or units. However, the present invention is not limited to the order of these steps; rather, some steps may be performed simultaneously with other steps or in a different order. Furthermore, those skilled in the art should understand that the steps in the flowchart are not mutually exclusive, and other steps may be added to the flowchart or some steps may be removed from the flowchart without affecting the scope of the present invention.
[0548] The embodiments include various aspects of the examples. Not all possible combinations of the aspects may be described, but those skilled in the art will recognize different combinations. Therefore, the invention may include all substitutions, modifications, and alterations within the scope of the claims.
[0549] Embodiments of the present invention can be implemented in the form of program instructions, which can be executed by various computer components and recorded in a computer-readable recording medium. The computer-readable recording medium may individually include program instructions, data files, data structures, etc., or may include a combination of program instructions, data files, data structures, etc. The program instructions recorded in the computer-readable recording medium may be specifically designed and constructed for the present invention or are well known to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic recording media (such as hard disks, floppy disks, and magnetic tapes), optical data storage media (such as CD-ROMs or DVD-ROMs), magneto-optical media (such as floppy disks), and hardware devices specifically configured to store and implement program instructions (such as read-only memory (ROM), random access memory (RAM), flash memory, etc.). Examples of program instructions include not only machine language code formatted by a compiler but also high-level language code that can be implemented by a computer using an interpreter. The hardware device may be configured to be operated by one or more software modules or vice versa to implement the processing according to the present invention.
[0550] Although the invention has been described with respect to specific items (such as detailed elements) and limited embodiments and drawings, these are provided only to aid in a more complete understanding of the invention, and the invention is not limited to the embodiments described above. Those skilled in the art will understand that various modifications and changes can be made to the above description.
[0551] Therefore, the spirit of the present invention should not be limited to the above embodiments, and the entire scope of the appended claims and their equivalents shall fall within the scope and spirit of the present invention.
[0552] Industrial applicability
[0553] This invention can be used to encode / decode images.
Claims
1. A method for decoding an image, comprising: Obtain intra-prediction mode information for the current block from the bitstream; When the current block is not square, the intra prediction mode of the current block is obtained by shifting the number of the intra prediction mode indicated by the intra prediction mode information by a predetermined offset based on whether the intra prediction mode indicated by the intra prediction mode information is outside the threshold determined based on the combination of the width and height of the current block. Reference samples for deriving the intra-prediction mode of the current block; Generate a prediction block for the current block based on the intra-prediction mode of the current block and the reference sample points; The reconstructed block of the current block is generated based on the predicted block; as well as Apply the filter to the reconstruction block.
2. The method of claim 1, further comprising: The transform coefficients of the current block are generated by inverse quantization, and the residual block of the current block is generated by inverse transforming the transform coefficients of the current block.
3. The method as described in claim 2, wherein, The step of generating a reconstruction block includes generating the reconstruction block based on the sum of the residual block and the prediction block of the current block.
4. The method of claim 2, wherein, The step of performing an inverse transformation on the transform coefficients of the current block includes: performing a second inverse transformation on the transform coefficients of the current block based on the transform matrix determined by the intra-prediction mode of the current block.
5. A method for encoding an image, comprising: Determine the intra-prediction mode for the current block; When the current block is not square, shift the number of the intra prediction mode of the current block by a predetermined offset if the intra prediction mode of the current block is outside a threshold determined by the combination of the width and height of the current block. A predicted block for the current block is generated by performing intra-frame prediction on the current block. Generate the residual block of the current block based on the predicted block; as well as Based on the intra-prediction mode number of the current block after shifting and the residual block, a bitstream including the intra-prediction mode information of the current block is generated. The step of generating the prediction block includes: deriving reference samples for the intra-frame prediction.
6. The method of claim 5, wherein, The steps for generating a bitstream include: The transformation coefficients of the residual block are generated through transformation and quantization processes.
7. The method of claim 6, wherein, The transformation process is performed based on the transformation matrix determined by the intra-prediction mode of the current block.
8. A method for providing video data to a video decoding apparatus, the method comprising: The video data is encoded into a bitstream by performing a video encoding method; as well as The bitstream is sent to the video decoding device. The video encoding method includes: Determine the intra-prediction mode for the current block; When the current block is not square, shift the number of the intra prediction mode of the current block by a predetermined offset if the intra prediction mode of the current block is outside a threshold determined by the combination of the width and height of the current block. A predicted block for the current block is generated by performing intra-frame prediction on the current block. Generate the residual block of the current block based on the predicted block; and Based on the intra-prediction mode number of the current block after shifting and the residual block, a bitstream including the intra-prediction mode information of the current block is generated. The step of generating the prediction block includes: deriving reference samples for the intra-frame prediction.
Citation Information
Patent Citations
Applying non-square transforms to video data
CN103636215A
Restriction of prediction units in b slices to uni-directional inter prediction
CN104160704A
