Device and method for intra-prediction

Abstract

To provide devices and methods relating to the field of picture processing, for example still picture / image and / or video picture / image coding.SOLUTION: In particular, the present invention relates to a device and corresponding method for intra-predicting a prediction block of a video image. The device is configured to select a directional intra-prediction mode from a set of directional intra-prediction modes, and each directional intra-prediction mode corresponds to a different intra-prediction angle. Further, the present device is configured to select a filter from a set of filters based on the selected directional intra-prediction mode. Further, the device is configured to determine, for a given prediction sample of the prediction block, a reference sample from a set of reference samples based on the selected directional intra-prediction mode, and apply the selected filter to the determined reference sample.SELECTED DRAWING: Figure 8

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of image processing, for example, still pictures / images and / or video pictures / images coding. In particular, the present invention relates to a device for intra prediction, i.e., for intra predicting a prediction block of a video image. The device may be a video picture encoder or a video image decoder, or a part thereof. The device is particularly configured to perform directional intra prediction of a prediction block. The present invention also relates to a corresponding intra prediction method.

Background Art

[0002] Video coding (video encoding and decoding) is used in a wide range of digital video applications such as, for example, broadcast digital TV, video transmission via the Internet and mobile networks, real-time conversation applications such as video chat, video conferencing, DVDs and Blu-ray discs, video content acquisition and editing systems, and security-use camcorders.

[0003] Since the development of block-based hybrid video coding techniques in the H.261 standard in 1990, new video coding techniques and tools have been developed, forming the basis for new video coding standards. One of the goals of most video coding standards has been to achieve bitrate reduction compared to previous standards without sacrificing image quality. Further video coding standards include MPEG-1 video, MPEG-2 video, ITU-T H.262 / MPEG-2, ITU-T H.263, ITU-T H.264 / MPEG-4, Part 10, Advanced Video Coding (AVC), ITU-T H.265, High Efficiency Video Coding (HEVC), as well as extensions of these standards, for example, scalability and / or three-dimensional (3D) extensions.

[0004] Video compression can achieve the desired bitrate reduction, but it is a complex task. Specifically, video compression is constrained by two conflicting parameters: compression efficiency and computational complexity. Video coding standards such as ITU-T H.264 / AVC or ITU-T H.265 / HEVC provide a good trade-off between these parameters. Therefore, support for video coding standards is an essential requirement for almost any video compression application.

[0005] State-of-the-art video coding standards are based on dividing the source picture into blocks. The processing of these blocks varies depending on their size, spatial position, and the coding mode specified by the encoder.

[0006] The coding mode can be classified into two groups: intra prediction mode and inter prediction mode, depending on the type of prediction. The intra prediction mode generates reference samples using the pixels of the same picture to calculate the predicted values of the pixels in the reconstructed block. Intra prediction is also called spatial prediction. The inter prediction mode is designed for temporal prediction and uses reference samples from the previous or next picture to predict the pixels in the block of the current picture.

[0007] After the prediction stage, transform coding is performed on the prediction error, which is the difference between the original signal and its prediction. Then, the transform coefficients and side information are encoded using an entropy coder (e.g., CABAC in the case of AVC / H.264 and HEVC / H.265). The recently adopted ITU-T H.265 / HEVC standard (ISO / IEC 23008-2:2013, "Information technology-High efficiency coding and media delivery in heterogeneous environments-Part 2:High efficiency video coding", November 2013) declares a set of state-of-the-art video coding tools that provide a reasonable trade-off between coding efficiency and computational complexity. The overview of the ITU-T H.265 / HEVC standard is given by Gary J. Sullivan, "Overview of the High Efficiency Video Coding (HEVC) Standard", in IEEE Transactions on Circuits and Systems for Video Technology, Vol. 22, No. 12, December 2012, the entire content of which is incorporated herein by reference.

[0008] Similar to the ITU-T H.264 / AVC video coding standard, the HEVC / H.265 video coding standard specifies dividing the source picture into blocks, such as coding units (CUs). Each CU can be further divided into smaller CUs or prediction units (PUs). A PU can be intra-predicted or inter-predicted depending on the type of processing applied to the pixels of the PU. In the case of inter-prediction, the PU represents a region of pixels that is processed by motion compensation using the motion vectors specified for the PU. In intra-prediction, the neighboring pixels of adjacent blocks are used as reference samples for predicting the current block.

[0009] The PU specifies a prediction mode selected from the set of intra prediction modes of all transform units (TUs) included in this PU. That is, the intra prediction mode is the same for each TU of the PU. The TUs can have various sizes (e.g., 4×4, 8×8, 16×16, and 32×32 pixels) and can be processed in various ways. In the case of TUs, transform coding is performed, that is, the prediction error is transformed and quantized by a discrete cosine transform or a discrete sine transform (applied to intra-coded blocks in the HEVC / H.265 standard). Therefore, the reconstructed pixels include quantization noise (which may be revealed as, for example, blockiness between units, ringing artifacts associated with sharp edges, etc.) that in-loop filters such as DBF, SAO, and ALF attempt to suppress. Through advanced prediction coding (such as motion compensation and intra prediction) and partitioning techniques (e.g., Quad-Tree (QT) for CUs and PUs, and Residual Quad-Tree (RQT) for TUs in the HEVC / H.265 standard, and Quad-Tree and Binary Tree (QTBT) in the case of the Joint Exploration Model (JEM) reference software from version JEM-3.0 onwards), the standardization committee has been able to significantly reduce the redundancy in the PUs. The basic difference between the QT and QTBT partitioning mechanisms is that in the latter mechanism, by using partitioning based on both a quad-tree and a binary tree, not only square but also rectangular blocks are made effective. The present invention relates to directional intra prediction and introduces a new modification to the directional intra prediction mode.

[0010] According to the HEVC / H.265 standard, 35 intra prediction modes are available. As shown in FIG. 9, this set includes the following modes. · Planar mode (the index of the intra prediction mode is 0), · DC mode (the index of the intra prediction mode is 1), · The directivity mode indicated by the solid arrows in FIG. 9 (the range of the index values of the intra prediction mode is from 2 to 34). The set of directivity intra prediction modes was extended to a maximum of 65 modes (i.e., approximately twice) by reducing the angular step between the directivity intra prediction modes by half. These additional modes are indicated by the dashed arrows in FIG. 9.

[0011] In the JEM-3.0 software, a new partitioning mechanism known as QTBT was proposed. As shown in FIG. 10, QTBT partitioning can provide not only square but also rectangular blocks. Of course, some signaling overhead and increased computational complexity on the encoder side are the price of QTBT partitioning compared to the conventional QT-based partitioning used, for example, in the HEVC / H.265 standard. Nevertheless, the QTBT-based partitioning has better segmentation properties and thus shows significantly higher coding efficiency than the conventional QT.

[0012] However, when introducing QTBT, the set of available directivity intra prediction modes was not changed accordingly. In particular, as shown in FIG. 11, the asymmetry of the rectangular blocks was not considered. Therefore, the same number of reference samples are used along both the short side and the long side of the rectangular block. In the current implementation of the QTBT framework, the number of directivity intra prediction modes does not depend on the aspect ratio of the block nor on the actual availability of the reference samples. As a result, there are highly unlikely reference samples used for the short side of the rectangular block while there are likely reference samples not used for the long side.

[0013] In particular, as shown in FIG. 12, in this specification, the terms "vertical block" ("vertical direction of the block") and "horizontal block" ("horizontal direction of the block") are applied to the rectangular blocks generated by the QTBT framework. FIG. 12 shows, in particular, (a) a horizontal block and (b) a vertical block.

[0014] In the submitted JVET-D0113, it is further proposed to apply a mechanism in which the number of directional intra prediction modes is adjustable. In particular, it is proposed to further increase the number of directional intra prediction modes to 131 for large block sizes and to decrease the number of directional intra prediction modes for small block sizes. The switching of the number of directional intra prediction modes based on the block size is controlled by two thresholds signaled in the SPS as minus 4 and minus 6 respectively from the log2 value. The first threshold indicates the largest block size that can have 35 intra prediction mode directions, the second threshold indicates the largest block size that can have 67 intra prediction mode directions, and all other blocks use 131 intra prediction mode directions. In the default setting, the thresholds are signaled as 4 and 6 respectively, and are set to 5 and 8 for high-resolution images.

[0015] In the implementation form, the directional intra prediction mode index is always represented by a 131-mode range regardless of the number of directional intra prediction modes actually used. Among the 67 intra prediction modes actually used, only every other angular (directional) mode is permitted, and among the 35 modes, only every fourth angular (directional) mode is permitted. Therefore, as explained in FIG. 13, when the current block uses less than 131 intra prediction mode directions, during intra prediction mode signaling, the intra prediction mode of the adjacent block needs to be rounded to the nearest, second, or fourth angular intra prediction mode. This conversion is performed by applying one or two left or right shifts to the intra prediction mode. When the mode is not MPM, the mode signaling follows the same process as JEM-3.0, but the number of intra prediction modes is different. The planar mode and the DC mode remain unchanged and do not require mode conversion. To accommodate the increase in the number of intra prediction modes, the 4-tap intra filter has been extended from 1 / 32 to 1 / 64 fractional pels.

[0016] Furthermore, recently, techniques have been proposed to address the problem of how many directional intra prediction modes should be included in the intra prediction mode set for rectangular blocks. As shown in FIG. 14, according to the proposed technique, the set of directional intra prediction modes can be extended according to the aspect ratio of the prediction block, and signals can be sent by mapping the added directional intra prediction modes to the conventional subset.

[0017] FIG. 15 shows, in this regard, the case of intra prediction in a diagonal direction having an angle equal to 45° associated with the directional intra prediction mode. The corresponding HEVC intra mode indices in this case are 2 (from the lower left) and 35 (from the upper right).

[0018] However, when a similar intra prediction mechanism is applied to an angle less than 45°, i.e., in the case of the extended directional intra prediction mode, the situation becomes as shown in FIG. 16. That is, when the intra prediction direction is specified to be an acute angle (i.e., less than 45°), obvious discontinuities may be observed in the prediction. The cause of these discontinuities is, in particular, that the difference in the positions of the reference samples between two adjacent lines of the prediction samples may be larger than one reference sample. This problem is related to the processing of the reference samples and the method of performing intra prediction interpolation.

Prior Art Documents

Non-Patent Documents

[0019]

Non-Patent Document 1

Non-Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0020] In consideration of the above-described implementation forms, the present invention aims to further improve hybrid video coding. In particular, the present invention aims to provide a device and method for improved intra prediction of prediction blocks in video images. The present invention particularly aims at additional coding gain without adding hardware and computational complexity. Specifically, the present invention is intended to overcome the above-described problems occurring at acute angles of less than 45°, that is, to suppress discontinuities at these acute angles. The present invention should be easily implemented in a codec using a conventional directional intra prediction mechanism.

Means for Solving the Problems

[0021] [[ID=…]]The object of the present invention is solved according to embodiments of the present invention defined by the features of the independent claims. Further advantageous implementation forms of the embodiments are defined by the features of the dependent claims.

[0022] In particular, the present invention proposes to reduce discontinuities by extending the length of the filter at acute angles of intra prediction, that is, less than 45°. This solution is applicable mainly to rectangular blocks created by split frameworks such as QTBT and MTT.

[0023] A first aspect of the present invention provides a device for intra-predicting a prediction block of a video image, the device being configured to select a directional intra-prediction mode from a set of directional intra-prediction modes, each directional intra-prediction mode corresponding to a different intra-prediction angle, select a filter from a set of filters based on the selected directional intra-prediction mode, determine a reference sample from a set of reference samples for a given prediction sample of the prediction block based on the selected directional intra-prediction mode, and apply the selected filter to the determined reference sample.

[0024] The device according to the first aspect provides the following advantages: · It is possible to achieve additional coding gain. · It can be used in many potential applications in a hybrid video coding paradigm that is compatible with the HM software and VPX video codec family, which are the state-of-the-art video coding framework and the next-generation video coding framework respectively, as well as the JEM and VTM software and the VPX / AV1 video codec family. · It keeps the hardware and computational complexity low. · The device can be easily implemented in a codec that employs a conventional directional intra-prediction mechanism.

[0025] In particular, by selecting the filter length according to the angle, the above-mentioned problem of the intra-prediction angle of less than 45° acute angle can be overcome. When the distance between the two reference samples used for the intra-prediction of two adjacent prediction samples becomes large and the two reference samples are no longer adjacent to each other, increasing the length of the selected filter can avoid discontinuity. For angles greater than 45° where the reference pixel is adjacent to another pixel, the filter length can be selected to be short to retain details.

[0026] In particular, the prediction block can be a TU or a PU. The device is configured to process each prediction sample within the prediction block as described for a given prediction sample. Thus, the device is configured to perform intra prediction for the entire prediction block within the video image. A sample is the common part of a channel and a pixel in the video image. For example, each pixel of the video image can include three samples of red, green, and blue.

[0027] In an implementation of the first aspect, the device is configured to determine the length of the filter based on a selected directional intra prediction mode and to select as the filter one having at least the determined length of the filter.

[0028] Thus, the device ensures that the length of the filter is long enough to avoid discontinuity in any case.

[0029] In a further implementation of the first aspect, the set of filters includes filters having different filter lengths, particularly filters having a length spanning one, three, or five adjacent reference samples.

[0030] In a further implementation of the first aspect, each filter within the set of filters performs different smoothing on the determined reference sample and one or more adjacent reference samples when applied to the determined reference sample.

[0031] In the case of a sharper angle, for example, stronger smoothing for more adjacent reference samples can be selected by selecting the filter accordingly, while for an angle that is not sharper (or not an acute angle), for example, softer smoothing for fewer adjacent reference samples can be selected.

[0032] In a further implementation of the first aspect, the device is configured to determine an intra prediction angle corresponding to the selected directional intra prediction mode and to select a filter according to the determined intra prediction angle.

[0033] Therefore, an optimal filter can be selected for each angle of intra prediction.

[0034] In a further implementation of the first aspect, the device is configured to determine an intra prediction angle corresponding to the selected directional intra prediction mode, to specify a further reference sample from a set of reference samples for further prediction samples of the prediction block based on the selected directional intra prediction mode, to determine a distance between the determined reference sample and another reference sample, and to select a filter according to the determined distance.

[0035] The device may be configured to execute a filter selection algorithm that takes the selected directional intra prediction mode as an input to obtain the intra prediction angle as an output. The device may determine the intra prediction angle based on the index of the selected directional intra prediction mode. Further, the device may be configured to determine the angle based on the aspect ratio of the prediction block.

[0036] A further reference sample may be specified for a further prediction sample in the same way as the determined reference sample is determined for a given prediction sample, in particular based on the intra prediction direction of the selected mode, i.e., the intra prediction angle. The distance between the determined reference sample and the further reference sample may be derived based on the distance between the given prediction sample and the further prediction sample in the prediction block and the intra prediction angle associated with the selected mode. The distance may be determined as an integer or a fraction of the reference sample.

[0037] If the determined distance is smaller, a filter with a longer filter length can be selected; if the determined distance is larger, a filter with a shorter filter length can be selected. In particular, a filter having at least the filter length of the determined distance can be selected. If there is no selectable filter having at least the filter length of the determined distance, a filter having the maximum filter length in the filter set can be selected.

[0038] Therefore, a filter can be selected so that the distance between reference samples does not cause discontinuity after intra prediction.

[0039] In a further implementation of the first aspect, the device is configured to select the same filter for each directional intra prediction mode selected from a first subset of directional intra prediction modes and to select different filters for each directional intra prediction mode selected from a second subset of directional intra prediction modes.

[0040] For example, the first subset may include directional intra prediction modes associated with intra prediction angles of 45° or more, and the second subset may include directional intra prediction modes associated with intra prediction angles of less than 45°.

[0041] In a further implementation of the first aspect, the device is configured to intra predict a given prediction sample directly from the determined reference sample, and the device is configured to apply the selected filter to the determined reference sample before or during the intra prediction of the given prediction sample.

[0042] In a further implementation of the first aspect, the device is configured to generate a transposed reference sample by interpolating a reference sample determined based on a selected intra prediction mode, and to perform intra prediction of a given prediction sample from the transposed reference sample. The device is configured to apply a selected filter to the determined reference sample before or during the generation of the transposed reference sample.

[0043] In a further implementation of the first aspect, the device is configured to transpose each reference sample within a set of reference samples, such that the rows of the reference samples become the columns of the transposed reference samples and the columns of the reference samples become the rows of the transposed reference samples.

[0044] In a further implementation of the first aspect, the reference samples of the set of reference samples are arranged in the rows of the video image adjacent above and diagonally above the prediction block, and / or in the columns of the video image adjacent below and diagonally below the prediction block.

[0045] In a further implementation of the first aspect, the device is configured to encode and / or decode a video image, or the device is a video encoder and / or a video decoder.

[0046] For example, the device of the first aspect can be included in or be the intra prediction unit of an encoder or a decoder.

[0047] A second aspect of the present invention provides a method for intra predicting a prediction block of a video image, the method comprising the steps of: selecting a directional intra prediction mode from a set of directional intra prediction modes, each directional intra prediction mode corresponding to a different intra prediction angle; selecting a filter from a set of filters based on the selected directional intra prediction mode; determining a reference sample from a set of reference samples based on the selected directional intra prediction mode for a given prediction sample of the prediction block; and applying the selected filter to the determined reference sample.

[0048] In an implementation of the second aspect, the method further comprises the steps of: determining a filter length based on the selected directional intra prediction mode; and selecting as the filter one having at least the determined filter length.

[0049] In a further implementation of the second aspect, the set of filters includes filters having different filter lengths, particularly filters having a length spanning one, three, or five adjacent reference samples.

[0050] In a further implementation of the second aspect, each filter in the set of filters, when applied to the determined reference sample, performs a different smoothing on the determined reference sample and one or more adjacent reference samples.

[0051] In a further implementation of the second aspect, the method further comprises the steps of: determining an intra prediction angle corresponding to the selected directional intra prediction mode; and selecting a filter according to the determined intra prediction angle.

[0052] In a further implementation of the second aspect, the method includes determining an intra prediction angle corresponding to a selected directional intra prediction mode; for further prediction samples of a prediction block, specifying further reference samples from a set of reference samples based on the selected directional intra prediction mode; determining a distance between the determined reference samples and another reference sample; and selecting a filter according to the determined distance.

[0053] In a further implementation of the second aspect, the method includes selecting the same filter for each directional intra prediction mode selected from a first subset of directional intra prediction modes, and selecting different filters for each directional intra prediction mode selected from a second subset of directional intra prediction modes.

[0054] In a further implementation of the second aspect, the method includes directly intra predicting a given prediction sample from the determined reference samples, and the method includes applying the selected filter to the determined reference samples before or during the intra prediction of the given prediction sample.

[0055] In a further implementation of the second aspect, the method includes generating a transposed reference sample by interpolating the reference samples determined based on the selected intra prediction mode, and intra predicting a given prediction sample from the transposed reference sample, and the method includes applying the selected filter to the determined reference samples before or during the generation of the transposed reference sample.

[0056] In a further implementation of the second aspect, the method includes transposing each reference sample in the set of reference samples, where the rows of the reference samples become the columns of the transposed reference samples and the columns of the reference samples become the rows of the transposed reference samples.

[0057] In a further implementation of the second aspect, the reference samples of the set of reference samples are arranged in a row of the video image adjacent to and above and diagonally above the prediction block, and / or arranged in a column of the video image adjacent to and below and diagonally below the prediction block.

[0058] In a further implementation of the second aspect, the method is executed to encode and / or decode a video image, or the method is executed in a video encoder and / or a video decoder.

[0059] The method of the second aspect and its implementation achieve the same advantages and effects as those described above for the device of the first aspect and its respective implementations.

[0060] Note that all devices, elements, units, and means described in this application can be implemented in software or hardware elements, or any combination of them. All steps executed by various entities described in this application, as well as the functions described as being executed by various entities, are intended to mean that each entity is adapted or configured to execute its respective steps and functions. Even if in the description of the following specific embodiments, the specific functions or steps to be executed by an external entity are not reflected in the description of the specific detailed elements of that entity that executes that specific step or function, it should be obvious to those skilled in the art that these methods and functions can be implemented in respective software or hardware elements, or any combination of them.

[0061] Details of one or more embodiments are set forth in the accompanying drawings and the description below. Other features, objects, and advantages will be apparent from the description, drawings, and claims.

[0062] In the following embodiments of the present invention, it will be described in more detail with reference to the accompanying drawings and figures.

Brief Description of the Drawings

[0063]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

[0064] In the following description, reference is made to the accompanying drawings which form a part hereof and which illustrate, by way of example, specific aspects of embodiments of the invention or specific aspects in which embodiments of the invention may be used. It is understood that embodiments of the invention may be used in other aspects and may include structural or logical changes not shown in the drawings. Accordingly, the following detailed description should not be construed in a limiting sense, and the scope of the invention is defined by the appended claims.

[0065] For example, it is understood that the disclosure related to the described method may also apply to a corresponding device or system configured to perform the method, and vice versa. For example, if one or more specific method steps are described, the corresponding device may include one or more units, such as functional units, for performing the one or more described method steps (e.g., one unit for performing one or more steps, or multiple units each performing one or more of a plurality of steps), even if such one or more units are not explicitly described or illustrated in the figures. On the other hand, for example, if a specific device is described based on one or more units, such as functional units, the corresponding method may include one step for performing the function of one or more units (e.g., one step for performing the function of one or more units, or multiple steps each performing one or more of the functions of a plurality of units), even if such one or more steps are not explicitly described or illustrated in the figures. Further, it is understood that the features of the various exemplary embodiments and / or aspects described herein can be combined with each other unless otherwise specified.

[0066] Video coding generally refers to the processing of a series of pictures that form a video or a video sequence. Instead of the term "picture", the terms "frame" or "image" can be used as synonyms in the field of video coding. Video coding includes two parts: video encoding and video decoding. Video encoding is performed on the source side and generally includes processing the original video picture (e.g., by compression) to reduce the amount of data required to represent the video picture (for more efficient storage and / or transmission). Video decoding is performed on the destination side and generally includes the reverse process compared to the encoder for reconstructing the video picture. Embodiments referring to the "coding" of a video picture (or, as will be described later, a video image or generally a picture) are to be understood as relating to both the "encoding" and the "decoding" of the video picture. The combination of the encoding part and the decoding part is also called a CODEC (COding and DECoding).

[0067] In the case of reversible video coding, the original video picture can be reconstructed, i.e., (assuming no transmission loss or other data loss during storage or transmission) the reconstructed video picture has the same quality as the original video picture. In the case of irreversible video coding, in order to reduce the amount of data representing the video picture, further compression, e.g., by quantization, is performed and cannot be fully reconstructed at the decoder, i.e., the quality of the reconstructed video picture is degraded or worsened compared to the quality of the original video picture.

[0068] Several video coding standards since H.261 belong to the group of "irreversible hybrid video coders" (i.e., combining spatial and temporal prediction in the sample domain and 2D transform coding for applying quantization in the transform domain). Each picture of a video sequence is typically divided into a set of non-overlapping blocks, and coding is typically done at the block level. In other words, in the encoder, video is typically processed, i.e., encoded, at the block (video block) level by, for example, using spatial (intra-picture) prediction and temporal (inter-picture) prediction to generate prediction blocks, subtracting the prediction blocks from the current block (the block being currently processed / intended to be processed) to obtain residual blocks, transforming the residual blocks to reduce (compress) the amount of data to be transmitted, and quantizing the residual blocks in the transform domain. On the other hand, in the decoder, the reverse process compared to the encoder is applied to the encoded or compressed blocks to reconstruct the current block for presentation. Further, the encoder duplicates the decoder processing loop so that both generate the same prediction (e.g., intra prediction and inter prediction) and / or reconstruction for processing, i.e., coding, subsequent blocks.

[0069] Video picture processing (also called video processing) and still picture processing (the term including processing with coding) share many concepts and techniques or tools. Therefore, hereinafter, the term "picture" is used to refer to video pictures of a video sequence (as described above) and / or still pictures in order to avoid unnecessary repetition and distinction between video pictures and still pictures when not necessary. If the description only refers to a still picture (or, still image), the term "still picture" shall be used.

[0070] Before explaining the embodiments of the present invention in more detail based on FIGS. 4 to 11 below, an encoder 100, a decoder 200, and a coding system 300 for implementing the embodiments of the present invention will be described based on FIGS. 1 to 3.

[0071] FIG. 3 is a conceptual or schematic block diagram showing an embodiment of a coding system 300, for example, a picture coding system 300. The coding system 300 includes a source device 310 configured to provide encoded data 330, for example, an encoded picture 330, to a destination device 320 for decoding the encoded data 330.

[0072] The source device 310 includes an encoder 100 or an encoding unit 100, and additionally, i.e., optionally, may include a picture source 312, a preprocessing unit 314, for example, a picture preprocessing unit 314, and a communication interface or communication unit 318.

[0073] The picture source 312 may include, for example, any type of picture capture device for capturing real-world pictures, and / or any type of picture generation device such as a computer graphics processor for generating computer animation pictures, or any type of device for acquiring and / or providing real-world pictures, computer animation pictures (e.g., screen content, virtual reality (VR) pictures), and / or any combination thereof (e.g., augmented reality (AR) pictures). Hereinafter, unless otherwise specified, all these types of pictures and any other types of pictures will be referred to as "pictures", and the foregoing explanation regarding the term "picture" covering "video pictures", "video images", "still pictures", and "still images" will continue to apply unless explicitly specified otherwise.

[0074] (Digital) pictures may be, and may be regarded as, two-dimensional arrays or matrices of samples having intensity values. Samples within the array may also be referred to as pixels (short for picture elements) or pels. The number of samples in the horizontal and vertical directions (or axes) of the array or picture defines the size and / or resolution of the picture. For color representation, typically three color components are used, i.e., a picture may be represented or may also include three sample arrays. In the RGB format or color space, a picture is composed of corresponding red, green, and blue sample arrays. However, in video coding, each pixel typically includes a luminance component, denoted by Y (and sometimes L may also be used instead), and two chrominance components, denoted by Cb and Cr, and is represented in YCbCr. The luminance component Y represents the brightness or gray-level intensity (such as in a grayscale picture), and the two chrominance components Cb and Cr represent the chrominance or color information components. Thus, a picture in YCbCr format includes a luminance sample array of luminance sample values (Y) and two chrominance sample arrays of chrominance values (Cb and Cr). A picture in RGB format may be converted or transformed to YCbCr format, and vice versa, and this process is also known as color conversion or color transformation. If the picture is monochrome, the picture may include only a luminance sample array.

[0075] The picture source 312 can be, for example, a camera for capturing pictures, a memory that includes or stores previously captured or generated pictures, such as a picture memory, and / or any kind of interface (internal or external) for acquiring or receiving pictures. The camera can be, for example, a local or integrated camera integrated into the source device, and the memory can be, for example, a local or integrated memory integrated into the source device. The interface can be, for example, an external video source, such as an external picture capture device like a camera, an external memory, or an external picture generation device, such as an external computer graphics processor, a computer or a server, and can be an external interface for receiving pictures from it. The interface can be any kind of interface, such as a wired or wireless interface, an optical interface, according to any unique or standardized interface protocol. The interface for acquiring the picture data 312 can be the same interface as or a part of the communication interface 318.

[0076] Distinguished from the preprocessing unit 314 and the processing executed by the preprocessing unit 314, the picture or picture data 313 may also be called raw picture or raw picture data 313.

[0077] The preprocessing unit 314 is configured to receive the (raw) picture data 313 and perform preprocessing on the picture data 313 to obtain the preprocessed picture 315 or the preprocessed picture data 315. The preprocessing executed by the preprocessing unit 314 may include, for example, trimming, color format conversion (e.g., from RGB to YCbCr), color correction, or noise removal.

[0078] The encoder 100 is configured to receive the pre - processed picture data 315 and provide the encoded picture data 171 (for example, further details are described based on FIG. 1).

[0079] The communication interface 318 of the source device 310 is configured to receive the encoded picture data 171, directly transmit it to another device, such as the destination device 320 or any other device, for storage or direct reconstruction, or process the encoded picture data 171 before storing the encoded data 330 and / or before transmitting the encoded data 330 to another device, such as the destination device 320 or any other device for decoding or storage.

[0080] The destination device 320 includes a decoder 200 or a decoding unit 200, and further, optionally, may include a communication interface or communication unit 322, a post - processing unit 326, and a display device 328.

[0081] The communication interface 322 of the destination device 320 is configured to receive the encoded picture data 171 or the encoded data 330, for example, directly from the source device 310 or from any other source, such as a memory, for example, an encoded picture data memory.

[0082] The communication interface 318 and the communication interface 322 are each configured to transmit and receive the encoded picture data 171 or the encoded data 330 via a direct communication link between the source device 310 and the destination device 320, such as a direct wired or wireless connection, or via any type of network, such as a wired or wireless network, or any combination thereof, or any type of private and public network, or any combination of those types.

[0083] The communication interface 318 may be configured to package, for example, the encoded picture data 171 into an appropriate format, such as a packet, for transmission via a communication link or communication network, and may further include data loss protection and data loss recovery.

[0084] The corresponding communication interface 322 of the communication interface 318 may be configured to unpack the encoded data 330 to obtain the encoded picture data 171, and may be further configured to perform data loss protection and data loss recovery, including, for example, error concealment.

[0085] Both the communication interface 318 and the communication interface 322 may be configured as a unidirectional communication interface or a bidirectional communication interface, as indicated by the arrow of the encoded picture data 330 in FIG. 3 pointing from the source device 310 to the destination device 320, and may be configured to, for example, send and receive messages, for example, set up a connection, confirm and / or retransmit lost or delayed data including picture data, and exchange any other information related to the communication link and / or data transmission, for example, the transmission of the encoded picture data.

[0086] The decoder 200 is configured to receive the encoded picture data 171 and provide the decoded picture data 231 or the decoded picture 231 (for example, further details are described based on FIG. 2).

[0087] The post-processor 326 of the destination device 320 is configured to post-process the decoded picture data 231, such as the decoded picture 231, to obtain post-processed picture data 327, such as the post-processed image 327. The post-processing executed by the post-processing unit 326 may include, for example, color format conversion (e.g., from YCbCr to RGB), color correction, trimming, or resampling, or any other processing for preparing the decoded picture data 231 for display, for example, by the display device 328.

[0088] The display device 328 of the destination device 320 is configured to receive, for example, the post-processed picture data 327 for displaying a picture to a user or viewer. The display device 328 can be or include any kind of display for representing the reconstructed picture, such as an integrated or external display or monitor. The display can include, for example, a cathode ray tube (CRT), a liquid crystal display (LCD), a plasma display, an organic light-emitting diode (OLED) display, or any other kind of display, beam, or hologram (3D).

[0089] FIG. 3 shows the source device 310 and the destination device 320 as separate devices, but embodiments of the devices may also include both or both functions, the source device 310 or corresponding functions, and the destination device 320 or corresponding functions. In such embodiments, the source device 310 or corresponding functions and the destination device 320 or corresponding functions can be implemented using the same hardware and / or software, or by separate hardware and / or software or any combination thereof.

[0090] As will be apparent to those skilled in the art based on the description, the functions of different units, or the presence and (exact) partitioning of functions within the source device 310 and / or destination device 320 as shown in FIG. 3, may vary depending on the actual devices and applications.

[0091] Accordingly, the source device 310 and destination device 320 shown in FIG. 3 are merely exemplary embodiments of the present invention, and embodiments of the present invention are not limited to those shown in FIG. 3.

[0092] The source device 310 and destination device 320 may include any of a wide range of devices, such as any type of handheld or fixed device, for example, a notebook or laptop computer, a mobile phone, a smartphone, a tablet or tablet computer, a camera, a desktop computer, a set-top box, a television, a display device, a digital media player, a video game console, a video streaming device, a broadcast receiver device, etc., and may not use an operating system or may use any type of operating system.

[0093] Encoder and Encoding Method FIG. 1 shows a schematic / concept block diagram of an embodiment of an encoder 100, such as a picture encoder 100, comprising an input 102, a residual calculation unit 104, a conversion unit 106, a quantization unit 108, an inverse quantization unit 110, an inverse conversion unit 112, a reconstruction unit 114, a buffer 118, a loop filter 120, a decoded picture buffer (DPB) 130, a prediction unit 160 (including an inter-estimation unit 142, an inter-prediction unit 144, an intra-estimation unit 152, and an intra-prediction unit 154), a mode selection unit 162, an entropy encoding unit 170, and an output 172. The video encoder 100 as shown in FIG. 1 may also be referred to as a hybrid video encoder or a video encoder by a hybrid video codec.

[0094] For example, the residual calculation unit 104, the conversion unit 106, the quantization unit 108, and the entropy encoding unit 170 form the forward signal path of the encoder 100. On the other hand, for example, the inverse quantization unit 110, the inverse conversion unit 112, the reconstruction unit 114, the buffer 118, the loop filter 120, the decoded picture buffer (DPB) 130, the inter prediction unit 144, and the intra prediction unit 154 form the reverse signal path of the encoder, and the reverse signal path of the encoder corresponds to the signal path of the decoder (see the decoder 200 in FIG. 2).

[0095] The encoder 100 is configured to receive, for example, by the input 102, a picture 101 or a picture block 103 of the picture 101, for example, a sequence of pictures forming a video or a video sequence. (In particular, in video coding, to distinguish the current picture from other pictures, for example, previously encoded and / or decoded pictures of the same video sequence, i.e., the video sequence including the current picture) The picture block 103 may also be referred to as the current picture block or the picture block to be encoded, and the picture 101 may be referred to as the current picture or the picture to be encoded.

[0096] Residual calculation The residual calculation unit 104 is configured to calculate the residual block 105 based on the picture block 103 and the prediction block 165, for example, by subtracting the sample values of the prediction block 165 from the sample values of the picture block 103 for each sample (for each pixel) to obtain the residual block 105 in the sampled domain. (Further details about the prediction block 165 will be provided later).

[0097] Conversion The transformation unit 106 is configured to apply a discrete cosine transform (DCT) or a discrete sine transform (DST) to the sample values of the residual block 105 in order to obtain transformed coefficients 107 in a transformation, for example, a spatial frequency transformation or a linear spatial (frequency) transformation, such as in a transform domain. The transformed coefficients 107 may also be referred to as transformed residual coefficients and represent the residual block 105 in the transform domain.

[0098] The transformation unit 106 may be configured to apply an integer approximation of DCT / DST, such as a core transformation specified in HEVC / H.265. Compared to the orthonormal DCT transform, such an integer approximation is typically scaled by a specific coefficient. To maintain the norm of the residual block processed by the forward and inverse transforms, an additional scaling coefficient is applied as part of the transformation process. The scaling coefficient is usually selected based on specific constraints such as a trade-off between the scaling coefficient which is a power of 2 in a shift operation, the bit depth of the transformed coefficients, accuracy, and implementation cost. A specific scaling coefficient is specified, for example, for the inverse transform by the inverse transformation unit 212 in the decoder 200 (and by the corresponding inverse transformation unit 112 in the encoder 100, for example), and the corresponding scaling coefficient for the forward transformation by, for example, the transformation unit 106 in the encoder 100 may be specified accordingly.

[0099] Quantization The quantization unit 108 is configured to quantize the transformed coefficient 107, for example, by applying scalar quantization or vector quantization, in order to obtain the quantized coefficient 109. The quantized coefficient 109 can also be referred to as the quantized residual coefficient 109. For example, in the case of scalar quantization, various scalings can be applied to achieve finer or coarser quantization. A smaller quantization step size corresponds to finer quantization, and a larger quantization step size corresponds to coarser quantization. The applicable quantization step size can be indicated by a quantization parameter (QP). The quantization parameter can be, for example, an index to a predefined set of applicable quantization step sizes. For example, a small quantization parameter can correspond to finer quantization (small quantization step size), a large quantization parameter can correspond to coarser quantization (large quantization step size), or vice versa. Quantization can include division by the quantization step size, and for example, the corresponding inverse dequantization by the inverse quantization 110 can include multiplication by the quantization step size. Embodiments according to HEVC can be configured to use the quantization parameter to determine the quantization step size. Generally, the quantization step size can be calculated based on the quantization parameter using a fixed-point approximation of an equation that includes division. Additional scaling factors can be introduced for quantization and dequantization to restore the norm of the residual block, which can be modified for the scaling used in the fixed-point approximation of the equation for the quantization step size and the quantization parameter. In one exemplary implementation, the scaling and dequantization of the inverse transform can be combined. Alternatively, a customized quantization table can be used to send a signal, for example, in a bitstream, from the encoder to the decoder. Quantization is a non-invertible operation, and the loss increases with an increase in the quantization step size.

[0100] Embodiments of the encoder 100 (or, respectively, the quantization unit 108) may be configured to output a quantization scheme and a quantization step size, for example, by corresponding quantization parameters, such that the decoder 200 may receive and apply the corresponding inverse quantization. Embodiments of the encoder 100 (or the quantization unit 108) may be configured to output the quantization scheme and the quantization step size, for example, directly, or the entropy encoded by the entropy encoding unit 170 or any other entropy encoding unit.

[0101] The inverse quantization unit 110 is configured to apply an inverse quantization of the quantization unit 108 to the quantized coefficients in order to obtain the unquantized coefficients 111, for example, based on the same quantization step size as the quantization unit 108 or using the same quantization step size as the quantization unit 108, by applying the inverse of the quantization scheme applied by the quantization unit 108. The unquantized coefficients 111 may also be referred to as unquantized residual coefficients 111 and typically correspond to the transformed coefficients 107, although they are not identical to the transformed coefficients due to losses due to quantization.

[0102] The inverse transform unit 112 is configured to apply an inverse transform of the transform applied by the transform unit 106, for example, an inverse discrete cosine transform (DCT) or an inverse discrete sine transform (DST), in order to obtain the inverse transformed block 113 in the sample domain. The inverse transformed block 113 may also be referred to as the inverse transformed unquantized block 113 or the inverse transformed residual block 113.

[0103] The reconstruction unit 114 is configured to combine the inverse transformed block 113 and the prediction block 165 in order to obtain the reconstructed block 115 in the sample domain, for example, by adding the sample values of the decoded residual block 113 and the sample values of the prediction block 165 on a sample-by-sample basis.

[0104] The buffer unit 116 (or simply "buffer" 116), e.g., the line buffer 116, is configured to buffer or store the reconstructed blocks and their respective sample values, e.g., for intra prediction and / or intra estimation. In further embodiments, the encoder may be configured to use the non-filtered reconstructed blocks and / or the respective sample values stored in the buffer unit 116 for any kind of estimation and / or prediction.

[0105] The loop filter unit 120 (or simply "loop filter" 120) is configured to filter the reconstructed block 115 by applying, e.g., a non-blocking sample adaptive offset (SAO) filter or other filter, e.g., a sharpening or smoothing filter or a collaborative filter, to obtain the filtered block 121. The filtered block 121 may also be referred to as the filtered reconstructed block 121.

[0106] An embodiment of the loop filter unit 120 (not shown in FIG. 1) may comprise a filter analysis unit and an actual filter unit, where the filter analysis unit is configured to determine the loop filter parameters of the actual filter. The filter analysis unit may be configured to apply fixed pre-determined filter parameters to the actual loop filter, adaptively select filter parameters from a set of pre-determined filter parameters, or adaptively calculate the filter parameters of the actual loop filter.

[0107] An embodiment of the loop filter unit 120 (not shown in FIG. 1) may include one or more filters (loop filter components / sub-filters), for example, one or more different types or kinds of filters, for example, connected in series, or connected in parallel, or any combination thereof, and each of the filters may include, for example, as described in the previous paragraph, a filter analysis unit for determining respective loop filter parameters, either individually or jointly with other filters of the plurality of filters.

[0108] An embodiment of the encoder 100 (each loop filter unit 120) may be configured to output loop filter parameters, for example, directly or via the entropy encoding unit 170 or any other entropy encoding unit, such that, for example, the decoder 200 may receive and apply the same loop filter parameters for decoding.

[0109] The decoded picture buffer (DPB) 130 is configured to receive and store the filtered block 121. The decoded picture buffer 130 may be further configured to store other previously filtered blocks of the same current picture or different pictures, for example, previously reconstructed pictures, for example, previously reconstructed and filtered blocks 121, for example, to provide fully previously reconstructed, i.e., decoded pictures (and corresponding reference blocks and samples) and / or partially reconstructed current pictures (and corresponding reference blocks and samples) for, for example, intra prediction and / or inter prediction.

[0110] A further embodiment of the present invention may also be configured to use previously filtered blocks and corresponding filtered sample values of the decoded picture buffer 130 for any kind of prediction or estimation, for example, intra prediction and inter prediction and prediction.

[0111] Motion Estimation and Prediction The prediction unit 160, also referred to as the block prediction unit 160, is configured to receive or obtain reference samples of the picture block 103 (the current picture block 103 of the current picture 101) and decoded or at least reconstructed picture data, for example, the same (current) picture from the buffer 116 and / or decoded picture data 231 from one or more previously decoded pictures from the decoded picture buffer 130, and to process such data for prediction, i.e., to provide a prediction block 165, which can be an inter-predicted block 145 or an intra-predicted block 155.

[0112] The mode selection unit 162 may be configured to select a prediction mode (e.g., an intra-prediction mode or an inter-prediction mode) and / or calculate the residual block 105, and select a corresponding prediction block 145 or 155 to be used as the prediction block 165 for the reconstruction of the reconstructed block 115.

[0113] Embodiments of the mode selection unit 162 may be configured to select a prediction mode (e.g., from those supported by the prediction unit 160), which provides an optimal match, or in other words, a minimum residual (a minimum residual means a high compression ratio for transmission or storage), or a minimum signaling overhead (a minimum signaling overhead means a high compression ratio for transmission or storage), or both are considered or balanced. The mode selection unit 162 may be configured to determine the prediction mode based on rate-distortion optimization (RDO), i.e., to provide a minimum rate-distortion optimization, or select a prediction mode whose associated rate-distortion at least meets the prediction mode selection criteria.

[0114] The predictive processing (e.g., the prediction unit 160) and the mode selection (e.g., by the mode selection unit 162) performed by the exemplary encoder 100 are described in more detail below.

[0115] As described above, the encoder 100 is configured to determine or select the best or optimal prediction mode from a set of (predetermined) prediction modes. The set of prediction modes may include, for example, an intra prediction mode and / or an inter prediction mode.

[0116] The set of intra prediction modes may include 32 different intra prediction modes, such as non - directional modes like the DC (or average) mode and the planar mode, or directional modes as defined, for example, in H.264, or may include 65 different intra prediction modes, such as non - directional modes like the DC (or average) mode and the planar mode, or directional modes as defined, for example, in H.265.

[0117] The set of (or possible) inter prediction modes depends on the available reference pictures (i.e., previously at least partially decoded pictures stored, for example, in the DBP 230), and other inter prediction parameters, such as whether only the entire reference picture or only a part of the search window area around the area of the current block of the reference picture is used to search for the most matching reference block, and / or whether pixel interpolation, for example, half / semi - pel, and / or quarter - pel interpolation is applied.

[0118] In addition to the above - mentioned prediction modes, a skip mode and / or a direct mode may be applied.

[0119] The prediction unit 160 may be further configured to repeatedly use, for example, quadtree partitioning (QT), binary tree partitioning (BT), or ternary tree partitioning (TT), or any combination thereof, to partition the block 103 into smaller block partitions or sub-blocks, and, for example, perform a prediction for each of the block partitions or sub-blocks. The mode selection includes the selection of the tree structure of the partitioned block 103 and the prediction mode applied to each of the block partitions or sub-blocks.

[0120] The inter-estimation unit 142, also referred to as the inter-picture estimation unit 142, is configured to receive or acquire, for inter-estimation (or, "inter-picture estimation"), the picture block 103 (the current picture block 103 of the current picture 101) and the decoded picture 231, or at least one or a plurality of previously reconstructed blocks, for example, the reconstructed blocks of one or more other / different previously decoded pictures 231. For example, the video sequence may include the current picture and the previously decoded picture 231. In other words, the current picture and the previously decoded picture 231 may be part of the sequence of pictures forming the video sequence, or may form it.

[0121] The encoder 100 may be configured to select a reference block from a plurality of reference blocks of the same or different pictures of a plurality of other pictures, and provide an offset (spatial offset) between the reference picture (or, reference picture index) and / or the position (x, y coordinates) of the reference block and the position of the current block as the inter-prediction parameter 143 for the inter-prediction unit 144. This offset is also called a motion vector (MV). Inter-estimation is called motion estimation (ME), and inter-prediction is also called motion prediction (MP).

[0122] The inter prediction unit 144 is configured to, for example, obtain, for example, receive the inter prediction parameter 143, and execute an inter prediction based on or using the inter prediction parameter 143 to obtain the inter prediction block 145.

[0123] FIG. 1 shows two separate units (or steps) for inter coding, namely, the inter estimation 142 and the inter prediction 152. However, both functions can be executed as one by, for example, repeatedly testing a predetermined subset of all possible or pre - determined possible prediction modes while memorizing the current best inter prediction mode and each inter prediction block, and using the current best inter prediction mode and each inter prediction block as the (final) inter prediction parameter 143 and the inter prediction block 145 without performing another time inter prediction 144. (The inter estimation usually includes calculating an inter prediction block, that is, the inter prediction 154 or a "kind of" inter prediction 154.)

[0124] The intra estimation unit 152 is configured to obtain, for example, receive, for intra estimation, the picture block 103 (the current picture block) and one or more previously reconstructed blocks of the same picture, for example, the reconstructed adjacent blocks. The encoder 100 can be configured to select an intra prediction mode from a plurality of intra prediction modes and provide it to the intra prediction unit 154 as the intra estimation parameter 153.

[0125] An embodiment of the encoder 100 can be configured to select an intra prediction mode based on an optimization criterion, for example, the minimum residual (for example, the intra prediction mode that provides the prediction block 155 most similar to the current picture block 103) or the minimum rate - distortion.

[0126] The intra prediction unit 154 is configured to determine based on the intra prediction parameter 153, for example, the selected intra prediction mode 153 and the intra prediction block 155.

[0127] FIG. 1 shows two separate units (or steps) for intra coding, namely, intra estimation 152 and intra prediction 154. However, both functions can be performed as one by, for example, repeatedly testing all possible or a predetermined subset of possible intra prediction modes while storing the current best intra prediction mode and respective intra prediction blocks, and using the current best intra prediction mode and respective intra prediction blocks as the (final) intra prediction parameter 153 and intra prediction block 155 without re - executing the intra prediction 154. (Intra estimation typically includes calculating an intra prediction block, i.e., the intra prediction 154 or a "kind of" intra prediction 154.)

[0128] As will be further described below with respect to device 500 (FIG. 5) and method 800 (FIG. 8) according to embodiments of the present invention, the present invention can be applied at this position of the encoder 100. That is, the device 500 may be, or may be a part of, the encoder 100, specifically the intra prediction unit 154.

[0129] The entropy coding unit 170 is configured to apply an entropy coding algorithm or scheme (e.g., variable - length coding (VLC) scheme, context - adaptive VLC scheme (CALVC), arithmetic coding scheme, context - adaptive binary arithmetic coding (CABAC)) to, for example, the quantized residual coefficients 109, inter prediction parameters 143, intra prediction parameters 153, and / or loop filter parameters, either individually or jointly (or not at all) to obtain the encoded picture data 171 that can be output in the form of, for example, an encoded bitstream 172 at the output 172.

[0130] FIG. 2 shows an exemplary video decoder 200 configured to receive an encoded picture data (e.g., an encoded bitstream) 171 encoded by an encoder 100, for example, to obtain a decoded picture 231.

[0131] The decoder 200 includes an input 202, an entropy decoding unit 204, an inverse quantization unit - 210, an inverse transform unit 212, a reconstruction unit 214, a buffer 216, a loop filter 220, a decoded picture buffer 230, a prediction unit 260 (including an inter-prediction unit 244 and an intra-prediction unit 254), a mode selection unit 260, and an output 232.

[0132] The entropy decoding unit 204 is configured to perform entropy decoding on the encoded picture data 171 to obtain, for example, quantized coefficients 209 and / or decoded coding parameters (not shown in FIG. 2), for example, any or all of (decoded) inter-prediction parameters 143, intra-prediction parameters 153, and / or loop filter parameters.

[0133] In an embodiment of the decoder 200, the inverse quantization unit 210, the inverse transform unit 212, the reconstruction unit 214, the buffer 216, the loop filter 220, the decoded picture buffer 230, the prediction unit 260, and the mode selection unit 260 are configured to perform processing inverse to that of the encoder 100 (and respective functional units) to decode the encoded picture data 171.

[0134] In particular, the inverse quantization unit 210 may have the same function as the inverse quantization unit 110, the inverse transform unit 212 may have the same function as the inverse transform unit 112, the reconstruction unit 214 may have the same function as the reconstruction unit 114, the buffer 216 may have the same function as the buffer 116, the loop filter 220 may have the same function as the loop filter 120 (the loop filter 220 usually does not include a filter analysis unit for determining filter parameters based on the original picture 101 or block 103, but for example, for receiving or obtaining (explicitly or implicitly) the filter parameters used for encoding from the entropy decoding unit 204, with respect to the actual loop filter), and the decoded picture buffer 230 may have the same function as the decoded picture buffer 130.

[0135] The prediction unit 260 may include an inter prediction unit 244 and an intra prediction unit 254. The inter prediction unit 144 may have the same function as the inter prediction unit 244, and the intra prediction unit 154 may have the same function as the intra prediction unit 254. The prediction unit 260 and the mode selection unit 262 are usually configured to perform block prediction and / or obtain the predicted block 265 only from the encoded data 171 (without further information regarding the original picture 101), for example, to receive or obtain (explicitly or implicitly) the prediction parameters 143 or 153 and / or information regarding the selected prediction mode from the entropy decoding unit 204.

[0136] As further described below with respect to the device 500 (see FIG. 5) and method 800 (see FIG. 8) according to embodiments of the present invention, the present invention may be applied at this position of the decoder 200. That is, the device 500 may be or may be a part of the decoder 200, specifically the intra prediction unit 154.

[0137] The decoder 200 is configured to output the decoded picture 230 for presentation or display to the user, for example, via output 232.

[0138] Referring to FIGS. 15 and 16, FIG. 4 more specifically shows, in (a), the causes of discontinuities that can be removed by an embodiment of the present invention. In particular, the reason for these discontinuities is that two vertically adjacent prediction samples 401 within a prediction block 400 (e.g., a PU or a TU) can be predicted from reference samples 403 that are not adjacent to each other due to an acute intra prediction angle, which is a defect in interpolation. This defect can be partially reduced by applying a smoothing filter or an interpolation filter of length N f to the reference samples, but may not be large enough with a fixed length when the intra prediction angle is significantly less than 45°. The filtering process can reduce the impact of discontinuities by convolving the reference samples 403 shown in FIG. 4 during the filtering process. However, if the reference samples 403 selected for the vertically adjacent prediction samples 401 are too far apart, discontinuities may still occur. For example, an example of such a visually observable discontinuity in the case of a synthesized reference (upper row) is shown in (b).

[0139] FIG. 5 schematically shows a device 500 according to an embodiment of the present invention configured to intra predict a prediction block 400 of a video picture in an improved manner, i.e., to eliminate the above-described causes of discontinuities shown in FIG. 4. The device 500 may be, or may be part of, the encoder 100 or the decoder 200 shown in FIGS. 1 or 2, specifically, the intra prediction unit 154 or 254.

[0140] Device 500 is configured to perform some functions implemented by, for example, a processor or other types of processing circuits. Specifically, device 500 is configured to select a directional intra prediction mode 501a from a set of directional intra prediction modes 501, and each directional intra prediction mode 501 corresponds to a different intra prediction angle. These directional intra prediction modes 501 may include the directional / angle intra prediction modes shown in FIG. 9 (and as defined in the standard), and may also include extended directional intra prediction modes corresponding to additional intra prediction angles, for example, as shown in FIG. 14. In particular, for the rectangular prediction block 400, the directional intra prediction mode 501 may include modes related to acute intra prediction angles (angles less than 45°). The intra prediction angle is based on the direction of intra prediction of the prediction sample 401 from the reference sample 403. For example, the angle is defined between this intra prediction direction and the upper edge (horizontal edge) of the prediction block 400.

[0141] Furthermore, device 500 is configured to select a filter 402a from a set of filters 402 based on the selected directional intra prediction mode 501a. Specifically, device 500 includes determining the length of the filter based on the selected directional intra prediction mode 501a, and selecting one filter 402 from the set having at least the determined filter length as the filter 402a.

[0142] For a given prediction sample 401 of the prediction block 400, the device 500 is further configured to determine a reference sample 403a from the set of reference samples 403 based on the selected directional intra prediction mode 501a, and apply a selected filter 402a to the determined reference sample 403a. The device 500 may be specifically configured to proceed in this manner for each prediction sample 401 of the prediction block 400. That is, for each prediction sample 401, the device 500 may determine a reference sample 403a from the reference samples 403 and apply the selected filter 402a to each reference sample 403. In this way, the device 500 can perform intra prediction on the entire prediction block 400.

[0143] An exemplary set of filters for which the device 500 is configured to select a filter 402 is shown in the following table. The set of filters specifically includes different filters 402. For example, the set of filters 402 may include filters 402 having different filter lengths N f , in particular filter lengths N spanning one, three, or five adjacent reference samples 403 f . Further, each filter 402 of the set of filters 402, when applied to the determined reference sample 403a, may perform different smoothing on the determined reference sample 403a and one or more adjacent reference samples 403. This smoothing can be represented by different coefficients as shown in the table, and the number of coefficients indicates the relative weighting of the determined reference sample 403a and other adjacent reference samples (further numbers of 0, 2, or 4 for the adjacent reference samples 403 from the central number for the determined reference sample 403a).

[0144]

Table 1

[0145] FIG. 6 shows an exemplary flowchart of a reference sample filter selection mechanism 600 that the device 500 may be configured to perform. The device 500 can specifically select the reference sample filter 402a according to the intra prediction angle. For the mechanism 600, it is assumed that a set of filters (denoted here as F) is sorted in ascending order by the filter length N f is sorted in ascending order by.

[0146] In block 601, the device 500 is configured to derive the intra prediction angle α as an input to the selection mechanism 600. The device 500 may be configured to determine the intra prediction angle corresponding to the selected directional intra prediction mode 501.

[0147] Next, in block 602, the device 500 is configured to derive the distance Δp α (see, for example, FIG. 4) between the determined reference sample 403a and the further reference sample 403b, and may be specified for the further prediction sample 401 of the prediction block 400 from the set of reference samples 403 based on the selected directional intra prediction mode 501a.

[0148] In block 603, the filter index is initialized to i = 0. In block 604, the filter 402 with the current index i is taken out from the set of filters. For example, the above table shows that the filter 402 can be indexed from i = 0 to 3.

[0149] In block 605, the device 500 is configured to determine whether the length N f of the filter 402 taken from the set is smaller than the distance Δp α If not, the selection mechanism 600 is completed, and the currently taken filter 402 is selected as the filter 402a to be applied to the determined reference sample 403a.

[0150] Otherwise, the device 500 is configured to check in block 606 whether the current filter index i is less than k, where k can be the highest possible filter index and / or can indicate the number of filters 402 in the filter set. If not, the selection mechanism 600 is complete, and in this case, assuming the set is sorted by filter length, the currently taken filter 402 having the maximum filter length N f is selected as the filter 402a to be applied to the determined reference sample 403a, corresponding to the filter 402 having the maximum filter length N. Otherwise, the filter index is incremented by one in block 607, and the selection mechanism proceeds to block 604 (i.e., the next filter 402 in the set is taken).

[0151] As shown in FIG. 7, the device 500 can also be configured to perform preprocessing of the reference sample 403. Specifically, the device 500 can be configured to generate a transposed reference sample 700a by interpolating from the determined reference sample 403a, i.e., the reference sample 403a determined based on the selected intra prediction mode 501a. Then, the device 500 can be configured to perform intra prediction of a given prediction sample 401 from the transposed reference sample 700a instead of directly from the determined reference sample 403a.

[0152] The first step of the preprocessing is exemplarily shown in FIG. 7(a), and the transposed reference sample 700 (

[0153]

Number

[0154] It can consist of calculating a set (shown by). The input to this step can be a set of reference samples 403 arranged above and to the upper right of the block 400 to be predicted. These reference samples 403 can be filtered as described above according to the intra prediction angle. That is, the device 500 can select the filter 402a as described above and then be configured to apply the selected filter 402a to the determined reference samples 403a before or during the generation of the transposed reference samples 700a.

[0155] The first step is specifically performed by interpolation executed on two parts of R. R L One part of the set shown by is arranged to the left of the upper right pixel of the block P TR The reference sample 403 at the position P TR is not changed in this first step, that is,

[0156]

Number

[0157] is. R R Another part shown by is arranged to the right of P TR For both parts, the interpolation is executed using the same mechanism (shown by B) that is used to predict the samples within the block 400 to be predicted. The prediction angles α used for these two parts are the same, but the prediction directions are opposite.

[0158] The second step of the preprocessing is shown in FIG. 7(b), that is, to intra-predict the prediction samples 401 of the block 400 to be predicted by performing intra-prediction interpolation from the set of transposed reference samples 700 calculated in the first step shown in (a). When the topmost row is not used for the intra-prediction direction, that is, when the angle α of the intra-prediction direction is greater than 180 degrees, the reference samples corresponding to the block are transposed (the row index becomes the column index and vice versa), and the intra-prediction is performed as described above. The final result in this case is obtained by transposing the calculated prediction block back.

[0159] FIG. 8 shows a method 800 according to an embodiment of the present invention. The method 800 is for intra-predicting a prediction block 400 of a video picture and can be executed by the device 500 shown in FIG. 5. Specifically, the method 800 includes a step 801 of selecting a directional intra-prediction mode 501a from a set of directional intra-prediction modes 501, where each directional intra-prediction mode 501 corresponds to a different intra-prediction angle. Further, the method 800 includes a step 802 of selecting a filter 402a from a set of filters 402 based on the selected directional intra-prediction mode 501a. Then, the method 800 includes a step 803 of determining a reference sample 403a from a set of reference samples 403 for a given prediction sample 401 of the prediction block 400 based on the selected directional intra-prediction mode 501a, and a step 804 of applying the selected filter 402a to the determined reference sample 403a.

[0160] This specification provides an explanation of a picture (frame), but it should be noted that in the case of an interlaced picture signal, a field is substituted for a picture.

[0161] Embodiments of the present invention have mainly been described based on video coding. However, it should be noted that embodiments of the encoder 100 and decoder 200 (and, correspondingly, system 300) can also be configured for still picture processing or coding, i.e., processing or coding of individual pictures independent of preceding or consecutive pictures as in the case of video coding. Generally, when picture processing coding is limited to a single picture 101, only inter-estimation 142, inter-prediction 144, and 242 are not available. Most, if not all, other functions (also referred to as tools or techniques) of the video encoder 100 and video decoder 200 can be equally used for still pictures, such as splitting, transform (scaling) 106, quantization 108, inverse quantization 110, inverse transform 112, intra-estimation 142, intra-prediction 154, 254, and / or loop filtering 120, 220, as well as entropy coding 170 and entropy decoding 204.

[0162] Those skilled in the art will understand that the "blocks" ("units") of the various figures (methods and apparatuses) represent or illustrate the functions of the embodiments of the present invention (not necessarily individual "units" in hardware or software), and thus equally describe the functions or features of the apparatus embodiments as well as the method embodiments (unit = step).

[0163] The term "unit" is used only for the purpose of describing the functions of the encoder / decoder embodiments and is not intended to limit the disclosure.

[0164] In some embodiments provided in the present application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the described embodiments of the devices are merely illustrative. For example, unit division is merely logical function division, and in actual implementation forms, it may be other divisions. For example, a plurality of units or components may be combined or integrated into another system, and some functions may be ignored or not executed. Further, the shown or described mutual coupling or direct coupling or communication connection can be implemented by using some interfaces. The indirect coupling or communication connection between devices or units can be implemented in electronic, mechanical, or other forms.

[0165] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, may be arranged in one location, or may be distributed among a plurality of network units. Some or all of the units can be selected according to the actual needs to achieve the purpose of the solution of the embodiment.

[0166] Furthermore, the functional units in the embodiments of the present invention may be integrated into one processing unit, each unit may physically exist alone, or two or more units may be integrated into one unit.

[0167] Embodiments of the present invention may further include a device comprising a processing circuit configured to execute any of the methods and / or processes described herein, for example, an encoder and / or a decoder.

[0168] Embodiments of the encoder 100 and / or decoder 200 may be implemented as hardware, firmware, software, or any combination thereof. For example, the encoder / encoding or decoder / decoding functions may be performed by processing circuitry, with or without firmware or software, such as a processor, microcontroller, digital signal processor (DSP), field programmable gate array (FPGA), application specific integrated circuit (ASIC), and the like.

[0169] The functions of the encoder 100 (and corresponding encoding method 100) and / or decoder 200 (and corresponding decoding method 200) may be implemented by program instructions stored on a computer-readable medium. When executed, the program instructions cause a processing circuit, computer, processor, etc. to perform the steps of the encoding and / or decoding method. The computer-readable medium can be any medium including non-transitory storage media in which a program is stored, such as a Blu-ray disc, DVD, CD, USB (flash) drive, hard disk, server storage accessible via a network, and the like.

[0170] One embodiment of the present invention includes, or is, a computer program including program code for performing any of the methods described herein when executed on a computer.

[0171] One embodiment of the present invention includes, or is, a computer-readable medium including program code that, when executed by a processor, causes a computer system to perform any of the methods described herein.

Explanation of Signs

[0172] 100 Encoder 102 Input (e.g., input port, input interface) 103 Picture block 104 Residual calculation [unit or step] 105 Residual block 106 Transformation (including additional scaling, for example) [unit or step] 107 Transformed coefficient 108 Quantization [unit or step] 109 Quantized coefficient 110 Inverse quantization [unit or step] 111 Dequantized coefficient 112 Inverse transformation (including additional scaling, for example) [unit or step] 113 Inverse transformed block 114 Reconstruction [unit or step] 115 Reconstructed block 116 (Line) buffer [unit or step] 117 Reference sample 120 Loop filter [unit or step] 121 Filtered block 130 Decoded picture buffer (DPB) [unit or step] 142 Interpolation (or, inter-image interpolation) [unit or step] 143 Interpolation parameter (for example, reference image / reference image index, motion vector / offset) 144 Inter prediction (or, inter-image prediction) [unit or step] 145 Inter prediction block 152 Intra prediction (or, intra-image prediction) [unit or step] 153 Intra prediction parameter (for example, intra prediction mode) 154 Intra prediction (intra-frame / image prediction) [unit or step] 155 Intra prediction block 162 Mode selection [unit or step] 165 Prediction block (either inter-prediction block 145 or intra-prediction block 155) 170 Entropy encoding [unit or step] 171 Encoded picture data (e.g., bitstream) 172 Output (output port, output interface) 231 Decoded picture 200 Decoder 202 Input (port / interface) 204 Entropy decoding 209 Quantized coefficients 210 Inverse quantization 211 Non-quantized coefficients 212 Inverse transform (scaling) 213 Inverse-transformed block 214 Reconstruction (unit) 215 Reconstructed block 216 (Line) buffer 217 Reference sample 220 Loop filter (in loop filter) 221 Filtered block 230 Decoded picture buffer (DPB) 231 Decoded picture 232 Output (port / interface) 244 Inter-prediction (inter-frame / picture prediction) 245 Inter-prediction block 254 Intra-prediction (intra-frame / picture prediction) 255 Intra-prediction block 260 Mode selection 265 Prediction block (either inter-prediction block 245 or intra-prediction block 255) 300 Coding system 310 Source device 312 Picture source 313 Raw picture data 314 Preprocessor / pre-processing unit 315 Pre-processed picture data 318 Communication unit / interface 320 Destination device 322 Communication unit / interface 326 Post-processor / Post-processing unit 327 Post-processed picture data 328 Display device / unit 330 Transmitted / received / communicated (encoded) picture data 400 Prediction block 401 Prediction sample 402 Filter 402a Selected filter 403 Reference sample 403a Determined reference sample 500 Device 501 Directional intra prediction mode 501a Selected directional intra prediction mode 600 Filter selection mechanism 601 - 607 Functional blocks of the mechanism 700 Transposed reference sample 700a Transposed reference sample 800 Method for intra predicting a prediction block 801 Step of selecting an intra prediction mode 802 Step of selecting a filter 803 Step of determining a reference sample for a given prediction sample 804 Step of applying the selected filter to the reference sample

Claims

1. A device (500) for intra-predicting a prediction block (400) of a video image, comprising: selecting a directional intra prediction mode (501a) from a set of directional intra prediction modes (501), each directional intra prediction mode (501) corresponding to a different intra prediction angle; selecting a filter (402a) from a set of filters (402) based on the selected directional intra prediction mode (501a); for a given prediction sample (401) of the prediction block (400), determining a reference sample (403a) from a set of reference samples (403) based on the selected directional intra prediction mode (501a); applying the selected filter (402a) to the determined reference sample (403a). The device (500) is configured to perform the above operations.

2. Determining the length of the filter based on the selected directional intra prediction mode (401a); Selecting as the filter (402a) one having at least the determined length of the filter. The device (500) according to claim 1, wherein the device is configured to perform the above operations.

3. The device (500) according to claim 2, wherein the set of filters (402) includes filters (402) having different filter lengths, particularly filter lengths spanning one, three, or five adjacent reference samples (403).

4. The device (500) according to any one of claims 1 to 3, wherein when each filter (402) in the set of filters (402) is applied to the determined reference sample (403a), different smoothing is performed on the determined reference sample (403a) and one or more adjacent reference samples (403).

5. Determining the intra prediction angle corresponding to the selected directional intra prediction mode (501a); Selecting the filter (402) according to the determined intra prediction angle. The device (500) according to any one of claims 1 to 4, wherein the device is configured to perform the above operations.

6. Determining the intra prediction angle corresponding to the selected directional intra prediction mode (501); For further prediction samples (401) of the prediction block (400), based on the selected directional intra prediction mode (501a), specifying a further reference sample (403b) from the set of reference samples (403); determining a distance between the determined reference sample (403a) and another said reference sample (403b); selecting the filter (402a) according to the determined distance The device (500) according to any one of claims 1 to 5, which is configured to perform the above.

7. selecting the same filter (402a) for each directional intra prediction mode (501a) selected from a first subset of the directional intra prediction modes (501); selecting a different filter (402a) for each directional intra prediction mode (501a) selected from a second subset of the directional intra prediction modes (501) The device (500) according to any one of claims 1 to 6, which is configured to perform the above.

8. The device is further configured to perform intra prediction of the given prediction sample (401) directly from the determined reference sample (403a), The device (400) is configured to apply the selected filter (402a) to the determined reference sample (403a) before or during the intra prediction of the given prediction sample (401). The device (500) according to any one of claims 1 to 7.

9. generating a transposed reference sample (700a) by interpolating the determined reference sample (403a) based on the selected intra prediction mode (501a); ​ ​ ​ ​ The device (500) according to claim 9, configured to transpose each reference sample (403) within the set of reference samples (403), such that the rows of the reference sample (403) become the columns of the transposed reference sample (700), and the columns of the reference sample (403) become the rows of the transposed reference sample (700).

11. The device (500) according to any one of claims 1 to 10, wherein the reference samples (403) of the set of reference samples (403) are arranged in rows of the video image adjacent above and diagonally above right of the prediction block (400), and / or are arranged in columns of the video image adjacent to the left and diagonally below left of the prediction block (400).

12. Is the device (500) configured to encode and / or decode the video image? The device (500) according to any one of claims 1 to 11, wherein the device (500) is a video encoder (100) and / or a video decoder (200).

13. A method (800) for intra-predicting a prediction block (400) of a video image, comprising: A step (801) of selecting a directional intra prediction mode (501a) from a set of directional intra prediction modes (501), each directional intra prediction mode (501) corresponding to a different intra prediction angle; A step (802) of selecting a filter (402a) from a set of filters (402) based on the selected directional intra prediction mode (501a); A step (803) of determining a reference sample (403a) from a set of reference samples (403) based on the selected directional intra prediction mode (501) for a given prediction sample (401) of the prediction block (400); A step (804) of applying the selected filter (402a) to the determined reference sample (403a). The method (800) comprising the above steps.

Images