Method and apparatus for video encoding and decoding based on block shape

By processing the reference sample set based on the block shape in video encoding, generating prediction blocks, and selecting appropriate prediction modes and filters, the problem of insufficient selection of prediction modes under block shape asymmetric segmentation in the prior art is solved, and the encoding efficiency is improved and predict performance.

CN111448798BActive Publication Date: 2024-11-01INTERDIGITAL VC HOLDINGS INC
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN201880079017.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2017-12-05
Filing Date
2018-12-04
Publication Date
2024-11-01
Estimated Expiration
2039-05-02

AI Technical Summary

Technical Problem

When existing video encoding technology is asymmetrically segmented block shapes, the prediction mode selection and filter application are insufficient, resulting in low encoding efficiency, especially poor prediction performance under rectangular block shapes.

Method used

By accessing the reference sample set of blocks in video pictures, processing the reference sample set based on the shape of the block, generating prediction blocks of blocks, and selecting appropriate prediction modes and filters based on the shape and prediction direction of the block, intra- or inter-frames Predictive encoding.

Benefits of technology

Improved coding efficiency, especially prediction performance under rectangular block shape, reduces the number of signaling flags, and enhances coding gain.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN111448798B_ABST
    Figure CN111448798B_ABST
Patent Text Reader

Abstract

Methods (1300, 1500) and apparatus (700, 1400, 1600) for video encoding and decoding are provided. The video encoding method (1300) includes accessing (1310) a reference sample set for predicting a block in a picture of a video, processing (1320) the reference sample set based on a shape of the block, generating (1330) a prediction block for the block based on the processed reference sample set, and encoding the block based on the prediction block. A bitstream formatted to include encoded data, a computer-readable storage medium, and a computer program product are also described.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present embodiments relate generally to video encoding and decoding, and more particularly to prediction of blocks based on block shape. Background Art

[0002] Any background information described herein is intended to introduce the reader to various aspects of the art that may be relevant to the present embodiments described below. This discussion is considered to be helpful in providing the reader with background information to better understand the various aspects of the present disclosure. Therefore, it should be understood that these statements should be understood from this perspective.

[0003] To achieve high compression efficiency, image and video coding schemes typically employ prediction and transform to exploit spatial and temporal redundancy in video content. Typically, intra-frame or inter-frame prediction is used to exploit intra-frame or inter-frame correlations. The difference between the original image and the predicted image (typically expressed as a prediction error or prediction residual) is then transformed, quantized, and entropy encoded. To reconstruct the video, the compressed data is decoded by the inverse process corresponding to prediction, transform, quantization, and entropy encoding.

[0004] In the High Efficiency Video Coding (HEVC) standard ("ITU-T H.265, ITU Telecommunication standardization sector of ITU (10 / 2014), series H: audiovisual and multimedia systems, infrastructure of audiovisual services-coding of moving video, High efficiency video coding, Recommendation ITU-T H.265"), a picture is partitioned into square coding tree units (CTUs), and the configurable size of the coding tree unit is typically 64×64, 128×128, or 256×256. Figure 1 As shown, CTU 110 is the root of a quadtree partitioned into leaves, which are called coding units (CUs). For each CU, a prediction mode is signaled, which indicates whether intra-frame prediction or inter-frame prediction is used to encode the CU. Figure 2As shown, a group of consecutive CTUs (e.g., CTU 220) can be grouped into slices 210. A CU (e.g., CU 230) can be partitioned into one or more prediction units (PUs) and form the root of a quadtree (called a transform tree) partitioned into transform units (TUs). In inter-frame prediction, the CU can also be subdivided into PUs asymmetrically, that is, if the CU has a size of N×N, the PU can have a size of N / 4×N, 3N / 4×N, N×N / 4, N×3N / 4. Each PU is assigned some prediction information, such as motion information, spatial intra prediction, etc. PU in Figure 1 is shown as a dotted line.

[0005] The Quad-Tree plus Binary-Tree (QTBT) coding tool is a new video coding tool that provides more flexible CTU representation and higher compression efficiency compared to the CU / PU / TU arrangement of the HEVC standard. Figure 3 As shown, a quadtree plus binary tree (QTBT) coding tool defines a coding tree 310, where coding units can be partitioned in a quadtree and binary tree manner. Figure 3 An exemplary coding tree representation of a coding tree unit 320 is shown, where solid lines indicate quadtree partitioning and dashed lines indicate binary partitioning of CUs 330 within the CTU 320, which are spatially embedded in the leaves of the quadtree.

[0006] The partitioning of a CTU into coding units is decided at the encoder side, for example, by a rate-distortion optimization process, which includes determining a QTBT representation of the CTU with the minimum rate-distortion cost. In the QTBT representation, the CU has a square or rectangular shape. The size of a coding unit is always a power of 2, typically from 4 to 128. The QTBT decomposition of a CTU consists of two stages: first, the CTU is partitioned into 4 CUs in a quadtree manner, and then each quadtree leaf can be further partitioned into two CUs in a binary manner or into 4 CUs in a quadtree manner, such as Figure 3 shown.

[0007] With QTBT representation, a CU may not be further partitioned into PUs or TUs. In other words, each CU is considered to be a single prediction unit and a single transform unit. Figure 3 As shown, this QTBT indicates that only symmetric partitioning of the CU is allowed. The four partitioning modes allowed by QTBT are: NO_SPLIT (CU is not partitioned), QT_SPLIT (partitioned into four quadrants), HOR (horizontally partitioned into two CUs of equal size) and VER (vertically partitioned into two CUs of equal size).

[0008] Recently, a CU with a new rectangular shape is proposed, which is generated by a new binary partitioning mode called asymmetric partitioning mode, such as Figure 4 and Figure 5 shown. Figure 4 The CU binary partitioning mode in QTBT is shown, which is called an asymmetric partitioning mode, and four exemplary partitioning modes 410 to 440 are depicted. Figure 4 The new rectangular shape includes a width and / or height equal to 3 2 n In addition, CUs whose width or height is a multiple of 3 can be further split horizontally or vertically in a binary manner.

[0009] A square CU of size (w,h) (width and height) is split by one of the proposed asymmetric binary splitting modes (e.g., HOR_UP (horizontally up) 410), resulting in a corresponding rectangular CU of size and Therefore, the encoder can choose a width or height equal to 3·2 n In this case, intra prediction or inter prediction process can be performed for rectangular blocks whose size is a multiple of 3. In addition, a CU with a width or height size of 3·2 n 2D transform and perform a subsequent transform coefficient entropy coding process.

[0010] like Figure 5 The other CU partitioning modes shown (referred to as horizontal triplet partitioning mode 510 and vertical triplet partitioning mode 520) include partitioning the CU into 3 sub-coding units (sub-CUs), where, in the direction of the considered spatial partitioning, the sizes of the corresponding sub-coding units are equal to 1 / 4, 1 / 2 and 1 / 4 of the size of the parent CU. Summary of the invention

[0011] According to one aspect of the present disclosure, a video encoding method is provided, the method comprising: accessing a reference sample set for prediction of a block in a picture of a video, processing the reference sample set based on a shape of the block, generating a prediction block of the block based on the processed reference sample set, and encoding the block based on the prediction block.

[0012] According to an aspect of the present disclosure, a device for video encoding is provided, the device comprising: a component for accessing a reference sample set for prediction of a block in a picture of a video; a component for processing the reference sample set based on a shape of the block; a component for generating a prediction block of the block based on the processed reference sample set; and a component for encoding the block based on the prediction block.

[0013] According to an aspect of the present disclosure, a device for video encoding is provided, the device comprising a processor and at least one memory coupled to the processor, the processor being configured to access a reference sample set for prediction of a block in a picture of a video, process the reference sample set based on a shape of the block, generate a prediction block of the block based on the processed reference sample set, and encode the block based on the prediction block.

[0014] According to an aspect of the present disclosure, a bitstream formatted to include encoded data representing a block of a picture is encoded by accessing a reference sample set for prediction of a block in a picture of a video, processing the reference sample set based on a shape of the block, generating a prediction block of the block based on the processed reference sample set, and encoding the block based on the prediction block.

[0015] According to an aspect of the present disclosure, a signal includes a bit stream formatted to include encoded data representing a block of a picture, the encoded data being encoded by accessing a reference sample set for prediction of a block in a picture of a video, processing the reference sample set based on a shape of the block, generating a predicted block based on the processed reference sample set, and encoding the block based on the predicted block.

[0016] According to one aspect of the present disclosure, a video decoding method is provided, the method comprising: accessing a reference sample set for prediction of a block in a picture of an encoded video, processing the reference sample set based on a shape of the block, generating a prediction block of the block based on the processed reference sample set, and decoding the block based on the prediction block.

[0017] According to one aspect of the present disclosure, a device for video decoding is provided, the device comprising: a component for accessing a reference sample set for prediction of a block in a picture of an encoded video; a component for processing the reference sample set based on a shape of the block; a component for generating a prediction block of the block based on the processed reference sample set; and a component for decoding the block based on the prediction block.

[0018] According to an aspect of the present disclosure, a device for video decoding is provided, the device comprising a processor and at least one memory coupled to the processor, the processor being configured to: access a reference sample set for prediction of a block in a picture of an encoded video, process the reference sample set based on a shape of the block, generate a prediction block of the block based on the processed reference sample set, and decode the block based on the prediction block.

[0019] According to an aspect of the present disclosure, a computer program product is provided, comprising program code instructions for performing the following steps: accessing a reference sample set for prediction of a block in a picture of a video, processing the reference sample set based on a shape of the block, generating a prediction block for the block based on the processed reference sample set, and encoding the block based on the prediction block.

[0020] According to an aspect of the present disclosure, a computer program product is provided, comprising program code instructions for performing the following steps: accessing a reference sample set for prediction of a block in a picture of an encoded video, processing the reference sample set based on a shape of the block, generating a prediction block for the block based on the processed reference sample set, and decoding the block based on the prediction block.

[0021] According to one aspect of the present disclosure, there is provided a computer-readable storage medium carrying a software program, the software program comprising program code instructions for performing the following steps: accessing a reference sample set for prediction of a block in a picture of a video, processing the reference sample set based on a shape of the block, generating a prediction block for the block based on the processed reference sample set, and encoding the block based on the prediction block.

[0022] According to one aspect of the present disclosure, there is provided a computer-readable storage medium carrying a software program, the software program including program code instructions for performing the following steps: accessing a reference sample set for predicting a block in a picture of an encoded video, processing the reference sample set based on a shape of the block, generating a prediction block of the block based on the processed reference sample set, and decoding the block based on the prediction block.

[0023] A simplified overview of the subject matter is presented above in order to provide a basic understanding of some aspects of the subject matter embodiments. This overview is not an extensive overview of the subject matter. It is not intended to identify key / important elements of the embodiments or to describe the scope of the subject matter. Its sole purpose is to present some ideas of the subject matter in a simplified form as a prelude to a more detailed description presented later.

[0024] Additional features and advantages of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The present disclosure may be better understood with reference to the exemplary drawings briefly described below:

[0026] Figure 1 illustrates a CTU that is partitioned into CUs according to the HEVC standard;

[0027] Figure 2 It shows the partitioning of CTU into CU, PU and TU according to the HEVC standard;

[0028] Figure 3 shows the CTU according to the QTBT tool;

[0029] Figure 4 shows a CU binary partitioning mode in QTBT, which is called an asymmetric partitioning mode;

[0030] Figure 5Horizontal (left) and vertical (right) ternary tree CU partitioning modes in QTBT are shown;

[0031] Figure 6 An exemplary set of CU partition modes according to an embodiment of the present disclosure is shown;

[0032] Figure 7 A simplified block diagram of an exemplary video encoder according to an embodiment of the present disclosure is shown;

[0033] Figure 8 A simplified block diagram of an exemplary intra prediction module according to an embodiment of the present disclosure is shown;

[0034] Fig. 9 shows an exemplary reference sample of a current block according to the present disclosure;

[0035] Fig.10 shows the intra prediction direction according to the HEVC standard;

[0036] Fig.11 An exemplary intra prediction mode for a square block shape according to the HEVC standard is shown;

[0037] Fig.12 An exemplary intra prediction mode for a rectangular block shape according to the present disclosure is shown;

[0038] Fig.13 A flowchart of an exemplary method of video encoding according to an embodiment of the present disclosure is shown;

[0039] Fig.14 A simplified block diagram of an exemplary video decoder according to an embodiment of the present disclosure is shown;

[0040] Fig.15 A flowchart showing an exemplary method of video decoding according to an embodiment of the present disclosure; and

[0041] Fig.16 A block diagram of a computing environment is shown in which aspects of the present disclosure may be implemented and executed. DETAILED DESCRIPTION

[0042] It should be understood that the elements shown in the accompanying drawings can be implemented with various forms of hardware, software or a combination thereof. Preferably, these elements are implemented with a combination of hardware and software on one or more appropriately programmed general-purpose devices, which may include a processor, a memory and an input / output interface. Herein, the phrase "coupling" is defined as a mode of direct connection or indirect connection through one or more intermediate components. Such intermediate components may include components based on hardware and software.

[0043] This specification illustrates the principles of the present disclosure. Therefore, it should be appreciated that those skilled in the art will be able to design various arrangements that, although not explicitly described or shown herein, embody the principles of the present disclosure and are included within the scope of the present disclosure.

[0044] All examples and conditional language recited herein are for teaching purposes to help readers understand the principles of the present disclosure and the concepts contributed by the inventors to advance the art, and are to be construed as not being limited to these specifically recited examples and conditions.

[0045] In addition, all statements and specific examples of the principles, aspects and embodiments of the present disclosure are intended to cover both structural and functional equivalents. In addition, it is intended that such equivalents include currently known equivalents and equivalents developed in the future, i.e., any element developed to perform the same function regardless of its structure.

[0046] Thus, for example, those skilled in the art will appreciate that the block diagrams presented herein represent conceptual diagrams of illustrative circuits embodying the principles of the present disclosure. Similarly, it will be appreciated that any flow charts, state transition diagrams, pseudo-codes, etc. represent various processes that can be substantially represented in a computer-readable medium and executed by a computer or processor, whether or not such a computer or processor is explicitly shown.

[0047] The functions of the various elements shown in the drawings may be provided through the use of dedicated hardware as well as hardware capable of executing software in association with appropriate software. When provided by a processor, the functions may be provided by a single dedicated processor, a single shared processor, or multiple separate processors (some of which may be shared). In addition, the explicit use of the term "processor" or "controller" should not be interpreted as referring exclusively to hardware capable of executing software, and may implicitly include, but is not limited to, digital signal processor (DSP) hardware, read only memory (ROM), random access memory (RAM), and non-volatile memory for storing software.

[0048] Other hardware, conventional and / or custom, may also be included. Likewise, any switches shown in the drawings are conceptual only. Their functions may be implemented by operation of program logic, by dedicated logic, by the interaction of program control and dedicated logic, or even manually, the specific technique being selectable by the implementer as more specifically understood from the context.

[0049] In the claims, any element expressed as a means for performing a particular function is intended to encompass any means of performing that function, including, for example, a) a combination of circuit elements that perform that function, or b) any form of software, thus including firmware, microcode, etc., in combination with appropriate circuitry for executing that software to perform that function. The disclosure defined by such claims resides in the fact that the functionalities provided by the various enumerated means are combined and put together in the manner required by the claims. Therefore, any means capable of providing those functionalities are considered equivalent to those shown herein.

[0050] It should be understood that the drawings and descriptions have been simplified to show elements relevant for a clear understanding of the present disclosure, while many other elements found in a typical encoding and / or decoding device have been eliminated for clarity.

[0051] It should be understood that although the terms first and second can be used to describe various elements in this article, these elements should not be limited by these terms. These terms are only used to distinguish one element from another element. Various methods are described above, and each method includes one or more steps or actions for implementing the described method. Unless the proper operation of the method requires a specific order of steps or actions, the order and / or use of specific steps and / or actions can be modified or combined.

[0052] It should be understood that a picture can be a luma sample array in a monochrome format, or two corresponding chroma sample arrays and a luma sample array in 4:2:0, 4:2:2, and 4:4:4 color formats, or three arrays of three color components (e.g., RGB). In HEVC, a "block" addresses a specific area in a sample array (e.g., luma Y), and a "unit" includes a concatenated block of all coded color components (luma Y and possibly chroma Cb and chroma Cr), syntax elements, and prediction data associated with the block (e.g., motion vectors). However, the term "block" is used more generally herein to refer to a block (e.g., coding block (CB), transform block (TB), coding group (CG), etc.) or a unit (e.g., CU).

[0053] It should be understood that a picture or pixel block or transform coefficient is a two-dimensional array or matrix. The horizontal or x direction (or axis) represents the width, and the vertical or y direction (or axis) represents the height. The index starts at 0. The x direction represents the columns, and the y direction represents the rows. The maximum x index is the width-1. The maximum y index is the height-1.

[0054] In the following sections, the words "reconstruction" and "decoding" can be used interchangeably. Usually, but not necessarily, "reconstruction" is used on the encoder side, and "decoding" is used on the decoder side. In addition, the words "coded" and "encoded" can be used interchangeably. In addition, the words "image", "picture", "frame" and "slice" (i.e., a portion of a picture) can be used interchangeably. In addition, the words "coding", "source coding" and "compression" can be used interchangeably.

[0055] The present disclosure is directed to predictions in video encoding and decoding. Intra-frame prediction in video compression refers to spatial prediction of pixel blocks using information from causal neighbor blocks, which are neighboring blocks that have been decoded in the same frame. This is a powerful coding tool because it allows high compression efficiency within a frame, and it also allows high compression efficiency between frames whenever there is no better temporal prediction. Therefore, intra-frame prediction has been included as a core coding tool in video compression standards (including H.264 / Advanced Video Coding (AVC), H.265 / HEVC, etc.). In HEVC intra-frame prediction, a CU is spatially predicted based on causal neighboring CUs (i.e., CUs on the upper and upper right sides, CUs on the left and lower left sides, and CUs on the upper left side).

[0056] Specifically, the present disclosure is directed to smoothing filtering of reference samples. In HEVC and earlier video coding standards, the intra prediction of the target block uses filtered reference samples. Specifically, the reference samples can be, for example, the decoded pixel rows (or sample rows) on the upper side of the current block and the decoded pixel columns on the left side. Based on the decoded pixel values ​​in these CUs, the encoder constructs different predictions for the target block and selects the prediction that results in the best RD performance.

[0057] In HEVC, the target block size has been increased to 64×64 to support compression of high-resolution video. Corresponding to this block size, the number of intra prediction modes has been increased to 35, one of which is a planar mode (indexed as mode 0), one is a DC mode (indexed as mode 1), and the remaining 33 (indexed as modes 2-34) are directional or angular modes. Recently, rectangular blocks and blocks of size up to 256×256 with up to 131 prediction modes are being explored.

[0058] In HEVC and some previous video coding standards, intra prediction modes are signaled for each CU. Depending on the block size, reference samples are filtered. Prediction modes and filters (e.g., Position Dependent Prediction Combination (PDPC) or Reference Sample Adaptive Filtering (RSAF)) are signaled.

[0059] Figure 6 An exemplary set of CU partitioning modes according to the present disclosure is shown. A rich set of CU topologies produces a coding structure that spatially matches the structure and discontinuity of the image contained in the bitstream. In the case of rectangular block shapes (rather than the traditional square shape), it is of interest to reduce the number of signaling flags by activating some prediction modes and filters based on the block shape (e.g., horizontal or vertical rectangular blocks) and, for example, intra-prediction direction.

[0060] The present disclosure solves some shortcomings existing in the prior art. Specifically, in at least some embodiments of the present disclosure, having a shape-dependent pattern can lead to better prediction and thus higher coding gain. Although the description generally relates to intra-frame prediction, similar concepts can be applied to inter-frame prediction without departing from the scope of the present disclosure. Specifically, similar concepts can be applied to filtering of reference samples for intra-frame prediction and / or inter-frame prediction mode.

[0061] coding

[0062] Figure 7A simplified block diagram of an exemplary video encoder 700 according to an embodiment of the present disclosure is shown. The encoder 700 may be included in a transmitter or a headend in a communication system. In order to encode a video sequence having one or more pictures, the picture may be partitioned into CTUs of squares of configurable size. A set of continuous CTUs may be grouped into slices. A CTU is the root of a QTBT partitioned into CUs. In the exemplary encoder 700, the picture is encoded by an encoder module as described below. Each block is encoded using an intra mode or an inter mode. When encoding a block in intra mode, the encoder 700 performs intra prediction (module 760) or spatial prediction based on at least one block in the same picture or frame. When encoding a block in inter mode, the encoder 700 performs inter prediction or temporal prediction based on at least one reference block from at least one reference picture or frame. In unidirectional (uni) inter prediction, the prediction is usually (but not necessarily) based on an earlier reference picture or frame. In bidirectional (bi) inter prediction, the prediction is usually (but not necessarily) based on earlier and later pictures or frames. In inter mode, motion estimation (module 775) and compensation (module 770) are performed. The encoder decides (module 705) which of intra mode or inter mode to use to encode the block, and indicates the intra / inter decision through a prediction mode flag. The residual is calculated by subtracting (module 710) a block of prediction samples (also called predictor) from the original image block.

[0063] As an example, blocks in intra mode are predicted from reconstructed neighboring samples. Inter prediction is performed by performing motion estimation (module 775) and motion compensation (module 770) on reference blocks stored in reference picture buffer 780. Motion estimation module 775 may include motion compensation because its purpose is to determine the best motion vector, which may use an iterative search that typically terminates when the rate-distortion cost (RD cost) is sufficiently low or has reached a minimum.

[0064] The residual is transformed (module 725) and quantized (module 730). The transform module 725 can transform the image from the pixel or time domain to the transform or frequency domain. The transform can be, for example, a cosine transform, a sine transform, a wavelet transform, etc. Quantization can be performed according to, for example, a rate-distortion standard. The quantized transform coefficients and motion vectors and other syntax elements are entropy encoded (module 745) to output a bitstream. Entropy coding can be, for example, context adaptive binary arithmetic coding (ContextAdaptive Binary Arithmetic Coding, CABAC), context adaptive variable length coding (ContextAdaptive Variable Length Coding, CAVLC), Huffman coding, arithmetic coding, empirical value coding, etc. The encoder can also skip the transform and apply quantization directly to the untransformed residual signal. The encoder can also bypass both transform and quantization, that is, directly encode the residual without applying the transform or quantization process. In direct PCM coding, prediction is not applied, and the block samples are directly encoded into the bitstream.

[0065] The encoder includes a decoding loop and thus decodes the coded block to provide a reference for further prediction. The quantized transform coefficients are dequantized (module 740) and inverse transformed (module 750) to decode the residual. The image block is reconstructed by combining (module 755) the decoded residual and the prediction sample block. The in-loop filter (765) can be applied to reconstruct the picture, for example, to perform deblocking filtering / sample adaptive offset (Sample Adaptive Offset, SAO) filtering to reduce coding artifacts. The filtered image is stored in a reference picture buffer 780.

[0066] The modules of the video encoder 700 may be implemented in software and executed by a processor, or may be implemented using circuit components known to those skilled in the art of compression. Specifically, the video encoder 700 may be implemented as an integrated circuit (IC).

[0067] Except for the differences described in the present disclosure, in particular, except for the differences in the module intra prediction 760 and / or the differences in the inter prediction modules 770, 775 as will be described in more detail in the following paragraphs and figures, the modules of the video encoder 700 may also be present in other video encoders (e.g., HEVC encoders). For functions other than the intra prediction 760 and / or the inter prediction 770, 775, the video encoder 700 may be similar to the HEVC video encoder, and these functions are not described in detail herein.

[0068] Figure 8A simplified block diagram 800 of an exemplary intra-frame prediction module according to an embodiment of the present disclosure is shown. The intra-frame prediction module 800 may be similar to Figure 7 The intra prediction module 800 receives the reconstructed block 811 and outputs a prediction block 835. The intra prediction module 800 includes three modules: a reference sample generation module 810, a block intra prediction module 820 and a prediction post-processing module 830.

[0069] In the reference sample generation module 810, the reference sample 813 may first be calculated or determined in the reference sample calculation module 812 based on the reconstructed neighboring blocks of the current block. At this stage, the reference pixel or sample value may be copied to the first reference sample buffer, and if the lower left pixel or the upper right pixel is not available, they are filled. Next, in the reference sample smoothing module 814, a smoothed version of the original reference sample / buffer 813 is calculated as a filtered reference sample / buffer 815. Alternatively, module 814 may be bypassed. Then, in the reference sample selection module 816, a reference sample is selected 819 from the original reference sample 813 or the filtered reference sample 815 according to the selection parameter 817. Next, in the module intra prediction 820, prediction is performed to calculate the intra prediction of the current block from the selected reference sample / buffer 819. Finally, according to the prediction direction, post-processing (e.g., filtering) may be applied to the intra prediction in the prediction post-processing module 830 to output a prediction block 835. The post-processing module 830 may be bypassed or removed.

[0070] In addition to the differences described in the present disclosure, in particular, in addition to the differences in modules 816, 820, and 830 and selection parameters 817 as will be described in more detail in the following paragraphs and figures, the modules of the intra prediction module 800 may also exist in other video encoders (e.g., HEVC encoders). For example, in the HEVC standard, the selection parameters in 817 are the number of pixels of the current block (e.g., block size) and the determined prediction direction. In addition, in the HEVC standard, modules 820 and 830 are functions of the determined prediction direction. According to the present disclosure, the block shape (e.g., horizontal or vertical rectangle) can be a selection parameter 817 and can also affect modules 816, 820, and 830, which will be described in more detail in the following paragraphs and figures. The block shape is characterized by the ratio of the width and height of the block, which is why horizontal and vertical rectangular blocks are considered to be different shapes. Therefore, the direction of the rectangular block is taken into account.

[0071] Fig. 9 Exemplary reference samples (920, 930) of a current block 910 are shown in accordance with the present disclosure. Fig. 9 Also applicable to the HEVC standard. Figure 8In module 812, for a current block 910 of size N×N (i.e., horizontal (x) to vertical (y) direction), a row of 2N (i.e., 2*N) reconstructed reference samples on the upper side 920 can be formed from previously reconstructed upper and upper right reference pixels to the current block 910. The upper reference samples 920 include samples P(0,-1), ..., P(N-1,-1), ..., P(2N-1,-1). Similarly, a column of 2N samples on the left side 930 can be formed from reconstructed left and lower left reference pixels to the current block 910. The left reference samples 930 include samples P(-1,0), ..., P(-1,N-1), ..., P(-1,2N-1). The corner pixel P(-1,-1) at the upper left corner position is also used to fill the gap between the upper row and left column references. If some of the samples on the upper or left side are not available (because the corresponding reference block is not in the same slice or the current block is at the frame boundary, etc.), a method called reference sample replacement can be performed, in which the missing samples are copied clockwise from the available samples.

[0072] According to the present disclosure, the block size does not need to be limited to N×N (i.e., a square block), and may be M×N (i.e., a general rectangular block, including a square block), where M is equal to or different from N. For a general block size of M×N, there may be a row of 2M reconstructed reference samples on the upper side of the current block, and a column of 2N reconstructed reference samples on the left side of the current block.

[0073] Intra-frame sample prediction Figure 8 The method is executed in module 820 and includes predicting pixels of a target CU based on a reference sample. As mentioned above, in order to effectively predict different types of content, HEVC supports a series of prediction models. Planar and DC prediction modes are used to predict smooth and gradually changing areas, while angular prediction modes are used to capture different directional structures. HEVC supports 33 directional prediction modes with indices from 2 to 34. Directional prediction modes correspond to different prediction directions.

[0074] Fig.10 Intra-prediction directions according to the HEVC standard and according to the present disclosure are shown. However, the present disclosure is not limited to HEVC directions. Numbers 2 to 34 represent prediction mode indices associated with the corresponding directions. Modes 2 to 17 indicate horizontal prediction (H-26 to H+32), and modes 18 to 34 indicate vertical prediction (V-32 to V+32). Fig.10 The symbols "H" and "V" in are used to indicate the horizontal and vertical directions, respectively, while the numeric portion of the identifier represents the displacement of the pixel (also called the "angular parameter").

[0075] The angle parameter A represents the position of the reference sample (resolution is 1 / 32 pixel) relative to the target pixel on the first row or first column. The values ​​of A for different prediction modes are shown in Tables 1 and 2.

[0076]

[0077] Table 1

[0078]

[0079] Table 2

[0080] Directions with non-negative displacements (i.e., H0 to H+32 and V0 to V+32) are also denoted as positive directions, and directions with negative displacements (i.e., H-2 to H-26 and V-2 to V-32) are also denoted as negative directions. A positive prediction direction may also be defined as a direction with a positive A value, and a negative prediction direction may be defined as a direction with a negative A value.

[0081] like Fig.10 As shown, the sample precision of the defined angle direction is 1 / 32. That is, the interval between two pixels in the horizontal or vertical direction is divided into 32 sub-intervals. As described above, the defined direction can be distinguished as vertical or horizontal. The prediction mode in the horizontal direction uses only the left reference sample, or uses some left reference samples and some upper reference samples. Similarly, the prediction mode in the vertical direction uses only the upper reference sample, or uses some upper reference samples and some left reference samples. The horizontal positive direction from H0 to H+32 uses only the left reference sample for prediction. Similarly, the vertical positive direction from V0 to V+32 uses only the upper reference sample for prediction. Negative horizontal and negative vertical directions (H-2 to H-26 and V-2 to V-32) use both the left reference sample and the upper reference sample for prediction.

[0082] In the HEVC reference code, a reference array is first constructed using the upper and left reference samples. For vertical prediction, the reference array is horizontal, and for horizontal prediction, the reference array is vertical. For modes with a positive angle parameter A (modes 2 to 10 and modes 26 to 34), the reference array is just the upper or left reference samples, depending on the direction:

[0083] For vertical prediction, topRef[x]=P[x-1][-1], 0≤x≤2N (1)

[0084] For horizontal prediction, leftRef[y]=P[-1][y-1], 0≤y≤2N (2)

[0085] Where N is the CU size. The sample coordinates are usually initialized to (0,0) at the top left pixel of the target CU. Therefore, the y coordinate of the upper reference sample is -1, and the x coordinate of the left reference sample is -1.

[0086] For modes with negative angle parameter A (modes 11 to 25), the reference array requires pixels from both the upper and left references. In this case, the reference array will be further extended to negative indices beyond -1. Sample values ​​on the reference array with positive indices are obtained as described above according to the vertical or horizontal prediction. Those pixels on the reference array with negative indices are obtained by projecting the left reference pixels (for vertical prediction) or the upper reference pixels (for horizontal prediction) on the reference array along the prediction direction.

[0087] Once the reference array is constructed, the prediction for any pixel position inside the target CU is obtained by projecting the pixel position along the chosen direction to the reference array and then copying the reference array sample value at (x,y). The reference sample value is calculated at a sample resolution of (1 / 32) by interpolating between two adjacent samples as shown below:

[0088] For vertical prediction, P[x][y]=((32-f)*topRef[x+i+1]+f*topRef[x+i+2]+16)>>5), 0≤x,y <N (3)

[0089] For horizontal prediction, P[x][y]=((32-f)*leftRef[y+i+1]+f*leftRef[y+i+2]+16)>>5),0≤x,y <N (4)

[0090] where i and f represent the integer and fractional parts of the displacement projected from the pixel position (x, y), and >> represents a rightward shift operation. If Δ represents the projected displacement, then

[0091] For horizontal prediction, Δ=(x+1)*A (5)

[0092] For vertical prediction, Δ=(y+1)*A (6)

[0093] Then, the integer and fractional parts of the displacement are obtained as follows:

[0094] i=Δ>>5 (7)

[0095] f=Δ&31 (8)

[0096] Where & represents a bitwise AND operation. Note that if f = 0, i.e., there is no fractional part, then the prediction is equal to the reference array sample value in the prediction direction. In this case, no interpolation is required.

[0097] Some of the prediction modes, such as DC mode, as well as direct horizontal mode (H0 or 10) and vertical mode (V0 or 26), may cause discontinuities at CU boundaries after prediction. Therefore, in HEVC and in this disclosure, such prediction modes are followed by Figure 8 The post-processing module 830 performs a post-processing step in which the boundary prediction samples are smoothed using a low-pass filter.

[0098] As mentioned before, to construct a prediction for a target block, the encoder (and decoder) can use only one row of reference samples on the upper side of the block and one column of reference samples on the left side of the block. Fig. 9 The reference samples in carry the largest amount of correlation with the target block content, so using additional decoded rows and columns is considered unnecessary due to higher complexity and memory requirements. But this logic is applicable when the target block size is small and there are only a few angular prediction modes. As the block size and number of prediction modes increase, the prediction from one reference row and one reference column can be made more accurate by using the directional information from additional reference rows and columns.

[0099] Since the block may no longer be square, the rectangular block shape may be disproportionately narrow in the horizontal or vertical direction. Therefore, the prediction mode and filtering performance of the reference sample may deviate depending on the shape of the block, such as Fig.11 and Fig.12 The comparison is shown.

[0100] Fig.11 An exemplary intra prediction mode for a square block shape according to the HEVC standard is shown. 1100 shows: a) a horizontal prediction example and b) a vertical prediction example with corresponding prediction directions. It can be observed that the block shape and the prediction direction are not discriminant, that is, there is a similarity between horizontal and vertical predictions for a square shape. The same applies to the intra prediction mode for a square block shape according to the present disclosure.

[0101] Fig.12 An exemplary intra prediction mode for a rectangular block shape according to the present disclosure is shown. 1200 shows: a) horizontal prediction example of a vertical rectangular block, b) vertical prediction example of a vertical rectangular block, c) horizontal prediction example of a horizontal rectangular block, and d) vertical prediction example of a horizontal rectangular block with corresponding prediction directions. It can be observed that in examples a) and d), the predicted pixels are close to the reference sample. However, in cases b) and c), the predicted pixels are far away from the reference sample, which is undesirable because correlation tends to decrease with distance.

[0102] In addition, propagating strong edges that are not aligned with block structure or texture may produce high frequencies in the residual, resulting in high-cost residuals to be encoded. In order to achieve a good balance between generating accurate predictions and avoiding erroneous strong edges, it is useful to low-pass filter the reference samples in some cases. In the HEVC standard, intra-frame reference samples are filtered according to the absolute value of the (minimum) angle between the pure horizontal direction or the pure vertical direction and the intra-frame prediction direction (module 814). The threshold or value depends on the block size (the number of pixels in the block). The threshold or value is independent of the block shape. If the angle is greater than the threshold or value, the reference sample smoothing filter is applied.

[0103] As a result, reference samples used for intra prediction are sometimes filtered with a three-tap [1 2 1] / 4 smoothing filter. The HEVC standard applies smoothing operations adaptively depending on the directionality and block size. The smoothing filter is not applied to 4×4 blocks. For 8×8 blocks, only the diagonal directions (Intra-Angular [k], i.e., k = 2, 18, or 34) use reference sample smoothing. For 16×16 blocks, reference samples in most directions are filtered except for the near-horizontal and near-vertical directions (i.e., k is in the range of 9–11 and 25–27). For 32×32 blocks, a smoothing filter is used in all directions except the horizontal (k = 10) and vertical (k = 26) directions. When the block size is greater than or equal to 8×8, the intra-plane mode also uses a smoothing filter, and smoothing is not used (or is not useful) for the DC intra case.

[0104] According to the present disclosure, it is therefore desirable to determine the intra prediction mode based on the block shape. Furthermore, it is desirable to determine the filtering tools based on the block shape and the possible intra prediction directions.

[0105] In one embodiment according to the present disclosure, a function, characteristic or condition is determined according to or in association with a block shape for enabling or disabling reference smoothing filtering (module 810). The smoothing filtering may be a low-pass filtering process. The smoothing filtering may be performed by a similar filter used in the HEVC standard. The function or characteristic may be defined as a block width for a horizontal rectangular block, a block height for a vertical rectangular block, and a block width or block height for a square block. The function or characteristic may be equivalently defined as, for example, a maximum size of a block and applied to the reference smoothing filtering. In one embodiment, if the maximum size is the width of the block (e.g., a horizontal rectangular block, such as Fig.12 ), then for reference smoothing filtering (810) of the reference sample set associated with the block, at least one vertical intra prediction mode ( Fig.10 If the maximum dimension is the height of the block (e.g., a vertical rectangular block such as Fig.12), then for reference smoothing filtering (810) of the reference sample set associated with the block, at least one horizontal intra prediction mode ( Fig.10 Otherwise, if the width and height of the block have similar dimensions, horizontal and vertical intra prediction modes may be enabled to perform reference smoothing filtering on a reference sample set associated with the block (810).

[0106] In other words, if the prediction directionality (horizontal or vertical) is not consistent with the block shape (defined by the maximum size, e.g. horizontal prediction with width > height, or vertical prediction with width < height, Fig.12 If the prediction is the same as in cases "c" and "b"), the prediction is considered long. Otherwise ( Fig.12 For short prediction, at least one intra-directional prediction mode may be disabled for reference smoothing filtering (810).

[0107] In one embodiment, the reference sample smoothing filter can be disabled for short prediction. For example, the reference smoothing filter is only applicable to Fig.12 Items "b" and "c" in .

[0108] In one embodiment according to the present disclosure, a function, characteristic or condition may be defined as the diagonal of the block. In one embodiment, a function, characteristic or condition may be defined as a weighted sum of the size of the block, such as 2*width+height. In one embodiment, other more complex functions may be designed to distinguish different shapes of blocks.

[0109] In one embodiment according to the present disclosure, the function, characteristic or condition may include a comparison of a shape threshold value / value or range of values ​​associated with the shape of the block. In one embodiment, the shape threshold value / value or range of values ​​may be associated with the block width if the prediction is horizontal, or with the block height if the prediction is vertical. In other words, the shape threshold value / value or range of values ​​may be associated with the maximum size of the block. For example, if the maximum size of the block is greater than the shape threshold value p=4, a reference smoothing filter may be applied.

[0110] In one embodiment according to the present disclosure, the function, characteristic or condition may further include a comparison of a direction threshold value / value or a range of values ​​associated with the prediction direction of the block. For example, if the maximum size of the block is greater than the shape threshold value p=4, and the prediction mode index k is less than 10 for a horizontal rectangular block or greater than 26 for a vertical rectangular block, a reference smoothing filter may be applied.

[0111] In one embodiment according to the present disclosure, a more complex function of the block shape and the prediction direction of the block can be established. For example, the shape threshold can depend on the prediction direction: for the horizontal direction, if the block width is greater than p, a filter is applied, where p=4 for k=9 to 11, p=16 for k<18, which means that p=16 is applied for almost all horizontal directions except k=9 to 11 (where p=4). For the vertical direction, if the block height is greater than p, a filter is applied, where p=4 for k=25 to 27, p=16 for k>18, which means that p=16 is applied for almost all vertical directions except k=25 to 27 (where p=4).

[0112] In one embodiment, at least one flag may be included and optionally encoded in the bitstream (e.g., as a syntax element) to indicate the shape threshold value or value for at least one of the current picture, slice, or block. The at least one flag may be retrieved at a decoder and used to decode the coded block.

[0113] In one embodiment according to the present disclosure, prediction, residual, transform and encoding can be performed with and without reference sample smoothing filtering. Between the two options, the option that produces better rate-distortion (RD) performance can be selected.

[0114] In one embodiment, at least one flag may be included and optionally encoded in the bitstream (e.g., as a syntax element) to indicate whether reference sample smoothing filtering is enabled / disabled for at least one of the current picture, slice, or block. The at least one flag may be retrieved at a decoder and used to decode the coded block.

[0115] In one embodiment, a one-bit flag at the CU level can be used to signal the selected prediction to the decoder. The flag can be encoded using CABAC using a context that depends on the prediction direction and block shape. For example, if the prediction is horizontal and width>height, or the prediction is vertical and height>width, context 1 can be used, otherwise context 2 can be used. The flag can be retrieved at the decoder and used to decode the coded block.

[0116] In one embodiment according to the present disclosure, a function, characteristic or condition is determined based on the block shape for enabling or disabling intra prediction mode (modules 820, 830). The function or characteristic can be defined as, for example, the maximum size of a block and applied to intra prediction. In one embodiment, if the maximum size is the width of the block (e.g., a horizontal rectangular block, such as Fig.12), at least one vertical intra prediction mode may be disabled to generate an intra prediction associated with the block (820, 830). If the maximum size is the height of the block (e.g., a vertical rectangular block, such as Fig.12 ), at least one horizontal intra prediction mode may be disabled to generate an intra prediction associated with the block (820, 830). Otherwise, if both the width and height of the block have similar sizes, both horizontal and vertical intra prediction modes may be enabled to generate an intra prediction associated with the block (820, 830).

[0117] In other words, if the prediction is short, at least one prediction mode may be disabled for intra prediction (820, 830).

[0118] In one embodiment, for short prediction, all (vertical or horizontal) prediction modes may be disabled. For example, a horizontal rectangular block is only predicted horizontally ( Fig.12 ), and vertical rectangular blocks are only predicted vertically ( Fig.12 Item “b” in .

[0119] In one embodiment according to the present disclosure, the function, characteristic or condition may include a comparison of a shape threshold / value or a range of values ​​associated with the shape of the block. In one embodiment, the shape threshold / value or range of values ​​may be associated with the block width if the prediction is horizontal, or associated with the block height if the prediction is vertical. In other words, the shape threshold / value or range of values ​​may be associated with the maximum size of the block. For example, the function, characteristic or condition may be the maximum size of the block that is greater than the shape threshold p=4. If the condition is true, at least one intra-frame prediction mode is allowed / selected for the block, strip or picture. In one example, the prediction direction (index) is available even only when the condition for the block shape is true. In another example, one of 4 (a multiple of 4) prediction directions (indexes) is available only when the condition is true.

[0120] In one embodiment according to the present disclosure, the function, characteristic or condition may further include a comparison of a direction threshold / value or a range of values ​​associated with or with the prediction direction of the block. For example, the function, characteristic or condition may be a maximum size of the block greater than a shape threshold p=4, and a prediction mode index k less than 10 for a horizontal rectangular block or greater than 26 for a vertical rectangular block. If the condition is true, at least one intra-frame prediction mode is allowed / selected for a block, strip or picture. In one example, the prediction direction (index) is available only when the condition of the block shape is true. In another example, one of 4 (a multiple of 4) prediction directions (indexes) is available only when the condition is true.

[0121] In one embodiment according to the present disclosure, a more complex function of the block shape and the prediction direction of the block can be established. If the condition is true, at least one intra prediction mode is allowed / selected for the block, slice or picture. In one example, the prediction direction (index) is only available when the condition of the block shape is true. In another example, one of 4 (multiples of 4) prediction directions (indexes) is only available when the condition is true.

[0122] In one embodiment according to the present disclosure, any of the above embodiments associated with the intra prediction mode (820, 830) is applicable to PDPC index coding. In another embodiment, any of the above embodiments is applicable to RSAF index coding. In yet another embodiment, any of the above embodiments is applicable to multiple reference sample switching. That is, if the condition is not met, intra prediction is performed using a single row or single column reference as in HEVC, otherwise multi-reference prediction is used. Multi-reference intra prediction refers to intra prediction using multiple rows and columns of reference pixels. It is also called arbitrary layer (tier) reference intra prediction or multi-row intra prediction. Where known, it is proposed to use weighted multi-references, where a weight is associated with each reference line (or layer).

[0123] As a variation, this condition may apply to any of the coding tools (PDPC, RSAF, multi-sample reference) associated with intra prediction described in the above embodiments. When the condition is met, the tool (i.e., prediction method) is always used, otherwise the tool is not used. In another variation, when the condition is met, the encoder chooses to use or not use the tool (with a classic RDO loop) and sends a flag indicating whether the tool has been used; otherwise (if the condition is not met), the tool is not used.

[0124] Fig.13 A flowchart 1300 of an exemplary method of video encoding according to an embodiment of the present disclosure is shown. The method 1300 includes, at step 1310, accessing a reference sample set for prediction of a block in a picture of a video. Then, at step 1320, the method 1300 includes processing the reference sample set based on the shape of the block. Next, at step 1330, the method 1300 includes generating a prediction block of the block based on the processed reference sample set. Finally, at step 1340, the method 1300 includes encoding the block based on the prediction block. Steps 1310 to 1340 can be performed, for example, by the encoder 700 (e.g., 760, 770, 775), including module 800. Specifically, steps 1310 to 1330 can be performed, for example, by modules 760, 800, 770, 775, including module 810 for step 1320 and modules 820, 830 for step 1330. The block shape can be, for example, one of a square shape, a vertical rectangular shape, and a horizontal rectangular shape.

[0125] According to an embodiment of the method, the prediction may be intra-frame prediction or inter-frame prediction.

[0126] According to one embodiment of the method, the process may be enabled or selected when a function or characteristic of or associated with a block shape is greater than a value or threshold. Thus, when the function or characteristic is greater than the value or threshold, the condition is true. In one embodiment, the process may be disabled or not selected when the function or characteristic is less than or equal to the value.

[0127] According to one embodiment of the method, the function or characteristic of the block shape or the function or characteristic associated with the block shape may be the maximum size of the block. In one embodiment, when the block shape is a horizontal rectangular shape, the maximum size is the block width; when the block shape is a vertical rectangular shape, the maximum size is the block height; when the block shape is a square, the maximum size is the block width or the block height.

[0128] According to one embodiment of the method, the function or characteristic may be the diagonal length of the block.

[0129] According to one embodiment of the method, the function or characteristic may be a weighted sum of the dimensions of the block (eg, 2*width+height).

[0130] According to an embodiment of the method, the processing may be further based on the prediction mode.In an embodiment, the processing may be further based on a directional mode of the prediction of the block.

[0131] According to one embodiment of the method, when the function or characteristic is greater than the value, the process can be enabled or selected for at least one prediction mode. For example, the process can be enabled for the horizontal prediction mode in the horizontal rectangular block and the vertical prediction mode in the vertical rectangular block.

[0132] According to one embodiment of the method, for non-square blocks, the process may be disabled or not selected for at least one directional prediction mode in the direction of the smallest size of the block. For example, the process may be disabled for a horizontal prediction mode in a vertical rectangular block and / or a vertical prediction mode in a horizontal rectangular block.

[0133] According to one embodiment of the method, at least one flag is included in the encoded video, the at least one flag indicating at least one of a value and whether processing is enabled. The flag may be retrieved at the decoder before or during prediction.

[0134] According to an embodiment of the method, the processing includes smoothing filtering the reference sample set. The smoothing filtering may be a low-pass filtering.

[0135] According to one embodiment of the method, the prediction is based on block shape. In one embodiment, for at least one block shape, at least one prediction mode (e.g., intra prediction direction) is allowed. In one embodiment, for at least one block shape, at least one prediction mode is not allowed. In one embodiment, as described above, when the condition of the block shape is true, at least one prediction mode is enabled or disabled.

[0136] It should be understood that method 1300 is also applicable to any of the additional embodiments and examples previously described in the present disclosure.

[0137] According to one embodiment, the method may further include receiving a picture, partitioning the picture into a plurality of blocks including a block, determining a prediction residual of the block, transforming and quantizing the residual to obtain a plurality of transform coefficients, and entropy encoding the transform coefficients. The steps of transforming and quantizing may be performed, for example, by modules 725 and 730 of encoder 700. The step of entropy encoding may be performed, for example, by module 745 of encoder 700. The steps of receiving, transforming, and quantizing may be optional, bypassed, or removed because they may have been pre-performed by another device and / or the results may have been stored in memory.

[0138] It should be understood that any of the embodiments of the above method 1300 may be implemented by the encoder 700 (e.g., 760, 770, 775) including the intra-frame prediction module 800. The blocks of the encoder 700 including the intra-frame prediction module 800 may be implemented by hardware (e.g., integrated circuits) or software, which is stored in a memory and executed by a processor.

[0139] decoding

[0140] Fig.14 1 shows a simplified block diagram of an exemplary video decoder 1400 according to an embodiment of the present disclosure. The video decoder 1400 may be included in a receiver of a communication system. Although not all operations in the video decoder 1400 are Figure 7The decoder 1400 performs the inverse operation of the encoding process (e.g., intra-frame prediction and inter-frame prediction) performed by the video encoder 700 described in , but the decoder generally performs a decoding process that is inverse to the encoding process. Specifically, the input of the decoder 1400 includes a video bitstream that can be generated by the video encoder 700. The bitstream is first entropy decoded (module 1430) to obtain transform coefficients, motion vectors, syntax elements and other encoding information. The transform coefficients are dequantized (module 1440) and inversely transformed (module 1450) to decode the residual. The decoded residual is then combined with a predicted sample block (also referred to as a predictor) (module 1455) to obtain a decoded / reconstructed image block. The encoder decides (e.g., module 705) which of the intra-frame mode or inter-frame mode is used to encode the block, and indicates the intra-frame / inter-frame decision through a prediction mode flag. The predicted sample block can be obtained (module 1405) from intra-frame prediction (module 1460) or motion compensated prediction (i.e., inter-frame prediction) (module 1470). An in-loop filter may be applied to the reconstructed image (module 1465). The in-loop filter may include a deblocking filter and a SAO filter. The filtered image is stored in a reference picture buffer 1480.

[0141] The modules of the video decoder 1400 may be implemented in software and executed by a processor, or may be implemented using circuit components known to those skilled in the art of compression. Specifically, the video encoder 1400 may be implemented as an integrated circuit (IC), alone or in combination with the video decoder 700 as a codec.

[0142] In addition to the differences described in the present disclosure, in particular, in addition to the differences described in more detail in the following paragraphs and the accompanying drawings, Figure 7 The difference between the intra prediction module 1460 of the module 760 or the Figure 7 Except for the difference of the intra-frame prediction module 1475 of the modules 770 and 775, the modules of the video decoder 1400 also exist in other video decoders (e.g., HEVC decoders). For functions other than the intra-frame prediction module 1460 and / or the inter-frame prediction 1475, the video decoder 1400 can be similar to the HEVC video decoder, and these functions are not described in detail herein.

[0143] In addition, the intra prediction module 1460 can be similar to Figure 7 The intra prediction module 760 and Figure 8 And the motion compensation module 1470 can be similar to Figure 7 The motion compensation module 770 is used.

[0144] Fig.15A flowchart 1500 of an exemplary method of video decoding according to an embodiment of the present disclosure is shown. The method 1500 includes, at step 1510, accessing a reference sample set for prediction of a block in a picture of an encoded video. Then, at step 1520, the method 1500 includes processing the reference sample set based on the shape of the block. Next, at step 1530, the method 1500 includes generating a prediction block of the block based on the processed reference sample set. Finally, at step 1540, the method 1500 includes decoding the block based on the prediction block. Steps 1510 to 1540 can be performed, for example, by the decoder 1400 (for example, including module 800). Specifically, steps 1510 to 1530 can be performed, for example, by modules 1460, 800, 1475, including module 810 for step 1520 and modules 820, 830 for step 1530. The block shape can be, for example, one of a square shape, a vertical rectangular shape, and a horizontal rectangular shape.

[0145] According to an embodiment of the method, the prediction may be intra-frame prediction or inter-frame prediction.

[0146] According to one embodiment of the method, the process may be enabled or selected when a function or characteristic of or associated with a block shape is greater than a value or threshold. Thus, when the function or characteristic is greater than the value or threshold, the condition is true. In one embodiment, the process may be disabled or not selected when the function or characteristic is less than or equal to the value.

[0147] According to one embodiment of the method, the function or characteristic or property of or associated with the block shape may be the maximum size of the block. In one embodiment, when the block shape is a horizontal rectangular shape, the maximum size is the block width; when the block shape is a vertical rectangular shape, the maximum size is the block height; when the block shape is a square, the maximum size is the block width or the block height.

[0148] According to one embodiment of the method, the function or characteristic may be the diagonal length of the block.

[0149] According to one embodiment of the method, the function or characteristic may be a weighted sum of the dimensions of the block (eg, 2*width+height).

[0150] According to an embodiment of the method, the processing may be further based on a prediction mode. In an embodiment, the processing may be further based on a directional mode of the prediction of the block.

[0151] According to one embodiment of the method, when the function or characteristic is greater than a value, the process may be enabled or selected for at least one prediction mode. For example, the process may be enabled for a horizontal prediction mode in a horizontal rectangular block and a vertical prediction mode in a vertical rectangular block.

[0152] According to one embodiment of the method, for non-square blocks, the process may be disabled or not selected for at least one directional prediction mode in the direction of the smallest size of the block. For example, the process may be disabled for a horizontal prediction mode in a vertical rectangular block and / or a vertical prediction mode in a horizontal rectangular block.

[0153] According to one embodiment of the method, at least one flag is included in the encoded video, the at least one flag indicating at least one of a value and whether the process is enabled. The flag may be retrieved at the decoder before or during prediction.

[0154] According to an embodiment of the method, the processing includes smoothing filtering the reference sample set. The smoothing filtering may be a low-pass filtering.

[0155] According to one embodiment of the method, the prediction is based on block shape. In one embodiment, for at least one block shape, at least one prediction mode (e.g., intra prediction direction) is allowed. In one embodiment, for at least one block shape, at least one prediction mode is not allowed. In one embodiment, as described above, when the condition of the block shape is true, at least one prediction mode is enabled or disabled.

[0156] It should be understood that method 1500 is also applicable to any of the additional embodiments and examples previously described in this disclosure in association with method 1300.

[0157] According to one embodiment, the method may further include receiving a coded picture, entropy decoding the coded block, inverse transforming the transform coefficient block to obtain a decoded residual, combining the decoded residual with the prediction sample block to obtain a decoded / reconstructed image block. The transform coefficients may be further inverse quantized before inverse transforming. The steps of entropy decoding, inverse transforming and inverse quantizing may be performed by, for example, modules 1430, 1450 and 1440 of the decoder 1400, respectively. The steps of receiving, entropy decoding, inverse transforming and inverse quantizing and combining may be optional, bypassed or removed because they may have been pre-performed by another device and / or provided to another device, or the results may have been retrieved from a memory and / or stored in a memory.

[0158] It should be understood that any of the embodiments of the above method 1500 can be implemented by the decoder 1400 (e.g., 1460, 800, 1475). The modules of the decoder 1400 can be implemented by hardware (e.g., integrated circuits) or in software, stored in a memory and executed by a processor.

[0159] Fig.16A block diagram 1600 of an exemplary system in which various aspects of the exemplary embodiments of the present disclosure may be implemented is shown. The system 1600 may be embodied as a device including the various components described below and configured to perform the above-described processes. Examples of such devices include, but are not limited to, personal computers, laptop computers, smart phones, smart watches, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home appliances, and servers. The system 1600 may be communicatively coupled to other similar systems and communicated with via, for example, Fig.16 The communication channel shown is coupled to a display and is used to implement the exemplary video system described above as known to those skilled in the art. System 1600 can independently or jointly implement encoder 700 (e.g., including module 800), decoder 1400 (e.g., including module 800), or (multiple) encoders and (multiple) decoders. In addition, system 1600 can be independently or jointly implemented and configured to perform any of the processes of the present disclosure, including method 1300 and / or 1500.

[0160] The system 1600 may include at least one processor 1610, which is configured to execute instructions loaded therein for implementing various processes as described above. The processor 1610 may include embedded memory, input and output interfaces, and various other circuits known in the art. The system 1600 may also include at least one memory 1620 (e.g., a volatile memory device such as RAM, a non-volatile memory device such as ROM). The system 1600 may additionally include a storage device 1640, which may include a non-volatile memory, including but not limited to an erasable programmable read-only memory (EPROM), a ROM, a programmable read-only memory (PROM), a dynamic RAM (DRAM), a static RAM (SRAM), a flash memory, a disk drive, and / or an optical drive. As a non-limiting example, the storage device 1640 may include an internal storage device, an attached storage device, and / or a network accessible storage device. The system 1600 may also include an encoder / decoder module 1630 configured to process data to provide encoded video or decoded video.

[0161] The encoder / decoder module 1630 represents a module that may be included in the device to perform encoding and / or decoding functions, such as according to Figure 7 (For example, including Figure 8 )and Fig.14 (For example, including Figure 8). As is known in the art of compression, the device may include one or both of the encoding and decoding modules. In addition, the encoder / decoder module 1630 may be implemented as a separate element of the system 1600, or may be incorporated into the processor 1610 as a combination of hardware and software known to those skilled in the art. For example, the encoder / decoder module 1630 may be implemented as one or two separate integrated circuits and / or field-programmable gate arrays (FPGAs).

[0162] Program code to be loaded onto the processor 1610 to perform the various processes described above may be stored in the storage device 1640 and subsequently loaded onto the memory 1620 to be executed by the processor 1610. According to an exemplary embodiment of the present disclosure, one or more of the processor(s) 1610, the memory 1620, the storage device 1640, and the encoder / decoder module 1630 may store one or more of the various items including, but not limited to, input video, decoded video, bitstreams, equations, formulas, matrices, variables, operations, and operation logic during the execution of the processes discussed above.

[0163] The system 1600 may also include a communication interface 1650 capable of communicating with other devices via a communication channel 1660. The communication interface 1650 may include, but is not limited to, a transceiver configured to send and receive data from the communication channel 1660. The communication interface may include, but is not limited to, a modem or a network card, and the communication channel may be implemented in a wired and / or wireless medium. The various components of the system 1600 may be connected or communicatively coupled together using a variety of suitable connections, including, but not limited to, internal buses, wires, and printed circuit boards.

[0164] According to the exemplary embodiments of the present disclosure, the computer software executed by the processor 1610 may be executed by hardware or by a combination of hardware and software. As a non-limiting example, according to the exemplary embodiments of the present disclosure, one or more integrated circuits may be implemented. As a non-limiting example, the memory 1620 may be any type suitable for the technical environment and may be implemented using any appropriate data storage technology, such as an optical storage device, a magnetic storage device, a semiconductor-based storage device, a fixed memory, and a removable memory. As a non-limiting example, the processor 1610 may be any type suitable for the technical environment and may include one or more of a microprocessor, a general-purpose computer, a special-purpose computer, and a processor based on a multi-core architecture.

[0165] The embodiments described herein can be implemented with, for example, methods or processes, devices, software programs, data streams or signals. Even if only discussed in the context of a single form of the embodiment (e.g., discussed only as a method), the embodiments of the features discussed can also be implemented in other forms (e.g., devices or programs). The device can be implemented with, for example, appropriate hardware, software and firmware. These methods can be implemented in, for example, a device (such as, for example, a processor, which generally refers to a processing device, including, for example, a computer, a microprocessor, an integrated circuit or a programmable logic device). The processor also includes a communication device, such as, for example, a computer, a cellular phone, a portable / personal digital assistant (PDA), and other devices that facilitate information communication between end users.

[0166] According to one aspect of the present disclosure, a device 1600 for video encoding is provided. The device includes a processor 1610 and at least one memory 1620 and 1640 coupled to the processor. The processor 1610 is configured to execute any embodiment of the above-mentioned video encoding method 1300.

[0167] According to one aspect of the present disclosure, a device 1600 for video decoding is provided. The device includes a processor 1610 and at least one memory 1620 and 1640 coupled to the processor. The processor 1610 is configured to execute any embodiment of the above-mentioned video decoding method 1500.

[0168] According to an aspect of the present disclosure, there is provided an apparatus for video encoding, comprising: a component for accessing a reference sample set for prediction of a block in a picture of a video; a component for processing the reference sample set based on a shape of the block; a component for generating a prediction block of the block based on the processed reference sample set; and a component for encoding the block based on the prediction block. Figure 7 (For example, including Figure 8 )and Fig.16 The video encoder may include structures or components of the apparatus, in particular modules 760 (eg, 800), 770, 775, 1710, and 1730. The apparatus for video encoding may perform any embodiment of any one of the methods 1300 of video encoding.

[0169] According to one aspect of the present disclosure, there is provided an apparatus for video decoding, comprising: a component for accessing a reference sample set for encoding a prediction of a block in a picture of a video; a component for processing the reference sample set based on a shape of the block; a component for generating a prediction block of the block based on the processed reference sample; and a component for encoding the block based on the prediction block. Fig.14 (For example, including Figure 8) and FIG. 17 may include structures or components of an apparatus for video decoding, particularly blocks 1460 (eg, 800), 1475, 1710, and 1730. The apparatus for video decoding may perform any embodiment of any one of the methods 1500 for video decoding.

[0170] It will be apparent to those skilled in the art that embodiments may generate various formatted signals to carry information that may be stored or transmitted, for example. The information may include, for example, instructions for executing a method, or data generated by one of the described embodiments. For example, a signal may be formatted to carry a bitstream of the described embodiments. Such a signal may be formatted as, for example, an electromagnetic wave (e.g., using a radio frequency portion of a spectrum) or a baseband signal. Formatting may include, for example, encoding a data stream and modulating a carrier with the encoded data stream. The information carried by the signal may be, for example, analog or digital information. As is well known, a signal may be transmitted over a variety of different wired or wireless links. The signal may be stored on a processor readable medium.

[0171] According to an aspect of the present disclosure, a signal comprises a bitstream formatted to include coded data of blocks representing a picture, the coded data being encoded according to any embodiment of any one of the methods 1300 of video encoding.

[0172] According to an aspect of the present disclosure, a bitstream is formatted to include coded data representing blocks of a picture, which is encoded according to any embodiment of any one of the methods 1300 of video encoding.

[0173] In addition, any of the methods 1300 and / or 1500 may be implemented as a computer program product comprising computer executable instructions that may be executed by a processor (independently or jointly). The computer program product having computer executable instructions may be stored in a corresponding transient or non-transitory computer readable storage medium of the system 1600, the encoder 700 (e.g., including the module 800), and / or the decoder 1400 (e.g., including the module 800).

[0174] According to an aspect of the present disclosure, a computer program product is provided, which includes program code instructions for executing (independently or jointly) any embodiment of any one of the methods 1300 and / or 1500 of the present disclosure.

[0175] It is important to note that in some embodiments, one or more elements of process 1300 and / or 1500 may be combined, performed in a different order, or eliminated while still implementing aspects of the present disclosure. Other steps may be performed in parallel, where the processor does not wait for one step to be fully completed before starting another step.

[0176] In addition, aspects of the present disclosure may take the form of a computer-readable storage medium. Any combination of one or more computer-readable storage media may be utilized. A computer-readable storage medium may take the form of a computer-readable program product embodied in one or more computer-readable media and having a computer-readable program code embodied thereon that can be executed by a computer. The computer-readable storage medium used herein is considered to be a non-transitory storage medium that has an inherent ability to store information therein and an inherent ability to provide retrieval of information therefrom. A computer-readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any suitable combination of the foregoing.

[0177] It should be understood that the following list, while providing more specific examples of computer-readable storage media to which the present disclosure may be applied, is merely an illustrative, non-exhaustive list, as will be readily appreciated by one of ordinary skill in the art. The list of examples includes a portable computer disk, a hard disk, a ROM, an EPROM, a flash memory, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0178] According to an aspect of the present disclosure, a computer-readable storage medium carrying a software program is provided, the software program including program code instructions for executing any embodiment of any one of the methods (including method 1300 and / or 1500) of the present disclosure.

[0179] It should be understood that references to "one embodiment" or "an embodiment" or "one implementation" or "implementation" of the present disclosure and other variations thereof mean that a particular feature, structure, characteristic, etc. described in conjunction with the embodiment is included in at least one embodiment of the present disclosure. Therefore, the phrases "in one embodiment" or "in an embodiment" or "in one implementation" or "in an implementation" and any other variations appearing in various places in the specification do not necessarily refer to the same embodiment.

[0180] Additionally, this disclosure or its claims may refer to "determining" various information. Determining information may include, for example, one or more of estimating information, calculating information, predicting information, or retrieving information from a memory.

[0181] Additionally, the disclosure or its claims may refer to "providing" various information. Providing information may include, for example, one or more of outputting information, storing information, transmitting information, transferring information, displaying information, showing information, or moving information.

[0182] Additionally, this disclosure or its claims may refer to "accessing" various information. Accessing information can include, for example, one or more of receiving information, retrieving information (e.g., from a memory), storing information, processing information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information.

[0183] Furthermore, the disclosure or its claims may refer to "receiving" various information. Like "accessing," receiving is a broad term. Receiving information may include, for example, one or more of accessing information or retrieving information (e.g., from a memory). Furthermore, "receiving" is generally involved in one way or another during an operation such as storing information, processing information, transmitting information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information.

[0184] It should be understood that the various features shown and described are interchangeable. Unless otherwise specified, the features shown in one embodiment can be combined in another embodiment. In addition, the features described in the various embodiments can be combined or separated, unless otherwise specified as inseparable or non-combinable.

[0185] As previously mentioned, the functions of the various elements shown in the figures can be provided by using dedicated hardware and hardware capable of executing software associated with appropriate software. In addition, when provided by a processor, the functions can be provided by a single dedicated processor, a single shared processor, or multiple individual processors (some of which can be shared).

[0186] It should also be understood that because some of the constituent system components and methods depicted in the drawings are preferably implemented in software, the actual connections between the system components or process function blocks may vary depending on the manner in which the processes of the present disclosure are programmed. Given the teachings herein, one of ordinary skill in the relevant art will be able to conceive of these and similar implementations or configurations of the present disclosure.

[0187] Although illustrative embodiments have been described herein with reference to the accompanying drawings, it should be understood that the present disclosure is not limited to those precise embodiments, and that persons of ordinary skill in the relevant art may implement various changes and modifications therein without departing from the scope of the present disclosure. In addition, the various embodiments may be combined without departing from the scope of the present disclosure. All such changes and modifications are intended to be included within the scope of the present disclosure as set forth in the appended claims.

Claims

1. A video encoding method, comprising: In response to the sample block being a rectangular block with unequal width and height, in response to which dimension of the rectangular block is the largest between the width of the rectangular block and the height of the rectangular block, excluding at least one intra prediction mode from a plurality of intra prediction modes for the sample block based on a comparison of an index of the intra prediction mode with a first value or a range of first values, the first value or the range of first values ​​depending on which dimension is the largest between the width of the rectangular block and the height of the rectangular block; determining an intra prediction mode to be applied to the rectangular block in view of at least one excluded intra prediction mode; determining, based on the determined intra prediction mode, which reference samples in the reference sample set to use for intra prediction of the block; determining a characteristic of filtering to be applied to the reference samples based on a comparison of the index of the intra prediction mode with a value or a range of values ​​and the information representative of the shape of the block; filtering the set of reference samples using filtering based on the determined characteristic to obtain filtered reference samples; generating a prediction block for the block based on the filtered reference samples; as well as The rectangular block is encoded based on the prediction block in the video data.

2. The method of claim 1 , wherein the intra-prediction modes among the plurality of intra-prediction modes are further excluded based on a comparison of a characteristic representing the shape of the rectangular block with a second value, the characteristic representing the shape of the rectangular block being one of a weighted sum of a diagonal of the rectangular block and a size of the rectangular block.

3. An apparatus for video encoding, comprising an electronic circuit adapted to: In response to the sample block being a rectangular block with unequal width and height, in response to which dimension of the rectangular block is the largest between the width of the rectangular block and the height of the rectangular block, excluding at least one intra prediction mode from a plurality of intra prediction modes for the sample block based on a comparison of an index of the intra prediction mode with a first value or a range of first values, the first value or the range of first values ​​depending on which dimension is the largest between the width of the rectangular block and the height of the rectangular block; determining an intra prediction mode to be applied to the rectangular block in view of at least one excluded intra prediction mode; determining, based on the determined intra prediction mode, which reference samples in the reference sample set to use for intra prediction of the block; determining a characteristic of filtering to be applied to the reference samples based on a comparison of the index of the intra prediction mode with a value or a range of values ​​and the information representative of the shape of the block; filtering the set of reference samples using filtering based on the determined characteristic to obtain filtered reference samples; generating a prediction block for the block based on the filtered reference samples; as well as The rectangular block is encoded based on the prediction block in the video data.

4. The apparatus of claim 3 , wherein the intra-prediction modes among the plurality of intra-prediction modes are further excluded based on a comparison of a characteristic representing the shape of the rectangular block with a second value, the characteristic representing the shape of the rectangular block being one of a weighted sum of a diagonal of the rectangular block and a size of the rectangular block.

5. A method for video decoding, comprising: In response to the sample block being a rectangular block with unequal width and height, in response to which dimension of the rectangular block is the largest between the width of the rectangular block and the height of the rectangular block, excluding at least one intra prediction mode from a plurality of intra prediction modes for the sample block based on a comparison of an index of the intra prediction mode with a first value or a range of first values, the first value or the range of first values ​​depending on which dimension is the largest between the width of the rectangular block and the height of the rectangular block; determining an intra prediction mode to be applied to the rectangular block in view of at least one excluded intra prediction mode; determining, based on the determined intra prediction mode, which reference samples in the reference sample set to use for intra prediction of the block; determining a characteristic of filtering to be applied to the reference samples based on a comparison of the index of the intra prediction mode with a value or a range of values ​​and the information representative of the shape of the block; filtering the set of reference samples using filtering based on the determined characteristic to obtain filtered reference samples; generating a prediction block for the block based on the filtered reference samples; as well as The rectangular block is decoded based on the prediction block.

6. The method of claim 5 , wherein the intra-prediction modes among the plurality of intra-prediction modes are excluded based on a comparison of a characteristic representing the shape of the rectangular block with a second value, the characteristic representing the shape of the rectangular block being one of a weighted sum of a diagonal of the rectangular block and a size of the rectangular block.

7. An apparatus for video decoding, comprising an electronic circuit adapted to: In response to the sample block being a rectangular block with unequal width and height, in response to which dimension of the rectangular block is the largest between the width of the rectangular block and the height of the rectangular block, excluding at least one intra prediction mode from a plurality of intra prediction modes for the sample block based on a comparison of an index of the intra prediction mode with a first value or a range of first values, the first value or the range of first values ​​depending on which dimension is the largest between the width of the rectangular block and the height of the rectangular block; determining an intra prediction mode to be applied to the rectangular block in view of at least one excluded intra prediction mode; determining, based on the determined intra prediction mode, which reference samples in the reference sample set to use for intra prediction of the block; determining a characteristic of filtering to be applied to the reference samples based on a comparison of the index of the intra prediction mode with a value or a range of values ​​and the information representative of the shape of the block; filtering the set of reference samples using filtering based on the determined characteristic to obtain filtered reference samples; generating a prediction block for the block based on the filtered reference samples; as well as The rectangular block is decoded based on the prediction block.

8. The apparatus of claim 7 , wherein the intra-prediction modes among the plurality of intra-prediction modes are excluded based on a comparison of a characteristic representing the shape of the rectangular block with a second value, the characteristic representing the shape of the rectangular block being one of a weighted sum of a diagonal of the rectangular block and a size of the rectangular block.

9. A non-transitory computer-readable storage medium carrying a software program, the software program comprising program code instructions, wherein when the program code instructions are executed by a processor, the processor is caused to perform the method according to claim 1.

10. A non-transitory computer-readable storage medium carrying a software program, the software program comprising program code instructions, wherein when the program code instructions are executed by a processor, the processor is caused to perform the method according to claim 5.