Method and device for decoder-side intra-mode derivation

Abstract

In one method, a decoder receives a signal flag indicating whether a template-based intra mode derivation (TIMD) mode is applied to a current video block, and in response to determining that the TIMD mode is applied to the current video block based on the signal flag, derives an intra prediction mode from the TIMD mode based on different reference lines within a reconstructed area of ​​the current video block, the reconstructed area including reconstructed neighboring blocks of the current video block.

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application is based on and claims priority to Provisional Application No. 63 / 281,606, filed November 19, 2021, the entire contents of which are incorporated herein by reference for all purposes. FIELD OF THE DISCLOSURE This disclosure relates to video encoding and compression, and more particularly, to decoder-side intra mode derivation (DIMD). [Background technology]

[0002] Digital video is supported by a variety of electronic devices, including digital televisions, laptop or desktop computers, tablet computers, digital cameras, digital recording devices, digital media players, video gaming consoles, smart phones, video teleconferencing devices, and video streaming devices. Electronic devices transmit and receive or otherwise communicate digital video data over communication networks and / or store the digital video data on storage devices. Due to limited bandwidth capacity in communication networks and limited memory resources in storage devices, video coding may be used to compress video data according to one or more video coding standards before it is communicated or stored. Examples of video coding standards include Versatile Video Coding (VVC), Joint Exploration test Model (JEM), High-Efficiency Video Coding (HEVC / H.265), Advanced Video Coding (AVC / H.264), and Moving Picture Expert Group (MPEG) coding. Video coding generally utilizes prediction methods (e.g., inter-prediction, intra-prediction, etc.) that exploit the redundancy inherent in video data. Video coding aims to compress video data into a format that uses a lower bitrate while avoiding or minimizing degradation of video quality. Summary of the Invention [Problem to be solved by the invention]

[0003] Examples of this disclosure provide a video decoding method and apparatus using intra-prediction coding modes. [Means for solving the problem]

[0004] According to a first aspect of the present disclosure, a video decoding method is provided. The method may include receiving, by a decoder, a signal flag indicating whether a template-based intra mode derivation (TIMD) mode is applied to a current video block, and in response to determining that the TIMD mode is applied to the current video block based on the signal flag, deriving, by the decoder, an intra-prediction mode from the TIMD mode based on different reference lines within a reconstructed area of ​​the current video block, where the reconstructed area includes reconstructed neighboring blocks of the current video block.

[0005] According to a second aspect of the present disclosure, a video encoding method is provided. The method may include signaling, by an encoder, a flag indicating whether a template-based intra-mode derivation (TIMD) mode is applied to a current video block, and in response to determining that the TIMD mode is applied to the current video block based on the signal flag, deriving, by the encoder, an intra-prediction mode from the TIMD mode based on different reference lines within a reconstructed area of ​​the current video block, where the reconstructed area includes reconstructed neighboring blocks of the current video block.

[0006] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not intended to be restrictive of the present disclosure.

[0007] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate examples consistent with the present disclosure and, together with the description, serve to explain the principles of the disclosure. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a block diagram illustrating an example system for encoding and decoding video blocks, according to some implementations of the present disclosure. [Figure 2] 1 is a block diagram illustrating an example video encoder according to some implementations of the present disclosure. [Figure 3] 1 is a block diagram illustrating an example video decoder according to some implementations of the present disclosure. [Figure 4A] A block diagram showing how a frame is recursively partitioned into multiple video blocks of different sizes and shapes, according to some implementations of this disclosure. [Figure 4B] A block diagram showing how a frame is recursively partitioned into multiple video blocks of different sizes and shapes, according to some implementations of this disclosure. [Figure 4C] A block diagram showing how a frame is recursively partitioned into multiple video blocks of different sizes and shapes, according to some implementations of this disclosure. [Figure 4D] A block diagram showing how a frame is recursively partitioned into multiple video blocks of different sizes and shapes, according to some implementations of this disclosure. [Figure 4E] A block diagram showing how a frame is recursively partitioned into multiple video blocks of different sizes and shapes, according to some implementations of this disclosure. [Figure 5A] FIG. 10 is a diagram illustrating sample definitions used by the PDPC applied to prediction modes, according to some implementations of the present disclosure. [Figure 5B] FIG. 10 is a diagram illustrating sample definitions used by the PDPC applied to prediction modes, according to some implementations of the present disclosure. [Figure 5C] FIG. 10 is a diagram illustrating sample definitions used by the PDPC applied to prediction modes, according to some implementations of the present disclosure. [Figure 5D] FIG. 10 is a diagram illustrating sample definitions used by the PDPC applied to prediction modes, according to some implementations of the present disclosure. [Figure 6] FIG. 10 illustrates examples of allowed GPM divisions, according to some implementations of the present disclosure. [Figure 7] FIG. 10 illustrates an example of selected pixels on which gradient analysis is performed, according to some implementations of the present disclosure. [Figure 8] FIG. 1 illustrates a convolution process, according to some implementations of the present disclosure. [Figure 9] FIG. 10 illustrates prediction fusion by weighted average of two HoG modes and one planar mode, according to some implementations of the present disclosure. [Figure 10] A diagram showing a template and its reference sample used in a TIMD, in accordance with some implementations of the present disclosure. [Figure 11A] FIG. 1 is a block diagram illustrating a video decoding process using TIMD, according to some implementations of the present disclosure. [Figure 11B] FIG. 1 is a block diagram illustrating a video decoding process using TIMD, according to some implementations of the present disclosure. [Figure 11C] FIG. 1 is a block diagram illustrating a video decoding process using TIMD, according to some implementations of the present disclosure. [Figure 11D] FIG. 1 is a block diagram illustrating a video decoding process using TIMD, according to some implementations of the present disclosure. [Figure 12A] FIG. 1 is a block diagram illustrating a video decoding process using DIMD, according to some implementations of the present disclosure. [Figure 12B] FIG. 1 is a block diagram illustrating a video decoding process using DIMD, according to some implementations of the present disclosure. [Figure 12C] FIG. 1 is a block diagram illustrating a video decoding process using DIMD, according to some implementations of the present disclosure. [Figure 13] FIG. 10 illustrates fractional bits used in the proposed integerization scheme, according to some implementations of the present disclosure. [Figure 14] FIG. 1 is a block diagram illustrating a computing environment coupled with a user interface, according to some implementations of the present disclosure. [Figure 15] FIG. 2 is a block diagram illustrating video decoding, according to some implementations of the present disclosure. [Figure 16] FIG. 2 is a block diagram illustrating video decoding, according to some implementations of the present disclosure. [Figure 17] FIG. 2 is a block diagram illustrating video decoding, according to some implementations of the present disclosure. [Figure 18] FIG. 10 is a diagram illustrating an example of four reference lines in the neighborhood of a prediction block, according to some implementations of the present disclosure. [Figure 19] FIG. 10 illustrates an example of an expanded MRL candidate list according to some implementations of the present disclosure. [Figure 20] FIG. 2 is a block diagram illustrating video decoding, according to some implementations of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0009] Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. The following description refers to the accompanying drawings, in which like numbers in different drawings represent the same or similar elements unless otherwise stated. The implementations described in the following description of the exemplary embodiments do not represent all implementations consistent with the present disclosure. Rather, the implementations are merely examples of apparatus and methods consistent with aspects related to the present disclosure as set forth in the appended claims.

[0010] The terminology used in this disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. As used in this disclosure and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly dictates otherwise. It is also to be understood that the term "and / or," as used herein, is intended to mean and include any and all possible combinations of one or more of the associated listed items.

[0011] Terms such as "first," "second," and "third" may be used herein to describe various pieces of information, but it should be understood that these terms are not intended to limit the information. These terms are used only to distinguish one category of information from another. For example, first information may be referred to as second information, and similarly, second information may be referred to as first information, without departing from the scope of this disclosure. The term "if," as used herein, may be understood to mean "when," "in the event of," or "at the discretion of," depending on the context.

[0012] Various video coding techniques may be used to compress video data. Video coding is performed according to one or more video coding standards. For example, today, well-known video coding standards include Versatile Video Coding (VVC), High Efficiency Video Coding (HEVC, also known as H.265 or MPEG-H Part 2), and Advanced Video Coding (AVC, also known as H.264 or MPEG-4 Part 10), which are jointly developed by ISO / IEC MPEG and ITU-T VECG. AOMedia Video 1 (AV1) was developed by the Alliance for Open Media (AOM) as a successor to its predecessor, VP9. Audio-Video Coding (AVS), referring to digital audio and digital video compression standards, is another video compression standard series developed by the China Audio and Video Coding Standards Workgroup. Most of the existing video coding standards are built on the well-known hybrid video coding framework, i.e., using block-based prediction methods (e.g., inter-prediction, intra-prediction) to reduce redundancy present in a video image or sequence, and using transform coding to compact the energy of the prediction error. An important goal of video coding techniques is to compress video data into a format that uses a lower bitrate while avoiding or minimizing degradation of video quality.

[0013] The first generation of AVS standards includes the Chinese national standards "Information Technology, Advanced Audio-Video Coding, Part 2: Video" (known as AVS1) and "Information Technology, Advanced Audio-Video Coding, Part 16: Radio, Television, and Video" (known as AVS+). Compared to the MPEG-2 standard, this achieves approximately 50% bitrate savings at the same perceptual quality. The video portion of the AVS1 standard was promulgated as a Chinese national standard in February 2006. The second generation of AVS standards includes a series of Chinese national standards, "Information Technology, Efficient Multimedia Coding" (known as AVS2), primarily aimed at transmitting additional HD TV programs. AVS2's coding efficiency is twice that of AVS+. AVS2 was published as a Chinese national standard in May 2016. Meanwhile, the video portion of the AVS2 standard has been proposed by the Institute of Electrical and Electronics Engineers (IEEE) as one of the international standards for applications. The AVS3 standard is one of a new generation of video coding standards for UHD video applications, aiming to exceed the coding efficiency of the latest international standard, HEVC. In March 2019, at the 68th AVS Conference, the AVS3-P2 baseline was finalized, achieving approximately 30% bitrate savings compared to the HEVC standard. Currently, there is one reference software, called the High Performance Model (HPM), maintained by the AVS group to demonstrate a reference implementation of the AVS3 standard.

[0014] 1 is a block diagram illustrating an example system 10 for encoding and decoding video blocks in parallel, according to some implementations of the present disclosure. As shown in FIG. 1, system 10 includes a source device 12 that generates and encodes video data that is subsequently decoded by a destination device 14. Source device 12 and destination device 14 may include any of a wide variety of electronic devices, including desktop or laptop computers, tablet computers, smartphones, set-top boxes, digital televisions, cameras, display devices, digital media players, video gaming consoles, video streaming devices, etc. In some implementations, source device 12 and destination device 14 are equipped with wireless communication capabilities.

[0015] In some implementations, destination device 14 may receive encoded video data to be decoded via link 16. Link 16 may comprise any type of communication medium or device capable of moving encoded video data from source device 12 to destination device 14. In one example, link 16 may comprise a communication medium that enables source device 12 to transmit encoded video data directly to destination device 14 in real time. The encoded video data may be modulated according to a communication standard, such as a wireless communication protocol, and transmitted to destination device 14. The communication medium may include any wireless or wired communication medium, such as the radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium may form part of a packet-based network, such as a local area network, a wide area network, or a global network such as the Internet. The communication medium may include routers, switches, base stations, or any other equipment that may be useful in facilitating communication from source device 12 to destination device 14.

[0016] In some other implementations, the encoded video data may be transmitted from output interface 22 to storage device 32. The encoded video data in storage device 32 may then be accessed by destination device 14 via input interface 28. Storage device 32 may include any of a variety of distributed or locally accessed data storage media, such as a hard drive, a Blu-ray disc, a digital versatile disc (DVD), a compact disc read-only memory (CD-ROM), flash memory, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded video data. In a further example, storage device 32 may correspond to a file server or another intermediate storage device capable of holding the encoded video data generated by source device 12. Destination device 14 may access the stored video data from storage device 32 via streaming or download. The file server may be any type of computer capable of storing encoded video data and transmitting the encoded video data to destination device 14. Exemplary file servers include web servers (e.g., for websites), File Transfer Protocol (FTP) servers, Network Attached Storage (NAS) devices, or local disk drives. Destination device 14 may access the encoded video data through any standard data connection, including a wireless channel (e.g., a Wireless Fidelity (Wi-Fi) connection), a wired connection (e.g., a Digital Subscriber Line (DSL), a cable modem, etc.), or a combination of both suitable for accessing encoded video data stored on a file server. Transmission of the encoded video data from storage device 32 may be a streaming transmission, a download transmission, or a combination of both.

[0017] As shown in FIG. 1 , source device 12 includes video source 18, video encoder 20, and output interface 22. Video source 18 may include sources such as a video capture device, e.g., a video camera, a video archive containing previously captured video, a video feed interface for receiving video from a video content provider, and / or a computer graphics system for generating computer graphics data as source video, or a combination of such sources. As an example, if video source 18 is a video camera in a security surveillance system, source device 12 and destination device 14 may form a camera phone or video phone. However, implementations described herein may be applicable to video encoding generally and may be applicable to wireless and / or wired applications.

[0018] The captured, pre-captured, or computer-generated video may be encoded by video encoder 20. The encoded video data may be transmitted directly to destination device 14 via output interface 22 of source device 12. The encoded video data may also (or alternatively) be stored in storage device 32 for later access by destination device 14 or other devices for decoding and / or playback. Output interface 22 may further include a modem and / or a transmitter.

[0019] Destination device 14 includes an input interface 28, a video decoder 30, and a display device 34. Input interface 28 may include a receiver and / or modem and receive encoded video data over link 16. The encoded video data communicated over link 16 or provided on storage device 32 may include various syntax elements generated by video encoder 20 for use by video decoder 30 in decoding the video data. Such syntax elements may be included within the encoded video data transmitted over a communication medium, stored on a storage medium, or stored on a file server.

[0020] In some implementations, destination device 14 may include a display device 34, which may be an integrated display device or an external display device configured to communicate with destination device 14. Display device 34 displays the decoded video data to a user and may comprise any of a variety of display devices, such as a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, or another type of display device.

[0021] Video encoder 20 and video decoder 30 may operate in accordance with proprietary or industry standards, such as VVC, HEVC, MPEG-4, Part 10, AVC, or extensions of such standards. It should be understood that the present application is not limited to a particular video encoding / decoding standard and may be applicable to other video encoding / decoding standards. It is generally contemplated that video encoder 20 of source device 12 may be configured to encode video data in accordance with any of these current or future standards. Similarly, it is generally contemplated that video decoder 30 of destination device 14 may be configured to decode video data in accordance with any of these current or future standards.

[0022] Video encoder 20 and video decoder 30 may each be implemented as any of a variety of suitable encoder and / or decoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or any combination thereof. If implemented partially in software, the electronic device may store instructions for the software on a suitable non-transitory computer-readable medium and execute the instructions in hardware using one or more processors to perform the video encoding / decoding operations disclosed in this disclosure. Each of video encoder 20 and video decoder 30 may be included in one or more encoders or decoders, any of which may be integrated as part of a combined encoder / decoder (CODEC) within the respective device.

[0023] 2 is a block diagram illustrating an example video encoder 20 according to some implementations described herein. Video encoder 20 may perform intra-predictive and inter-predictive coding of video blocks within video frames. Intra-predictive coding relies on spatial prediction to reduce or remove spatial redundancy in video data within a given video frame or picture. Inter-predictive coding relies on temporal prediction to reduce or remove temporal redundancy in video data within adjacent video frames or pictures of a video sequence. Note that the term "frame" is sometimes used synonymously with the terms "image" or "picture" in the field of video coding.

[0024] 2, video encoder 20 includes video data memory 40, prediction processing unit 41, decoded picture buffer (DPB) 64, adder 50, transform processing unit 52, quantization unit 54, and entropy coding unit 56. Prediction processing unit 41 further includes motion estimation unit 42, motion compensation unit 44, partition unit 45, intra-prediction processing unit 46, and intra-block copy (BC) unit 48. In some implementations, video encoder 20 also includes inverse quantization unit 58 for video block reconstruction, inverse transform processing unit 60, and adder 62. An in-loop filter 63, such as a deblocking filter, may be disposed between adder 62 and DPB 64 to filter block boundaries and remove blocky artifacts from the reconstructed video. In addition to the deblocking filter, another in-loop filter, such as a Sample Adaptive Offset (SAO) filter and / or an Adaptive in-loop filter (ALF), may also be used to filter the output of summer 62. In some examples, the in-loop filter may be omitted, and the decoded video block may be provided directly to DPB 64 by summer 62. Video encoder 20 may take the form of a fixed or programmable hardware unit, or may be divided into one or more of the illustrated fixed or programmable hardware units.

[0025] Video data memory 40 may store video data to be encoded by components of video encoder 20. The video data in video data memory 40 may be obtained, for example, from video source 18 shown in FIG. 1. DPB 64 is a buffer that stores reference video data (e.g., reference frames or reference pictures) for use in encoding video data by video encoder 20 (e.g., in an intra-predictive coding mode or an inter-predictive coding mode). Video data memory 40 and DPB 64 may be formed by any of a variety of memory devices. In various examples, video data memory 40 may be on-chip with other components of video encoder 20 or may be off-chip relative to those components.

[0026] As shown in FIG. 2, after receiving video data, partitioning unit 45 in prediction processing unit 41 partitions the video data into video blocks. This partitioning may also include partitioning the video frame into slices, tiles (e.g., sets of video blocks), or other larger coding units (CUs) according to a predefined partitioning structure, such as a quad-tree (QT) structure associated with the video data. A video frame may be or be considered as a two-dimensional array or matrix of samples having sample values. The samples in the array may also be referred to as pixels or pels. The number of samples in the horizontal and vertical directions (or axes) of the array or picture defines the size and / or resolution of the video frame. The video frame may be divided into multiple video blocks, for example, by using QT partitioning. A video block may also be or be considered as a two-dimensional array or matrix of samples having sample values, although with smaller dimensions than a video frame. The number of samples in the horizontal and vertical directions (or axes) of a video block defines the size of the video block. A video block may be further partitioned into one or more block partitions or sub-blocks (which may again form blocks), e.g., by repeatedly using QT partitioning, binary-tree (BT) partitioning, or triple-tree (TT) partitioning, or any combination thereof. Note that the term “block” or “video block” as used herein may be a portion of a frame or picture, particularly a rectangular (square or non-square) portion. For example, with reference to HEVC and VVC, a block or video block may be or correspond to a coding tree unit (CTU), CU, prediction unit (PU), or transform unit (TU), and / or may be or correspond to a corresponding block, e.g., a coding tree block (CTB), coding block (CB), prediction block (PB), or transform block (TB), and / or sub-block.

[0027] Prediction processing unit 41 may select one of multiple possible predictive coding modes, such as one of multiple intra-predictive coding modes or one of multiple inter-predictive coding modes, for the current video block based on the error result (e.g., coding rate and distortion level). Prediction processing unit 41 may provide the resulting intra-predictively coded block or inter-predictively coded block to summer 50 to generate a residual block and to summer 62 to reconstruct the coded block for later use as part of a reference frame. Prediction processing unit 41 also provides syntax elements, such as motion vectors, intra-mode indicators, partition information, and other such syntax information, to entropy coding unit 56.

[0028] To select an appropriate intra-prediction coding mode for a current video block, intra-prediction processing unit 46 within prediction processing unit 41 may perform intra-prediction coding of the current video block relative to one or more neighboring blocks in the same frame as the current block to be coded to provide spatial prediction. Motion estimation unit 42 and motion compensation unit 44 within prediction processing unit 41 may perform inter-prediction coding of the current video block relative to one or more predictive blocks in one or more reference frames to provide temporal prediction. Video encoder 20 may perform multiple coding passes to select an appropriate coding mode for each block of video data, for example.

[0029] In some implementations, motion estimation unit 42 determines the inter-prediction mode for a current video frame by generating motion vectors that indicate the displacement of video blocks in the current video frame relative to predictive blocks in a reference video frame according to a predetermined pattern in the sequence of video frames. Motion estimation performed by motion estimation unit 42 is the process of generating motion vectors that estimate the motion of video blocks. The motion vectors may indicate, for example, the displacement of video blocks in a current video frame or picture relative to predictive blocks in a reference frame relative to the current block being coded in the current frame. The predetermined pattern may designate video frames in the sequence as P-frames or B-frames. Intra BC unit 48 may determine vectors, e.g., block vectors, for intra BC coding in a manner similar to the determination of motion vectors by motion estimation unit 42 for inter prediction, or may utilize motion estimation unit 42 to determine block vectors.

[0030] A prediction block for a video block may be or correspond to a block or reference block of a reference frame that is deemed to closely match the video block to be encoded in terms of pixel differences, which may be determined by sum of absolute differences (SAD), sum of square differences (SSD), or other difference metrics. In some implementations, video encoder 20 may calculate values ​​for sub-integer pixel positions of the reference frame stored in DPB 64. For example, video encoder 20 may interpolate values ​​for quarter-pixel positions, eighth-pixel positions, or other fractional pixel positions of the reference frame. Thus, motion estimation unit 42 may perform motion searches for whole pixel positions and fractional pixel positions and output motion vectors with fractional pixel precision.

[0031] Motion estimation unit 42 calculates a motion vector for a video block in an inter-predictively coded frame by comparing the position of the video block to the position of a predictive block of a reference frame selected from a first reference frame list (List 0) or a second reference frame list (List 1), each of which identifies one or more reference frames stored in DPB 64. Motion estimation unit 42 sends the calculated motion vector to motion compensation unit 44 and then to entropy coding unit 56.

[0032] The motion compensation performed by motion compensation unit 44 may include fetching or generating a predictive block based on the motion vector determined by motion estimation unit 42. Upon receiving the motion vector for the current video block, motion compensation unit 44 may locate the predictive block pointed to by the motion vector in one of the reference frame lists, obtain the predictive block from DPB 64, and forward the predictive block to summer 50. Summer 50 then forms a residual video block of pixel difference values ​​by subtracting pixel values ​​of the predictive block provided by motion compensation unit 44 from pixel values ​​of the current video block being coded. The pixel difference values ​​forming the residual video block may include a luma difference component, a chroma difference component, or both. Motion compensation unit 44 may also generate syntax elements associated with the video blocks of the video frame for use by video decoder 30 in decoding the video blocks of the video frame. The syntax elements may include, for example, a syntax element defining the motion vector used to identify the predictive block, any flags indicating a prediction mode, or any other syntax information described herein. It should be noted that motion estimation unit 42 and motion compensation unit 44 may be highly integrated, but are shown separately for conceptual purposes.

[0033] In some implementations, the intra BC unit 48 may generate a vector and fetch a predictive block in a manner similar to that described above in connection with the motion estimation unit 42 and the motion compensation unit 44, except that the predictive block is within the same frame as the current block being coded, and the vector is referred to as a block vector rather than a motion vector. Specifically, the intra BC unit 48 may determine an intra prediction mode to use to code the current block. In some examples, the intra BC unit 48 may code the current block using various intra prediction modes, e.g., during separate coding passes, and test their performance through rate-distortion analysis. The intra BC unit 48 may then select an appropriate intra prediction mode to use from the various tested intra prediction modes and generate an intra mode indicator accordingly. For example, the intra BC unit 48 may calculate rate-distortion values ​​using the rate-distortion analysis for the various tested intra prediction modes and select the intra prediction mode with the best rate-distortion characteristics from the tested modes as the appropriate intra prediction mode to use. Rate-distortion analysis generally determines the amount of distortion (or error) between a coded block and the original uncoded block that was coded to create the coded block, as well as the bit rate (i.e., number of bits) used to create the coded block. Intra BC unit 48 may calculate ratios from the distortions and rates of the various coded blocks to determine which intra-prediction mode exhibits the best rate-distortion value for that block.

[0034] In other examples, intra BC unit 48 may use, in whole or in part, motion estimation unit 42 and motion compensation unit 44 to perform such functions for intra BC prediction according to the implementations described herein. In either case, for intra block copying, the predictive block may be a block that is deemed to closely match the block to be coded in terms of pixel differences, which may be determined by SAD, SSD, or other difference metrics, and identifying the predictive block may include calculating values ​​for sub-integer pixel positions.

[0035] Regardless of whether the predictive block is a block from the same frame via intra prediction or a block from a different frame via inter prediction, video encoder 20 may form a residual video block by subtracting pixel values ​​of the predictive block from pixel values ​​of the current video block being coded to form pixel difference values. The pixel difference values ​​forming the residual video block may include both luma and chroma component differences.

[0036] Intra-prediction processing unit 46 may intra-predict the current video block as an alternative to inter-prediction performed by motion estimation unit 42 and motion compensation unit 44 or intra-block copy prediction performed by intra BC unit 48, as described above. Specifically, intra-prediction processing unit 46 may determine an intra-prediction mode to use to encode the current block. To do so, intra-prediction processing unit 46 may encode the current block using various intra-prediction modes, e.g., during separate encoding passes, and intra-prediction processing unit 46 (or, in some examples, a mode selection unit) may select an appropriate intra-prediction mode to use from the tested intra-prediction modes. Intra-prediction processing unit 46 may provide information indicating the selected intra-prediction mode for the block to entropy coding unit 56. Entropy coding unit 56 may encode the information indicating the selected intra-prediction mode into the bitstream.

[0037] After prediction processing unit 41 determines a predictive block for the current video block by inter-prediction or intra-prediction, adder 50 forms a residual video block by subtracting the predictive block from the current video block. The residual video data in the residual block, which may be included in one or more TUs, is provided to transform processing unit 52. Transform processing unit 52 converts the residual video data into residual transform coefficients using a transform, such as a discrete cosine transform (DCT) or a conceptually similar transform.

[0038] Transform processing unit 52 may send the resulting transform coefficients to quantization unit 54, which quantizes the transform coefficients to further reduce the bit rate. The quantization process may also reduce the bit depth associated with some or all of the coefficients. The degree of quantization may be modified by adjusting a quantization parameter. In some examples, quantization unit 54 may then perform a scan of a matrix containing the quantized transform coefficients. Alternatively, entropy coding unit 56 may perform the scan.

[0039] Following quantization, entropy coding unit 56 entropy codes the quantized transform coefficients into a video bitstream using, for example, context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), syntax-based context-adaptive binary arithmetic coding (SBAC), probability interval partitioning entropy (PIPE) coding, or another entropy coding methodology or technique. The coded bitstream may then be transmitted to video decoder 30 as shown in FIG. 1 or archived to storage device 32 as shown in FIG. 1 for later transmission to or retrieval by video decoder 30. Entropy coding unit 56 may also entropy code motion vectors and other syntax elements of the current video frame being coded.

[0040] Inverse quantization unit 58 and inverse transform processing unit 60 apply inverse quantization and inverse transform, respectively, to reconstruct the residual video block in the pixel domain to generate reference blocks for prediction of other video blocks. As described above, motion compensation unit 44 may generate motion-compensated prediction blocks from one or more reference blocks of frames stored in DPB 64. Motion compensation unit 44 may also apply one or more interpolation filters to the prediction block to calculate sub-integer pixel values ​​for use in motion estimation.

[0041] Summer 62 adds the reconstructed residual block to the motion compensated prediction block produced by motion compensation unit 44 to create a reference block for storage in DPB 64. The reference block may then be used by intra BC unit 48, motion estimation unit 42, and motion compensation unit 44 as a prediction block for inter predicting another video block in a subsequent video frame.

[0042] 3 is a block diagram illustrating an example video decoder 30 according to some implementations of the present application. The video decoder 30 includes a video data memory 79, an entropy decoding unit 80, a prediction processing unit 81, an inverse quantization unit 86, an inverse transform processing unit 88, an adder 90, and a DPB 92. The prediction processing unit 81 further includes a motion compensation unit 82, an intra prediction unit 84, and an intra BC unit 85. The video decoder 30 may perform a decoding process that is generally the reverse of the encoding process described above for the video encoder 20 in connection with FIG. 2. For example, the motion compensation unit 82 may generate prediction data based on a motion vector received from the entropy decoding unit 80, while the intra prediction unit 84 may generate prediction data based on an intra prediction mode indicator received from the entropy decoding unit 80.

[0043] In some examples, units of video decoder 30 may be tasked with performing implementations of the present application. Also, in some examples, implementations of the present disclosure may be divided among one or more of the units of video decoder 30. For example, intra BC unit 85 may perform implementations of the present application alone or in combination with other units of video decoder 30, such as motion compensation unit 82, intra prediction unit 84, and entropy decoding unit 80. In some examples, video decoder 30 may not include intra BC unit 85, and the functions of intra BC unit 85 may be performed by other components of prediction processing unit 81, such as motion compensation unit 82.

[0044] Video data memory 79 may store video data, such as an encoded video bitstream, to be decoded by other components of video decoder 30. The video data stored in video data memory 79 may be obtained, for example, from storage device 32, from a local video source such as a camera, via wired or wireless network communication of video data, or by accessing a physical data storage medium (e.g., a flash drive or hard disk). Video data memory 79 may include a coded picture buffer (CPB) that stores coded video data from the coded video bitstream. A data buffer buffer (DPB) 92 of video decoder 30 stores reference video data for use in decoding video data by video decoder 30 (e.g., in an intra-predictive coding mode or an inter-predictive coding mode). Video data memory 79 and DPB 92 may be formed by any of a variety of memory devices, such as dynamic random access memory (DRAM), including synchronous dynamic random access memory (SDRAM), magnetoresistive random access memory (MRAM), resistive random access memory (RRAM), or other types of memory devices. 3, video data memory 79 and DPB 92 are depicted as two separate components of video decoder 30. However, it will be apparent to those skilled in the art that video data memory 79 and DPB 92 may be provided by the same memory device or separate memory devices. In some examples, video data memory 79 may be on-chip with other components of video decoder 30 or may be off-chip relative to those components.

[0045] During the decoding process, video decoder 30 receives an encoded video bitstream representing video blocks and associated syntax elements of encoded video frames. Video decoder 30 may receive the syntax elements at the video frame level and / or the video block level. Entropy decoding unit 80 of video decoder 30 entropy decodes the bitstream to generate quantized coefficients, motion vectors or intra-prediction mode indicators, and other syntax elements. Entropy decoding unit 80 then forwards the motion vectors or intra-prediction mode indicators and other syntax elements to prediction processing unit 81.

[0046] When a video frame is coded as an intra-predictively coded (I) frame or for intra-coded predictive blocks within other types of frames, intra prediction unit 84 of prediction processing unit 81 may generate predictive data for video blocks of the current video frame based on a signaled intra-prediction mode and reference data from previously decoded blocks of the current frame.

[0047] When a video frame is coded as an inter-predictive (i.e., B or P) frame, motion compensation unit 82 of prediction processing unit 81 creates one or more prediction blocks for video blocks of the current video frame based on the motion vectors and other syntax elements received from entropy decoding unit 80. Each of the prediction blocks may be created from a reference frame in one of the reference frame lists. Video decoder 30 may construct reference frame lists List 0 and List 1 using a default construction technique based on the reference frames stored in DPB 92.

[0048] In some examples, when a video block is encoded according to the intra BC mode described herein, intra BC unit 85 of prediction processing unit 81 creates a predictive block of the current video block based on the block vectors and other syntax elements received from entropy decoding unit 80. The predictive block may be within the same reconstructed region of the picture as the current video block as defined by video encoder 20.

[0049] Motion compensation unit 82 and / or intra BC unit 85 determine prediction information for video blocks of the current video frame by analyzing motion vectors and other syntax elements, and then use the prediction information to create a predictive block for the current video block being decoded. For example, motion compensation unit 82 uses some of the received syntax elements to determine the prediction mode (e.g., intra-prediction or inter-prediction) used to encode the video blocks of the video frame, the inter-prediction frame type (e.g., B or P), construction information regarding one or more of the frame's reference frame list, the motion vectors of each inter-predictively coded video block of the frame, the inter-prediction status of each inter-predictively coded video block of the frame, and other information for decoding video blocks in the current video frame.

[0050] Similarly, intra BC unit 85 may use some of the received syntax elements, such as flags, to determine that the current video block was predicted using intra BC mode, construction information regarding which video blocks of the frame are within the reconstruction domain and should be stored in DPB 92, block vectors for each intra BC predicted video block of the frame, the intra BC prediction status for each intra BC predicted video block of the frame, and other information for decoding video blocks in the current video frame.

[0051] Motion compensation unit 82 may also perform interpolation using an interpolation filter used by video encoder 20 during encoding of the video block to calculate sub-integer pixel interpolated values ​​of the reference block. In this case, motion compensation unit 82 may determine the interpolation filter used by video encoder 20 from the received syntax element and use the interpolation filter to create the predictive block.

[0052] Inverse quantization unit 86 uses the same quantization parameter calculated by video encoder 20 for each video block in a video frame to inverse quantize the quantized transform coefficients provided in the bitstream and entropy decoded by entropy decoding unit 80 to determine the degree of quantization. Inverse transform processing unit 88 applies an inverse transform, e.g., an inverse DCT, an inverse integer transform, or a conceptually similar inverse transform process, to the transform coefficients in order to reconstruct residual blocks in the pixel domain.

[0053] After motion compensation unit 82 or intra BC unit 85 generates a predictive block for the current video block based on the vectors and other syntax elements, adder 90 reconstructs a decoded video block for the current video block by adding the residual block from inverse transform processing unit 88 and the corresponding predictive block generated by motion compensation unit 82 and intra BC unit 85. To further process the decoded video block, an in-loop filter 91, such as a deblocking filter, an SAO filter, and / or an ALF filter, may be disposed between adder 90 and the DPB. In some examples, in-loop filter 91 may be omitted, and the decoded video block may be provided directly to DPB 92 by adder 90. The decoded video block in a given frame is then stored in DPB 92, which stores reference frames used for subsequent motion compensation of the next video block. DPB 92 or a memory device separate from DPB 92 may store the decoded video for later presentation on a display device, such as display device 34 of FIG. 1.

[0054] In a typical video coding process, a video sequence typically includes an ordered set of frames or pictures. Each frame may include three sample arrays, denoted SL, SCb, and SCr. SL is a two-dimensional array of luma samples. SCb is a two-dimensional array of Cb chroma samples. SCr is a two-dimensional array of Cr chroma samples. In other examples, a frame may be monochromatic and therefore include only one two-dimensional array of luma samples.

[0055] As shown in FIG. 4A, video encoder 20 (or, more specifically, partitioning unit 45) generates a coded representation of a frame by first partitioning the frame into a set of CTUs. A video frame may include an integer number of CTUs, sequentially ordered from left to right and top to bottom in raster scan order. Each CTU is the largest logical coding unit, and the width and height of the CTU are signaled by video encoder 20 in the sequence parameter set so that all CTUs in a video sequence have the same size, which may be one of 128x128, 64x64, 32x32, and 16x16. However, it should be noted that the present application is not necessarily limited to a particular size. As shown in FIG. 4B, each CTU may include one CTB of luma samples, two corresponding coding tree blocks of chroma samples, and syntax elements used to encode the samples in the coding tree blocks. The syntax elements describe how a video sequence may be reconstructed in video decoder 30, including characteristics of different types of units of coded blocks of pixels, as well as inter- or intra-prediction, intra-prediction mode, motion vectors, and other parameters. For a monochrome picture or a picture with three separate color planes, a CTU may include a single coding tree block and syntax elements used to encode samples of the coding tree block. A coding tree block may be an N×N block of samples.

[0056] To achieve better performance, video encoder 20 may recursively perform tree partitioning, such as binary tree partitioning, ternary tree partitioning, quad tree partitioning, or a combination thereof, on the coding tree blocks of the CTU to partition the CTU into smaller CUs. As depicted in FIG. 4C , 64×64 CTU 400 is first partitioned into four smaller CUs, each with a block size of 32×32. Of the four smaller CUs, CU 410 and CU 420 are each partitioned into four 16×16 CUs by block size. Two 16×16 CUs, CU 430 and CU 440, are each further partitioned into four 8×8 CUs by block size. FIG. 4D depicts a quad tree data structure showing the final result of the partitioning process for CTU 400 depicted in FIG. 4C , where each leaf node of the quad tree corresponds to one CU of a respective size ranging from 32×32 to 8×8. Similar to the CTU depicted in FIG. 4B, each CU may include a CB for luma samples, two corresponding coding blocks for chroma samples of the same size frame, and syntax elements used to encode the samples of the coding block. In a monochrome picture or a picture with three separate color planes, a CU may include a single coding block and syntax structures used to encode the samples of the coding block. Note that the quadtree partitioning depicted in FIGS. 4C and 4D is for illustrative purposes only, and a CTU may be divided into CUs based on quadtree / ternary tree / binary tree partitioning to accommodate various local characteristics. In a multi-type tree structure, a CTU is partitioned by a quadtree structure, and each quadtree leaf CU may be further partitioned by binary tree and ternary tree structures. As shown in FIG. 4E, there are five possible partition types of a coding block with width W and height H: quad-partition, horizontal 2-partition, vertical 2-partition, horizontal 3-partition, and vertical 3-partition.

[0057] In some implementations, video encoder 20 may further partition the coding blocks of a CU into one or more M×N PBs. A PB is a rectangular (square or non-square) block of samples to which the same prediction, inter prediction or intra prediction, is applied. A PU of a CU may include a PB of luma samples, two corresponding PBs of chroma samples, and syntax elements used to predict the PB. In a monochrome picture or a picture with three separate color planes, a PU may include a single PB and syntax structures used to predict the PB. Video encoder 20 may generate predictive luma blocks, predictive Cb blocks, and predictive Cr blocks for the luma, Cb, and Cr PBs of each PU of a CU.

[0058] Video encoder 20 may generate the predictive blocks of a PU using intra prediction or inter prediction. If video encoder 20 generates the predictive blocks of a PU using intra prediction, video encoder 20 may generate the predictive blocks of the PU based on decoded samples of a frame associated with the PU. If video encoder 20 generates the predictive blocks of a PU using inter prediction, video encoder 20 may generate the predictive blocks of the PU based on decoded samples of one or more frames other than the frame associated with the PU.

[0059] After video encoder 20 generates the predictive luma block, the predictive Cb block, and the predictive Cr block for one or more PUs of a CU, video encoder 20 may generate the luma residual block of the CU by subtracting the predictive luma block of the CU from its original luma coding block, such that each sample in the luma residual block of the CU indicates a difference between a luma sample in one of the predictive luma blocks of the CU and a corresponding sample in the original luma coding block of the CU. Similarly, video encoder 20 may generate the Cb residual block and the Cr residual block of the CU, such that each sample in the Cb residual block of the CU indicates a difference between a Cb sample in one of the predictive Cb blocks of the CU and a corresponding sample in the original Cb coding block of the CU, and such that each sample in the Cr residual block of the CU indicates a difference between a Cr sample in one of the predictive Cr blocks of the CU and a corresponding sample in the original Cr coding block of the CU.

[0060] Further, as shown in FIG. 4C , video encoder 20 may use quadtree partitioning to decompose the luma residual block, Cb residual block, and Cr residual block of a CU into one or more luma transform blocks, Cb transform blocks, and Cr transform blocks, respectively. A transform block is a rectangular (square or non-square) block of samples to which the same transform is applied. A TU of a CU may include a transform block of luma samples, two corresponding transform blocks of chroma samples, and syntax elements used to transform the transform block samples. Thus, each TU of a CU may be associated with a luma transform block, a Cb transform block, and a Cr transform block. In some examples, the luma transform block associated with a TU may be a sub-block of the luma residual block of the CU. The Cb transform block may be a sub-block of the Cb residual block of the CU. The Cr transform block may be a sub-block of the Cr residual block of the CU. In a monochrome picture or a picture with three separate color planes, a TU may include a single transform block and syntax structures used to transform the samples of the transform block.

[0061] Video encoder 20 may apply one or more transforms to a luma transform block of a TU to generate a luma coefficient block of the TU. The coefficient block may be a two-dimensional array of transform coefficients. The transform coefficients may be scalar quantities. Video encoder 20 may apply one or more transforms to a Cb transform block of the TU to generate a Cb coefficient block of the TU. Video encoder 20 may apply one or more transforms to a Cr transform block of the TU to generate a Cr coefficient block of the TU.

[0062] After generating a coefficient block (e.g., a luma coefficient block, a Cb coefficient block, or a Cr coefficient block), the video encoder 20 may quantize the coefficient block. Quantization generally refers to a process by which transform coefficients are quantized to potentially reduce the amount of data used to represent the transform coefficients, thereby achieving further compression. After the video encoder 20 quantizes the coefficient block, the video encoder 20 may entropy encode syntax elements indicating the quantized transform coefficients. For example, the video encoder 20 may perform CABAC on the syntax elements indicating the quantized transform coefficients. Finally, the video encoder 20 may output a bitstream including a sequence of bits forming a representation of the encoded frame and associated data, which may be stored on a storage device 32 or transmitted to a destination device 14.

[0063] After receiving the bitstream generated by video encoder 20, video decoder 30 may parse the bitstream to obtain syntax elements from the bitstream. Video decoder 30 may reconstruct frames of video data based at least in part on the syntax elements obtained from the bitstream. The process of reconstructing video data is generally the reverse of the encoding process performed by video encoder 20. For example, video decoder 30 may perform an inverse transform on coefficient blocks associated with TUs of the current CU to reconstruct residual blocks associated with the TUs of the current CU. Video decoder 30 also reconstructs coding blocks of the current CU by adding samples of predictive blocks of PUs of the current CU to corresponding samples of transform blocks of TUs of the current CU. After reconstructing coding blocks for each CU of the frame, video decoder 30 may reconstruct the frame.

[0064] As mentioned above, video coding achieves video compression using two main modes: intra-frame prediction (or intra-prediction) and inter-frame prediction (or inter-prediction). Note that IBC can be considered as intra-frame prediction or a third mode. Of the two modes, inter-frame prediction contributes more to coding efficiency than intra-frame prediction because it uses motion vectors to predict a current video block from a reference video block.

[0065] However, as video data capture technology continues to improve and video block sizes become finer to preserve video data details, the amount of data required to represent the motion vectors of the current frame also increases significantly. One way to overcome this challenge is to benefit from the fact that a group of adjacent CUs in both the spatial and temporal domains not only have similar video data for prediction purposes, but also have similar motion vectors between these adjacent CUs. Therefore, the motion information of spatially adjacent CUs and / or temporally co-located CUs can be used as an approximation of the motion information (e.g., motion vector) of the current CU by examining their spatial and temporal correlations, which is also called the "motion vector predictor (MVP)" of the current CU.

[0066] Instead of encoding the actual motion vector of the current CU determined by motion estimation unit 42 into the video bitstream as described above in connection with Figure 3, the motion vector predictor of the current CU is subtracted from the actual motion vector of the current CU to create a Motion Vector Difference (MVD) for the current CU. By doing so, the motion vector determined for each CU of a frame by motion estimation unit 42 does not need to be encoded into the video bitstream, and the amount of data used to represent motion information in the video bitstream may be significantly reduced.

[0067] Similar to the process of choosing a predictive block in a reference frame during inter-frame prediction of a codeblock, a set of rules needs to be adopted by both video encoder 20 and video decoder 30 to construct a motion vector candidate list (also called a "merge list") for the current CU using potential candidate motion vectors associated with CUs that are spatially adjacent to and / or temporally co-located with the current CU, and then select one element from the motion vector candidate list as the motion vector predictor for the current CU. By doing so, the motion vector candidate list itself does not need to be transmitted from video encoder 20 to video decoder 30; the index of the selected motion vector predictor in the motion vector candidate list is sufficient for video encoder 20 and video decoder 30 to use the same motion vector predictor in the motion vector candidate list to encode and decode the current CU.

[0068] Position-dependent intra-prediction combination In VVC, the results of intra prediction for DC mode, planar mode, and some angle modes are further modified by the position dependent intra prediction combination (PDPC) method. PDPC is an intra prediction method that invokes the combination of boundary reference samples and HEVC-style intra prediction using filtered boundary reference samples. PDPC applies to the following intra modes without signaling: planar, DC, horizontal intra angles less than or equal to 100°, and vertical intra angles less than or equal to 80°. PDPC is not applied if the current block is in Bdpcm mode or the MRL index is greater than 0.

[0069] Using a linear combination of intra prediction modes (DC, planar, angular) and reference samples, the following equation pred(x',y')=Clip(0,(1<<BitDepth)-1,(wL×R-1,y’+wT×Rx’,-1+(64-wL-wT)×pred(x’,y’)+32)> >6) The predicted sample pred(x',y') is predicted according to, where Rx,-1, R-1,y represent the reference samples located at the upper and left boundaries of the current sample (x,y), respectively.

[0070] When PDPC is applied to DC, planar, horizontal, and vertical intra modes, it does not require the additional boundary filters required in the case of the HEVC DC mode boundary filter or horizontal / vertical mode edge filter. The PDPC process for DC mode and planar mode is identical. For angular modes, if the current angular mode is HOR_IDX or VER_IDX, the left or top reference sample is not used, respectively. The PDPC weights and scale factors depend on the prediction mode and block size. PDPC is applied to blocks with both width and height equal to or greater than 4.

[0071] 5A to 5D show definitions of PDPC reference samples (Rx,-1 and R-1,y) applied to various prediction modes. FIG. 5A shows an example of a diagonal-top-right mode. FIG. 5B shows an example of a diagonal-bottom-left mode. FIG. 5C shows an example of an adjacent diagonal-top-right mode. FIG. 5D shows an example of an adjacent diagonal-bottom-left mode. The prediction sample pred(x',y') is located at (x',y') within the prediction block. As an example, in the diagonal mode, the coordinate x of the reference sample Rx,-1 is given by x=x'+y'+1, and the coordinate y of the reference sample R-1,y is similarly given by y=x'+y'+1. In other angular modes, the reference samples Rx,-1 and R-1,y may be located at fractional sample positions. In this case, the sample value of the nearest integer sample position is used.

[0072] As mentioned above, intra prediction samples are generated from a set of unfiltered or filtered neighboring reference samples, which can cause discontinuities along block boundaries between the current coding block and its neighboring blocks. To address such issues, HEVC applies boundary filtering by combining the first row / column of prediction samples for DC, horizontal (i.e., mode 18), and vertical (i.e., mode 50) prediction modes with the unfiltered reference samples using a two-tap filter (for DC mode) or a gradient-based smoothing filter (for horizontal and vertical prediction modes).

[0073] Gradient PDPC In VVC, in some scenarios, PDPC may not be applied because the secondary reference sample is not available. Gradient-based PDPC, extended from horizontal / vertical mode, is applied. The PDPC weights (wT / wL) and the nScale parameter, which determines the attenuation in the PDPC weight with respect to the distance from the left / top boundary, are set equal to the corresponding parameters in horizontal / vertical mode, respectively. If the secondary reference sample is at a fractional sample position, bilinear interpolation is applied.

[0074] Geometric partition mode (GPM) VVC supports geometric partitioning mode for inter prediction. Geometric partitioning mode is signaled by a CU-level flag as a special merge mode. In the current GPM design, a total of 64 partitions are supported by GPM modes for each possible CU size where both width and height are between 8 and 64, excluding 8x64 and 64x8.

[0075] When this mode is used, a CU is divided into two parts by a geometrically positioned line, as shown in Figure 6. The position of the division line is mathematically derived from the angle and offset parameters of a particular partition. Each part of a geometric partition within a CU is inter-predicted using its own motion, and only uni-prediction is allowed for each partition; that is, each part has one motion vector and one reference index. Similar to traditional bi-prediction, a uni-prediction motion constraint is applied to ensure that only two motion-compensated predictions are required for each CU. When the geometric partition mode is used for the current CU, a geometric partition index (angle and offset) and two merge indices (one for each partition) indicating the partition mode of the geometric partition are also signaled. The number of maximum GPM candidate sizes is explicitly signaled at the sequence level.

[0076] Blending along the edges of geometric divisions After each geometric partition is obtained using its unique motion, blending is applied to the two uni-predictive signals to derive samples around the edges of the geometric partition. The blending weights for each position of the CU are derived based on the distance from the individual sample position to the corresponding partition edge.

[0077] GPM signal transmission design According to the current GPM design, the use of GPM is indicated by signaling one flag at the CU level. The flag is signaled only if the current CU is coded using merge or skip mode. Specifically, if the flag is equal to 1, the flag indicates that the current CU is predicted by GPM. Otherwise (if the flag is equal to 0), the CU is coded using another merge mode, such as normal merge mode, merge mode with motion vector differential, or combined inter and intra prediction. If GPM is enabled for the current CU, one syntax element, merge_gpm_partition_idx, is further signaled to indicate the applied geometric partitioning mode (specifying the direction and offset of the line from the CU center that divides the CU into two partitions, as shown in Figure 6). Then, two syntax elements, merge_gpm_idx0 and merge_gpm_idx1, are signaled to indicate the indices of the unipredictive merge candidates used for the first and second GPM partitions. More specifically, these two syntax elements are used to determine the unidirectional MVs of two GPM partitions from the unipredictive merge list, as described in Section "Constructing a Unipredictive Merge List." According to the current GPM design, to make the two unidirectional MVs more different, the two indices cannot be the same. Based on such prior knowledge, first, the unipredictive merge index of the first GPM partition is signaled and used as a predictor to reduce the signaling overhead of the unipredictive merge index of the second GPM partition. In detail, if the second unipredictive merge index is smaller than the first unipredictive merge index, its original value is signaled directly. Otherwise (if the second unipredictive merge index is larger than the first unipredictive merge index), its value is subtracted by 1 before being signaled to the bitstream. On the decoder side, first, the first unipredictive merge index is DecryptedThen, for decoding the second unipredictive merge index, if the parsed value is less than the first unipredictive merge index, the second unipredictive merge index is set equal to the parsed value; otherwise (if the parsed value is greater than or equal to the first unipredictive merge index), the second unipredictive merge index is set equal to the parsed value plus 1. Table 1 shows the existing syntax elements used for GPM mode in the current VVC specification.

[0078] [Table 1]

[0079] On the other hand, in the current GPM design, a truncated unary code is used to binarize the two unipredictive merge indices, i.e., merge_gpm_idx0 and merge_gpm_idx1. Furthermore, since two unipredictive merge indices cannot be the same, different maximum values ​​are used to truncate the codewords of the two unipredictive merge indices, and the maximum values ​​are set equal to MaxGPMMergeCand-1 and MaxGPMMergeCand-2 for merge_gpm_idx0 and merge_gpm_idx1, respectively. MaxGPMMergeCand is the number of candidates in the unipredictive merge list.

[0080] When GPM / AWP mode is applied, two different binarization methods are applied to convert the syntax merge_gpm_partition_idx into a string of binary bits. Specifically, the syntax element is binarized by fixed length code and truncated binary code in the VVC and AVS3 standards, respectively. On the other hand, in the AWP mode of AVS3, Syntax elements For binarization, different maximum values ​​are used.

[0081] Spatial angular weighted prediction (SAWP) AVS uses the Spatial Angle Weighted Prediction (SAWP) mode, which extends the GPM mode to intra blocks. In SAWP mode, instead of weighting two inter-predicted blocks, two intra-predicted blocks are weighted. The two intra-predicted blocks are predicted using two different intra-prediction modes selected from the intra-prediction modes. The intra-prediction modes are selected from angular modes 5 to 30. The maximum size is 32x32. To derive the MPM for SAWP mode, the two most probable modes (MPMs) of the regular intra modes are used.

[0082] Multi-direction intra prediction (MDIP) designs follow the same design spirit as SAWP, but with some minor differences in certain design details.

[0083] Decoder-side Intra-mode Derivation (DIMD) DIMD is an intra-coding tool in which the luma intra-prediction mode (IPM) is not transmitted via the bitstream. Instead, the IPM is derived using previously coded / decoded pixels in the same manner at the encoder and decoder. The DIMD method performs texture gradient processing to derive two optimal modes. These two modes and the planar mode are then applied to the block, and their predictors are weighted-averaged. The DIMD selection result is signaled in the bitstream of the intra-coded block using a flag. At the decoder, if the DIMD flag is true, the intra-prediction mode is derived in the reconstruction process using the same previously coded neighboring pixels. If not true, the intra-prediction mode is parsed from the bitstream in the same way as a classical intra-coding mode.

[0084] To derive the intra prediction mode for a block, we must first select a set of neighboring pixels on which we will perform gradient analysis. For standardity purposes, these pixels should be within the pool of decoded / reconstructed pixels. As shown in Figure 7, we choose a template that surrounds the current block by T pixels to the left and T pixels above. We then perform gradient analysis on the template's pixels. This allows us to determine the dominant angular orientation of the template, which we assume is likely to be identical to one of the current blocks (a core premise of our method). Therefore, we convolve the following matrix with the template:

number

[0085] For each pixel in the template, we multiply each of these two matrices by a 3x3 window centered on the current pixel and consisting of its eight immediate neighbors, and then sum the results, thus obtaining two values ​​Gx (from multiplication with Mx) and Gy (from multiplication with My), which correspond to the horizontal and vertical gradients at the current pixel, respectively.

[0086] Figure 8 shows the convolution process . picture Element 910 is the current pixel. The pixels in the box 920 represented by the right-slashed dot and the pixel 910 is a pixel for which gradient analysis is possible. (Hereinafter, for simplicity, this will be referred to as the "first pixel") is. Within box 920, represented by the left-hand diagonal dot The pixel is one for which gradient analysis is not possible due to a lack of neighbors. . picture Basic 930 are the available (reconstructed) pixels outside the considered template, 1st Used for pixel gradient analysis (e.g., when a block is too close to a picture boundary). ) picture Basic 930 If this is not available, Picture of Basic 930 All that use 1st No pixel gradient analysis is performed. 1st For each pixel, the gradient strength (G) and orientation (O) are calculated using Gx and Gy,

number

[0087] The gradient orientation is then converted to an intra-angle prediction mode and used to index into the histogram (initialized to zero). The histogram value in that intra-angle mode is increased by G. 1st As a pixel is processed, the histogram contains the cumulative gradient magnitude values ​​for each intra-angle mode. The IPM corresponding to the two highest histogram bars is selected for the current block. If the maximum value in the histogram is 0 (meaning that gradient analysis could not be performed or the area that makes up the template is flat), DC mode is selected as the intra-prediction mode for the current block.

[0088] The two IPMs corresponding to the two highest HoG bars are combined with a planar mode. In one or more instances, predictive fusion is applied as a weighted average of the three predictors mentioned above. For this purpose, the planar weight is fixed at 21 / 64 (approximately 1 / 3). The remaining weight, 43 / 64 (approximately 2 / 3), is then shared between the two HoG IPMs in proportion to the amplitude of the HoG bars. Figure 9 visualizes this process.

[0089] The derived intra modes are included in the primary list of intra most probable modes (MPMs), and therefore the DIMD process is performed before the MPM list is built. The primary derived intra modes of a DIMD block are stored with the block and are used to build the MPM lists of neighboring blocks.

[0090] Template-based intra mode derivation (TIMD) For each intra mode in MPM, the sum of absolute transformed difference (SATD) between the predicted samples and the reconstructed samples of the template region shown in Figure 10 is calculated, and the intra mode with the first two modes with the smallest SATD cost is selected, and then they are fused with weights, and such weighted intra prediction is used to encode the current CU.

[0091] The costs of the two selected modes are compared to a threshold and a cost factor of 2 is applied in the test as follows: costMode2<2*costMode1

[0092] If this condition is true, fusion is applied, otherwise only mode1 is used.

[0093] The weights of modes are calculated from their SATD costs as follows: weight1=costMode2 / (costMode1+costMode2) weight2=1-weight1

[0094] Multiple reference line (MRL) intra prediction Multiple Reference Line (MRL) intra prediction uses more reference lines for intra prediction. Figure 18 illustrates an example of four reference lines, where samples from segments A and F are not fetched from reconstructed neighboring samples but are padded with the nearest samples from segments B and E, respectively. HEVC intra picture prediction uses the nearest reference line (i.e., reference line 0). In MRL, two additional lines (reference line 1 and reference line 3) are used.

[0095] The index of the selected reference line (mrl_idx) is signaled and used to generate the intra predictor. For a reference line index greater than 0, include only additional reference line modes in the MPM list and signal only the mpm index without the remaining modes. The reference line index is signaled before the intra prediction modes, and if a non-zero reference line index is signaled, planar modes are excluded from the intra prediction modes.

[0096] To prevent the use of extended reference samples outside the current CTU line, MRL is disabled for the first line of a block inside the CTU. Also, PDPC is disabled if additional lines are used. For MRL mode, the derivation of DC values ​​in DC intra prediction mode for non-zero reference line indices is consistent with the derivation for reference line index 0. MRL requires storing three nearby luma reference lines with the CTU to generate predictions. The Cross-Component Linear Model (CCLM) tool also requires three nearby luma reference lines for its downsampling filter. The definition of MRL, which uses the same three lines, is consistent with CCLM to reduce decoder storage requirements.

[0097] Although DIMD mode can improve intra prediction efficiency, there is still room for further improvement in its performance. Meanwhile, some parts of the existing DIMD mode also need to be simplified for efficient codec hardware implementation or improved for better coding efficiency. Furthermore, the trade-off between its implementation complexity and its coding efficiency benefits needs to be further improved.

[0098] After the final decision on VVC, the JVET group continued to explore compression efficiencies beyond VVC. JVET maintained a single reference software called the Extended Compression Model (ECM) by integrating several additional coding tools onto the VVC Test Model (VTM). In the current ECM, PDPC is applied depending on the intra mode. For DIMD mode, PDPC is applied depending on each intra mode. As shown in Figure 11D, two different positions of the PDPC scheme are used and applied to each intra mode in DIMD mode. For intra prediction using angular mode in DIMD mode, PDPC is applied before prediction fusion. For intra prediction using DC mode or planar mode in DIMD mode, PDPC is applied after prediction fusion. Such non-uniform designs may not be optimal from a standardization perspective.

[0099] Similarly, two different designs of fusion schemes are available, applicable to DIMD and TIMD, respectively. Each different fusion design is associated with different candidate and weight calculations. For blocks to which DIMD is applied, the two IPMs and plane modes corresponding to the two highest HoG bars are selected for fusion. The plane weight is fixed at 21 / 64 (approximately 1 / 3). The remaining weight, 43 / 64 (approximately 2 / 3), is then shared between the two HoG IPMs in proportion to the amplitude of the HoG bars. For blocks to which TIMD is applied, the intra-mode with the first two modes with the smallest SATD cost is selected, and the mode weights are calculated from those SATD costs. Such non-uniform designs may not be optimal from a standardization perspective. In addition to the above, there is room for further improvement in performance through various fusion schemes.

[0100] In the current ECM design, intra-modes derived from DIMD are included in the primary list of intra-most probable modes (MPMs), regardless of whether the derived intra-modes are already used in DIMD. There is room for further improvement in its performance.

[0101] In the existing design of DIMD and TIMD, multiple floating-point operations (including addition, multiplication, and division) are involved to calculate the parameters used to derive the optimal intra-prediction mode and generate the corresponding predicted sample of one current DIMD / TIMD coding block. Specifically, in the existing design of DIMD and TIMD in ECM, the following floating-point operations are applied:

[0102] 1) Gradient Orientation Derivation in DIMD: As explained previously, in DIMD mode, two optimal intra-prediction modes are selected based on an analysis of the histogram of gradients (HoG) of neighboring reconstructed samples (i.e., templates) above and to the left of the current block. During such analysis, the gradient orientation of each template sample needs to be calculated, which is then further converted into one of the existing angular intra-prediction directions. In ECM, to calculate such an orientation based on horizontal and vertical gradients, a pair of floating-point division and multiplication operations is applied to each template sample, i.e.,

number

[0103] 2) Blending of predicted samples in DIMD: In the existing DIMD design, the predicted samples generated using the two angular intra prediction modes with the largest and second largest gradient histogram amplitudes are blended with the predicted samples of the planar mode to form the final predicted samples of the current block. Furthermore, the weights of the two angular intra prediction modes are determined based on their gradient histogram amplitudes.

number

[0104] 3) Mixing of Predicted Samples in TIMD: In the existing TIMD design, if the SATDs of the two selected intra modes are close enough, the predicted samples generated from the two intra modes are mixed together to generate the final predicted sample for the intra mode of the current block. According to the current design, the weights applied to the two intra modes are calculated according to their respective SATD values,

number

[0105] All such above floating point operations are very expensive for practical codec implementations in both hardware and software.

[0106] In this disclosure, methods are provided to simplify and / or further improve existing designs of DIMD modes to address previously pointed out problems. In general, the main features of the techniques proposed in this disclosure are summarized as follows:

[0107] 1) Unify the PDPC used under angular mode and DC / planar mode in DIMD mode by applying PDPC to all intra predictions before prediction fusion. An example of such a method is shown in the block diagram of Figure 12A.

[0108] 2) Unify the PDPC used under angular mode and DC / planar mode in DIMD mode by applying PDPC to all intra predictions after prediction fusion. An example of such a method is shown in the block diagram of Figure 12B.

[0109] 3) Unify the PDPC used under angular mode and DC / planar mode in DIMD mode by disabling PDPC for all intra prediction in DIMD mode. An example of such a method is shown in the block diagram of Figure 12C.

[0110] 4) Unify the PDPC used under angular mode and DC / planar mode in DIMD mode by disabling PDPC for DC / planar intra prediction in DIMD mode.

[0111] 5) Unify the PDPC used under angular mode and DC / planar mode in DIMD mode by disabling PDPC for angular intra prediction in DIMD mode.

[0112] 6) The fusion method used under the DIMD mode is applied to the TIMD mode, thereby unifying the fusion methods used under the DIMD mode and the TIMD mode.

[0113] 7) The fusion method used under the DIMD mode and the TIMD mode is unified by applying the fusion method used under the TIMD mode to the DIMD mode.

[0114] 8) Unify the fusion methods used under DIMD mode and TIMD mode by signaling the selection result of the fusion method.

[0115] 9) Derive intra-modes from the DIMD into a list of intra-most probable modes (MPMs), taking into account whether the derived intra-modes are already used in the DIMD.

[0116] 10) Derive intra-modes from TIMD into a list of intra-most probable modes (MPMs).

[0117] It should be noted that the proposed method can also be applied to other intra-prediction coding modes such as TIMD / MDIP. Another set of examples applied to TIMD mode is shown in the block diagrams of Figures 11A to 11C. Figure 11A shows an example of applying all PDPC processes before the fusion process of TIMD. Figure 11B shows an example of applying all PDPC processes after the fusion process of TIMD. Figure 11C shows an example of disabling all PDPC processes in TIMD.

[0118] It should be noted that the proposed method can also be applied to other combined intra- and inter-prediction coding modes, such as combined inter and intra prediction (CIIP).

[0119] It should be noted that the disclosed methods can be applied singly or in combination.

[0120] Matching of PDPCs used for angular mode and DC / planar mode in DIMD According to one or more embodiments of the present disclosure, the same PDPC position is applied for both angular and DC / planar modes under DIMD mode. Various methods may be used to achieve this goal.

[0121] In one example of the present disclosure, as shown in Figure 12A, it is proposed to apply PDPC calculation before prediction fusion in DIMD mode. In other words, before prediction fusion in DIMD mode, each intra prediction mode is applied to PDPC based on its intra mode. The proposed method can also be applied to other intra prediction coding modes such as TIMD.

[0122] In another example of the present disclosure, as shown in FIG. 12B, it is proposed to apply a PDPC operation after prediction fusion in DIMD mode. In other words, a weighted combination of three predictors is applied to PDPC based on a specific mode, e.g., DC mode or planar mode. In one example, the specific mode is planar mode, and then PDPC with the planar mode is applied after prediction fusion in DIMD mode. In another example, the IPM corresponding to the highest histogram bar is selected as the specific mode, and then PDPC with the specific mode is applied after prediction fusion in DIMD mode. In yet another example, the IPM corresponding to the second highest histogram bar is selected as the specific mode, and then PDPC with the specific mode is applied after prediction fusion in DIMD mode.

[0123] In another example of the present disclosure, it is proposed to disable PDPC calculation in DIMD mode, in other words, PDPC calculation is not used in DIMD mode, as shown in FIG.

[0124] In another example of the present disclosure, it is proposed to disable PDPC operations for DC / planar intra prediction in DIMD mode. In other words, PDPC operations are not used for DC / planar intra prediction in DIMD mode. In one example as shown in FIG. 15, in step 1502, the decoder may determine predicted sample values ​​of one or more video blocks based on PDPC operations for one or more intra predictions of the one or more video blocks to unify position-dependent intra prediction combining (PDPC) operations in intra-prediction coding mode, where the PDPC operations modify the results of the one or more intra predictions based on a combination of boundary reference samples. In step 1504, the decoder may disable PDPC operations for direct current (DC) mode or planar mode in response to determining that DC mode or planar mode is applied in one or more intra predictions of one or more video blocks to unify PDPC operations in intra-prediction coding mode.

[0125] In yet another example of the present disclosure, it is proposed to disable PDPC operations for angular intra prediction in DIMD mode. In other words, PDPC operations are not used for angular intra prediction in DIMD mode. In one example as shown in FIG. 16, in step 1602, the decoder may determine predicted sample values ​​of one or more video blocks based on PDPC operations for one or more intra predictions of the one or more video blocks to unify position-dependent intra prediction combining (PDPC) operations in intra-prediction coding mode, where the PDPC operations modify the results of the one or more intra predictions based on a combination of boundary reference samples. In step 1604, the decoder may disable PDPC operations for the angular mode in response to determining that the angular mode is applied in one or more intra predictions of the one or more video blocks to unify PDPC operations in intra-prediction coding mode.

[0126] It should be noted that the proposed method can also be applied to other intra-prediction coding modes such as TIMD / MDIP.

[0127] Harmonization of fusion methods used in DIMD and TIMD modes According to one or more embodiments of the present disclosure, the same fusion scheme is applied to both DIMD and TIMD modes. Various methods may be used to achieve this goal. The fusion scheme is applied as a weighted average of predictors in DIMD and TIMD modes.

[0128] In one example of the present disclosure, we propose to apply the fusion method used under the DIMD mode to the TIMD mode. In other words, for the TIMD mode, the first two modes with the smallest SATD cost and the plane mode are selected as predictors for fusion, and a weighted average of the predictors is calculated. In addition, the weight of the plane mode is fixed at 2 1 / 64 (approximately 1 / 3). In this case, the remaining weight of 4 3 / 64 (approximately 2 / 3) is shared between the other two modes in proportion to the amplitude of the SATD cost.

[0129] In another example of this disclosure, we propose applying the fusion method used in TIMD mode to DIMD mode. In other words, in DIMD mode, the first two modes with the highest HoG bars are selected as predictors for fusion, and the mode weights are calculated from the HoG IPM in proportion to the amplitude of the HoG bars. If the maximum value in the histogram is 0 (meaning that gradient analysis could not be performed or the area constituting the template is flat), a default mode, such as DC mode or planar mode, is selected as the intra prediction mode for the current block.

[0130] In yet another example of the present disclosure, it is proposed to signal the selection result of the fusion scheme in TIMD and / or DIMD mode. In one example, for a given CU, a flag is signaled to a decoder to indicate whether the block uses the DIMD mode. If coded using the DIMD mode, a flag is further signaled to the decoder to indicate which fusion scheme is used, for example, as the first fusion method or the second fusion method described above.

[0131] Modify the DIMD mode used in the MPM list Another aspect of the present disclosure proposes deriving intra modes from a DIMD into a list of intra most probable modes (MPMs) depending on whether the derived intra modes are already used in the DIMD. According to one or more embodiments of the present disclosure, if a fusion scheme is used in the DIMD mode, the intra modes derived from the DIMD may be used as candidates for the MPM list. In other words, if a fusion scheme is not used in the DIMD mode, the intra modes derived from the DIMD cannot be used as candidates for the MPM list.

[0132] In other aspects of the present disclosure, directional modes with an offset from the available directional modes of the DIMD may be used as candidates for the MPM list. In one particular example, the offset may be 1, -1, 2, -2, 3, -3, 4, -4.

[0133] 17, in one example, in step 1702, the decoder may determine whether a fusion scheme is applied in DIMD mode, where the fusion scheme is applied as a weighted average of predictors in DIMD mode. In step 1704, the decoder may apply an offset to an available directional mode in DIMD mode to obtain an offset directional mode. In step 1706, the decoder may determine whether to add the offset directional mode to a list of most probable modes (MPMs) based on whether the fusion scheme is applied in DIMD mode.

[0134] As an example, a general MPM list with 22 entries is first constructed, then the first 6 entries in this general MPM list are included in the primary MPM (PMPM) list, and the remaining entries form the secondary MPM (SMPM) list. The first entry in the general MPM list is the planar mode. The remaining entries are the intra modes of the left (L), top (A), bottom-left (BL), top-right (AR), and top-left (AL) neighboring blocks, and the DIMD mode. Do and, the first two available directional modes and the DIMD mode of the neighboring block Doka These offsets are added to the directional modes, and the default mode {DC_IDX(1), VER_IDX(50), HOR_IDX(18), VER_IDX-4(46), VER_IDX+4(54), 14, 22, 42, 58, 10, 26, 38, 62, 6, 30, 34, 66, 2, 48, 52, 16}.

[0135] If the CU blocks are oriented vertically, the order of the adjacent blocks is A, L, BL, AR, AL; otherwise, the order is L, A, BL, AR, AL.

[0136] In this example, the DIMD mode with no offset is Do It is first added to the MPM list. If the list is not full, the DIMD mode with the offset is Do It will be added to the MPM list.

[0137] TIMD mode used in MPM list Another aspect of the present disclosure proposes deriving intra modes from TIMD into a list of intra most probable modes (MPMs). Generally, VVC has 67 intra prediction modes, including non-directional modes (planar, DC) and 65 angular modes, which efficiently model various directional structures typically present in video and image content. In one or more embodiments of the present disclosure, intra modes derived from TIMDs may be used as candidates for the MPM list. In one example, intra modes derived from DIMDs cannot be used as candidates for the MPM list, but intra modes derived from TIMDs may be used as candidates for the MPM list.

[0138] Another aspect of the present disclosure proposes deriving intra modes from a TIMD into a list of intra most probable modes (MPMs) depending on whether the derived intra modes are already used in the TIMD. According to one or more embodiments of the present disclosure, if a fusion scheme is used in the TIMD mode, the intra modes derived from the TIMD may be used as candidates for the MPM list. In other words, if a fusion scheme is not used in the TIMD mode, the intra modes derived from the TIMD cannot be used as candidates for the MPM list.

[0139] The above methods may be implemented using an apparatus including one or more circuits, including an application specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a controller, a microcontroller, a microprocessor, or other electronic components. The apparatus may use the circuits in combination with other hardware or software components to perform the above-described methods. Each module, sub-module, unit, or sub-unit disclosed above may be implemented at least in part using one or more circuits.

[0140] Simplified DIMD and TIMD using integer arithmetic As previously pointed out, existing designs of DIMD and TIMD involve several floating-point operations (i.e., addition, multiplication, and division) to derive DIMD / TIMD parameters, which are unacceptable for practical codec implementations in both software and hardware. In this section, a look-up table (LUT)-based scheme is proposed to simplify the implementation of DIMD and TIMD by replacing all floating-point operations with integer additions and multiplications. In one example, a decoder identifies floating-point division operations to be performed to derive parameters in DIMD or TIMD mode, and then the decoder obtains the parameters in DIMD or TIMD by replacing the floating-point division operations with integer additions and multiplications based on the look-up table (LUT).

[0141] Specifically, as shown in FIG. 13, one integer L is divided into two parts by one exponent and a significant part of K bits (including K most significant bits (MSB) after the exponent):

number

number

[0142] If the fractional part 1 / L is quantized with M-bit precision, the above equation becomes:

number

number

number

[0143] Correspondingly, the proposed integerization scheme can be achieved as follows, and division between any two integers, e.g.,

number

number

[0144] In practice, various combinations of LUT size (i.e., K) and parameter precision (i.e., M) may be applied to achieve various trade-offs between the accuracy of derived parameters and the complexity of implementation. For example, using a large LUT size and higher parameter precision is beneficial to maintain high parameter precision, but comes at the expense of increased storage size for maintaining the LUT and increased bit depth for performing corresponding integer operations (e.g., integer multiplication, addition, and bitwise shift). Based on such considerations, in one particular example, it is proposed to set the values ​​of K and M to 4. Based on such settings, the corresponding LUT is

number

[0145] Using the DIMD derivation as an example, for the integerization scheme above (where both K and M are set to 4), the derivation of the gradient orientation at each template sample is:

number

[0146] Please note that the values ​​of K and M used in the above example are for illustration purposes only. In practice, the proposed LUT-based scheme using different values ​​of K and M may be applied to convert floating-point division into integer operations in the process of other encoding techniques in the future.

[0147] TIMD with multiple reference line selection Another aspect of the present disclosure proposes deriving intra modes from TIMD using multiple reference line selection. According to one or more embodiments of the present disclosure, different reference lines within a reconstruction area may be used to derive intra modes in TIMD. In one example, for a given CU, a flag indicating whether a block uses TIMD mode is signaled to a decoder. If the block is coded using TIMD mode, a reference line index is further signaled to the decoder to indicate which reference line is used in TIMD mode. Furthermore, the reference line index may be signaled in a manner similar to existing MRL designs.

[0148] According to one or more embodiments of the present disclosure, the reference line index may be binarized by various methods. Various methods may be used to derive the variable set of binary code words, and some exemplary methods are listed below. 1. Fixed-length binarization 2. Truncated Rice Binarization 3. Truncated Unary Binarization Process 4. Truncated binary (TB) binarization process 5. k-th order Exponential Golomb Binarization (EGk) 6. Limited k-th Exponential-Golomb Binarization

[0149] According to one or more embodiments of the present disclosure, the indexes of the reference lines may be predefined to be groups and used for TIMD. Different reference lines may be used to form groups, and some exemplary methods are listed below. 1. {0,1,2,...,N}, where N is a predefined integer 2. {0,2,4,...,2*N}, where N is a predefined integer 3.{0,1,3,5,7,12} 4.{0,1,2} 5.{0,1,3} 6.{0,1,3,5,7} 7.{0,1,3,5}

[0150] In one example, the MRL candidate list {1, 3, 5, 7, 12} is used, as shown in FIG.

[0151] According to one or more embodiments of the present disclosure, TIMD with multiple reference line selection is disabled for the first line of a block inside a CTU to prevent using extended reference samples outside the current CTU line.

[0152] 20 is a block diagram illustrating a video encoding / decoding process according to an example of this disclosure. In step 2002, a decoder receives a signal flag indicating whether a template-based intra-mode derivation (TIMD) mode is applied to a current video block. In step 2004, in response to determining that the TIMD mode is applied to the current video block based on the signal flag, the decoder derives an intra-prediction mode from the TIMD mode based on different reference lines within a reconstructed area of ​​the current video block, the reconstructed area including reconstructed neighboring blocks of the current video block.

[0153] In another example of this disclosure, a video encoding process is provided. An encoder may signal a flag indicating whether TIMD mode applies to a current video block. Then, if the encoder determines that TIMD mode applies to the current video block, the encoder may derive an intra-prediction mode from the TIMD mode based on different reference lines within a reconstructed area of ​​the current video block. The encoder may further determine reference line indices indicating the reference lines used in the TIMD mode in response to determining that TIMD mode applies to the current video block, and the reference line indices are signaled to derive the intra-prediction mode for the current video block. In one or more examples, the reference line indices are binarized using a set of binary codewords from a variable set of binary codewords. Each set of binary codewords in the variable set of binary codewords may be derived by one of the following binarization processes: a fixed-length binarization process, a truncated Rice binarization process, a truncated unary binarization process, a truncated binary (TB) binarization process, a kth-order Exponential-Golomb (EGk) binarization process, and a restricted kth-order Exponential-Golomb (EGk) binarization process.

[0154] Here, the reference line indices may be predefined to be groups for the TIMD mode, and each group of reference line indices may include a different reference line. In one or more examples, the groups of reference line indices may include {0,1,2,...,N}, {0,2,4,...,2*N}, {0,1,3,5,7,12}, {0,1,2}, {0,1,3}, {0,1,3,5,7}, {0,1,3,5}, where N is a predefined integer.

[0155] 14 shows a computing environment 1610 coupled with a user interface 1650. The computing environment 1610 may be part of a data processing server. The computing environment 1610 includes a processor 1620, a memory 1630, and an input / output (I / O) interface 1640.

[0156] The processor 1620 typically controls the overall operation of the computing environment 1610, such as operations related to display, data acquisition, data communication, and image processing. The processor 1620 may include one or more processors for executing instructions called for performing all or some of the steps in the methods described above. Additionally, the processor 1620 may include one or more modules that facilitate interaction between the processor 1620 and other components. The processor may be a central processing unit (CPU), a microprocessor, a single-chip machine, a graphics processing unit (GPU), etc.

[0157] Memory 1630 is configured to store various types of data to support the operation of computing environment 1610. Memory 1630 may include predefined software 1632. Examples of such data include instructions for any applications or methods run on computing environment 1610, video data sets, image data, etc. Memory 1630 may be implemented using any type of volatile or non-volatile memory device, or combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0158] The I / O interface 1640 provides an interface between the processor 1620 and a peripheral interface module, such as a keyboard, a click wheel, buttons, etc. The buttons may include, but are not limited to, a home button, a start scan button, and a stop scan button. The I / O interface 1640 may be coupled to an encoder and a decoder.

[0159] In one embodiment, a non-transitory computer-readable storage medium is also provided that includes, for example in memory 1630, a plurality of programs executable by processor 1620 in computing environment 1610 for performing the above-described methods. In one example, the plurality of programs may be executed by processor 1620 in computing environment 1610 to receive (e.g., from video encoder 20 of FIG. 2 ) a bitstream or data stream including encoded video information (e.g., video blocks representing encoded video frames and / or one or more associated syntax elements, etc.) and to perform the above-described decoding method in accordance with the received bitstream or data stream. In another example, multiple programs may be executed by processor 1620 in computing environment 1610 to perform the above-described encoding method to encode video information (e.g., video blocks representing video frames and / or one or more associated syntax elements) into a bitstream or data stream and to transmit the bitstream or data stream (e.g., to video decoder 30 of FIG. 3). Alternatively, a non-transitory computer-readable storage medium may store a bitstream or data stream including encoded video information (e.g., video blocks representing encoded video frames and / or one or more associated syntax elements) generated by an encoder (e.g., video encoder 20 of FIG. 2) using, for example, the above-described encoding method, for use by a decoder (e.g., video decoder 30 of FIG. 3) in decoding the video data. The non-transitory computer-readable storage medium may be, for example, a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0160] In one embodiment, a computing device is also provided that includes one or more processors (e.g., processor 1620) and a non-transitory computer-readable storage medium or memory 1630 storing a plurality of programs executable by the one or more processors, wherein the one or more processors are configured, upon execution of the plurality of programs, to perform the methods described above.

[0161] In one embodiment, a computer program product is also provided that includes a plurality of programs, e.g., in memory 1630, executable by processor 1620 in computing environment 1610 to perform the methods described above. For example, the computer program product may include a non-transitory computer-readable storage medium.

[0162] In one embodiment, the computing environment 1610 may be implemented using one or more ASICs, DSPs, digital signal processing devices (DSPDs), programmable logic devices (PLDs), FPGAs, GPUs, controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0163] The description of the present disclosure has been presented for purposes of illustration and is not intended to be exhaustive or limiting of the disclosure. Many modifications, variations and alternative implementations will be apparent to one skilled in the art having the benefit of the teachings presented in the foregoing descriptions and the associated drawings.

[0164] Unless otherwise specified, the order of steps of the method according to the present disclosure is intended to be illustrative only, and the steps of the method according to the present disclosure are not limited to the order specifically described above, and may be changed according to actual conditions. Furthermore, at least one of the steps of the method according to the present disclosure may be adjusted, combined, or deleted according to actual requirements.

[0165] The examples have been chosen and described to explain the principles of the disclosure, to enable those skilled in the art to understand the disclosure in various implementations, and to make best use of the underlying principles and various implementations with various modifications as suited to the particular use contemplated. Therefore, it should be understood that the scope of the disclosure is not limited to the particular examples of implementations disclosed, and that modifications and other implementations are intended to be included within the scope of the disclosure.

Claims

1. Obtaining, by a decoder, a signal flag indicating whether a template-based intra mode derivation (TIMD) mode is applied to a current video block; deriving, by the decoder, an intra-prediction mode from the TIMD mode based on different reference lines within a reconstructed area of ​​the current video block in response to determining that the TIMD mode is to be applied to the current video block based on the signal flag, the reconstructed area including reconstructed neighboring blocks of the current video block; Including, deriving, by the decoder, the intra-prediction mode, , N}, {0, 2, 4, ..., 2*N}, where N is a predefined integer greater than 2.

10. A video decoding method comprising: obtaining, by the decoder, an index of a reference line indicating a reference line to be used in the TIMD mode; the index of the reference line being signaled to derive the intra-prediction mode of the current video block; the index of the reference line being predefined to be groups for the TIMD mode, each group of the index of the reference line including a different reference line; and the groups of the index of the reference line including one of: {0, 1, 3, 5, 7, 12}, {0, 1, 3, 5, 7}, {0, 1, 3, 5}, {0, 1, 2, ..., N}, {0, 2, 4, ..., 2*N}, where N is a predefined integer greater than 2.

2. 2. The video decoding method of claim 1, wherein the index of the reference line is binarized using a set of binary codewords from a variable set of binary codewords.

3. 3. The video decoding method of claim 2, wherein each set of binary codewords of the variable set of binary codewords is derived by one of the following binarization processes: a fixed-length binarization process, a truncated Rice binarization process, a truncated unary binarization process, a truncated binary (TB) binarization process, a kth-order Exponential-Golomb (EGk) binarization process, and a restricted kth-order Exponential-Golomb (EGk) binarization process.

4. signaling, by the encoder, a flag indicating whether a template-based intra mode derivation (TIMD) mode is applied to the current video block; deriving, by the encoder, in response to determining that the TIMD mode is applied to the current video block, an intra-prediction mode from the TIMD mode based on different reference lines within a reconstructed area of ​​the current video block, the reconstructed area including reconstructed neighboring blocks of the current video block; Including, deriving, by the encoder, the intra-prediction mode, , N}, {0, 2, 4, ..., 2*N}, where N is a predefined integer greater than 2.

10. A video encoding method comprising: determining, by the encoder, an index of a reference line indicating a reference line to be used in the TIMD mode; the index of the reference line being signaled to derive the intra-prediction mode of the current video block; the index of the reference line being predefined to be groups for the TIMD mode, each group of the index of the reference line including a different reference line; and the groups of the index of the reference line including one of: {0, 1, 3, 5, 7, 12}, {0, 1, 3, 5, 7}, {0, 1, 3, 5}, {0, 1, 2, ..., N}, {0, 2, 4, ..., 2*N}, where N is a predefined integer greater than 2.

5. 5. The video encoding method of claim 4, wherein the index of the reference line is binarized using a set of binary codewords from a variable set of binary codewords.

6. 6. The video encoding method of claim 5, wherein each set of binary codewords of the variable set of binary codewords is derived by one of the following binarization processes: a fixed-length binarization process, a truncated Rice binarization process, a truncated unary binarization process, a truncated binary (TB) binarization process, a kth-order Exponential-Golomb (EGk) binarization process, and a restricted kth-order Exponential-Golomb (EGk) binarization process.

7. An apparatus for video decoding, comprising: one or more processors; a memory configured to store instructions executable by the one or more processors; 4. Apparatus, wherein the one or more processors are configured, upon execution of the instructions, to perform the video decoding method of claim 1.

8. An apparatus for video encoding, comprising: one or more processors; a memory configured to store instructions executable by the one or more processors; Apparatus, wherein the one or more processors are configured, upon execution of the instructions, to perform the video encoding method of any of claims 4 to 6.

9. A non-transitory computer-readable storage medium for storing computer-executable instructions that, when executed by one or more computer processors, cause the one or more computer processors to perform the video decoding method of any of claims 1 to 3.

10. A non-transitory computer-readable storage medium for storing computer-executable instructions that, when executed by one or more computer processors, cause the one or more computer processors to perform a video encoding method described in any of claims 4 to 6.

11. 1. A method for storing a bitstream, comprising: generating a bitstream by performing an encoding method; storing the bitstream; The bitstream is decoded by a video decoding method according to any one of claims 1 to 3, The encoding method comprises: signaling, by the encoder, a flag indicating whether a template-based intra mode derivation (TIMD) mode is applied to the current video block; in response to determining that the TIMD mode is applied to the current video block, deriving, by the encoder, an intra-prediction mode from the TIMD mode based on different reference lines within a reconstructed area of ​​the current video block, the reconstructed area including reconstructed neighboring blocks of the current video block; deriving, by the decoder, the intra-prediction mode, , N}, {0, 2, 4, ..., 2*N}, where N is a predefined integer greater than 2.

10. The method of claim 10, further comprising: determining, by the encoder, indices of reference lines indicating reference lines to be used in the TIMD mode in response to determining that the TIMD mode is to be applied to the current video block; the indices of the reference lines are signaled to derive the intra-prediction mode of the current video block; the indices of the reference lines are predefined to be groups for the TIMD mode, each group of the indices of the reference lines including a different reference line; and the groups of indices of the reference lines include one of: {0, 1, 3, 5, 7, 12}, {0, 1, 3, 5, 7}, {0, 1, 3, 5}, {0, 1, 2, ..., N}, {0, 2, 4, ..., 2*N}, where N is a predefined integer greater than 2.

12. A method for storing a bitstream, comprising: Executing a video encoding method according to any one of claims 4 to 6 to generate a bitstream; storing said bitstream; A method comprising:

13. 1. A method for transmitting a bitstream, comprising: generating a bitstream by performing an encoding method; transmitting the bitstream; The bitstream is decoded by a video decoding method according to any one of claims 1 to 3, The encoding method comprises: signaling, by the encoder, a flag indicating whether a template-based intra mode derivation (TIMD) mode is applied to the current video block; in response to determining that the TIMD mode is applied to the current video block, deriving, by the encoder, an intra-prediction mode from the TIMD mode based on different reference lines within a reconstructed area of ​​the current video block, the reconstructed area including reconstructed neighboring blocks of the current video block; deriving, by the decoder, the intra-prediction mode, determining, by the encoder, an index of a reference line indicating a reference line to be used in the TIMD mode, the index of the reference line being signaled to derive the intra-prediction mode of the current video block, the index of the reference line being predefined to be a group for the TIMD mode, each group of the index of the reference line including a different reference line, the group of the index of the reference line including one of: {0, 1, 3, 5, 7, 12}, {0, 1, 3, 5, 7}, {0, 1, 3, 5}, {0, 1, 2, ..., N}, {0, 2, 4, ..., 2*N}, where N is a predefined integer greater than 2. method.

14. A method for transmitting a bitstream, comprising: Executing a video encoding method according to any one of claims 4 to 6 to generate a bitstream; transmitting the bitstream.

15. A computer program comprising instructions which, when executed by one or more computer processors, cause said one or more computer processors to perform the video decoding method of any of claims 1 to 3.

16. A computer program comprising instructions which, when executed by one or more computer processors, cause the one or more computer processors to perform the video encoding method of any of claims 4 to 6.

Images