Cross-component prediction for improved video coding

Abstract

A video decoding method is provided. The method comprises the following steps: obtaining a coding unit in a current picture from a video bit stream; selecting a plurality of groups of adjacent sample points of the coding unit; determining one or more cross-component prediction models based on the plurality of groups of adjacent sample points; obtaining at least one reconstructed luma sample in the luma block corresponding to the chroma sample in the at least one chroma block; and applying at least one of the one or more cross-component prediction models to the at least one reconstructed luma sample to predict the chroma sample.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application is filed based on and claims priority to U.S. Provisional Application No. 63 / 354,176 filed on June 21, 2022 and U.S. Provisional Application No. 63 / 354,679 filed on June 22, 2022, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present application relates to image / video encoding and decoding and compression. More specifically, the present application relates to a method and apparatus for improving the encoding and decoding efficiency of image / video blocks. Background Art

[0004] Various electronic devices such as digital televisions, laptop or desktop computers, tablet computers, digital cameras, digital recording devices, digital media players, video game consoles, smart phones, video teleconferencing devices, video streaming devices, etc. support digital video. Electronic devices transmit and receive or otherwise transmit digital video data through a communication network and / or store digital video data on a storage device. Due to the limited bandwidth capacity of the communication network and the limited memory resources of the storage device, before the video data is transmitted or stored, video codecs can be used to compress the video data according to one or more video codec standards. For example, video codec standards include universal video codec (VVC), joint exploration test model (JEM), high efficiency video codec (HEVC / H.265), advanced video codec (AVC / H.264), moving picture expert group (MPEG) codec, etc. Video codecs are usually performed using prediction methods (e.g., inter-frame prediction, intra-frame prediction, etc.), which use the redundancy inherent in video data. Video codecs are intended to compress video data into a form using a lower bit rate while avoiding or minimizing video quality degradation. Summary of the invention

[0005] Embodiments of the present disclosure provide a method and apparatus for video encoding and decoding.

[0006] According to a first aspect of the present disclosure, a video decoding method is provided. The method includes obtaining a coding unit in a current picture from a bitstream, wherein the coding unit includes a luminance block and a chrominance block; obtaining a reconstructed luminance sample in the luminance block; determining one or more cross-component prediction models based on a block size, wherein the block size is the size of the luminance block or the size of a reconstructed neighboring block located at the top or left of the luminance block; and applying at least one of the one or more cross-component prediction models to at least the reconstructed luminance sample to predict the chrominance sample in the chrominance block.

[0007] According to a second aspect of the present disclosure, a video encoding method is provided. The method includes partitioning a video frame into a plurality of coding units, wherein a coding unit in the plurality of coding units includes a luminance block and a chrominance block; obtaining a reconstructed luminance sample in the luminance block; determining one or more cross-component prediction models based on a block size, wherein the block size is the size of the luminance block or the size of a reconstructed neighboring block located at the top or left of the luminance block; and applying at least one of the one or more cross-component prediction models to at least the reconstructed luminance sample to predict the chrominance sample in the chrominance block.

[0008] According to a third aspect of the present disclosure, an electronic device is provided. The electronic device includes: one or more processors; a memory coupled to the one or more processors; and a plurality of programs stored in the memory, wherein when the plurality of programs are executed by the one or more processors, the electronic device receives a video bitstream to execute the decoding method provided in the embodiment of the present application, or the electronic device executes the encoding method provided in the embodiment of the present application to generate a video bitstream.

[0009] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium is provided. The non-transitory computer-readable storage medium stores a plurality of programs for execution by an electronic device having one or more processors, wherein the plurality of programs, when executed by the one or more processors, enable the electronic device to receive a video bitstream to execute the decoding method provided in the embodiment of the present application, or enable the electronic device to execute the encoding method provided in the embodiment of the present application to generate a video bitstream.

[0010] According to a fifth aspect of the present disclosure, a computer program product is provided. The computer program product includes instructions, which, when executed by a processor, cause the processor to receive a video bitstream to perform the decoding method provided in the embodiment of the present application, or cause the processor to perform the encoding method provided in the embodiment of the present application to generate a video bitstream.

[0011] It is to be understood that both the foregoing general description and the following detailed description are exemplary only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate examples consistent with the disclosure and, together with the description, serve to explain the principles of the disclosure.

[0013] Figure 1 is a block diagram illustrating an example system for encoding and decoding video blocks according to some embodiments of the present disclosure.

[0014] Figure 2is a block diagram illustrating an example video encoder according to some embodiments of the present disclosure.

[0015] Figure 3 is a block diagram illustrating an example video decoder according to some embodiments of the present disclosure.

[0016] FIG. 4A to FIG. 4E is a block diagram illustrating how a frame may be recursively partitioned into multiple video blocks of different sizes and shapes according to some embodiments of the present disclosure.

[0017] Figure 5 is an example of the positions of the left and upper samples and the samples of the current block.

[0018] Fig. 6A An example is shown where a multidirectional linear model (MDLM) works when the block content cannot be predicted from an L-shaped reconstructed region.

[0019] Figure 6B An example is shown where MDLM_L uses only the left-side reconstruction samples to derive the cross-component linear model (CCLM) parameters.

[0020] Figure 6C An example is shown where MDLM_T uses only the top reconstruction samples to derive CCLM parameters.

[0021] Figure 7 An example of dividing neighboring samples into two groups based on the value of a threshold is shown.

[0022] Figure 8 An example of dividing neighboring sample points into two groups based on the inflection point is shown.

[0023] FIG. 9A to FIG. 9B An example process for performing slope adjustment on a CCLM is illustrated.

[0024] Fig.10 An example of co-located reconstructed luma samples of the current chroma block is shown.

[0025] Fig.11 An example of selecting neighboring reconstructed luma and chroma samples is shown.

[0026] FIG. 12A to FIG. 12D An example process of decoder-side intra mode derivation (DIMD) is shown.

[0027] Fig.13 An example of four reference rows adjacent to a prediction block is shown.

[0028] Fig.14 An example of the location of luma samples in the convolution filter is shown.

[0029] Fig.15An example of a reference region for deriving filter coefficients is shown.

[0030] FIG. 16A to FIG. 16B An example is shown where one chroma sample is associated with multiple luma samples simultaneously.

[0031] Fig.17 Examples of luma and chroma samples used to derive parameters of the prediction model are shown.

[0032] Fig.18 Another example of luma and chroma samples used to derive parameters of a prediction model is shown.

[0033] Fig.19 An example is shown where the top or left reconstruction samples are used for a filter-based linear model (FLM).

[0034] Fig. 20 Another example in which the reconstruction samples are used for FLM is shown.

[0035] Fig.21 An example of 1-tap / 2-tap pre-operation is shown.

[0036] Fig. 22 Examples of different filter shapes and numbers of filter taps are shown.

[0037] Fig.23 Other examples of different filter shapes and numbers of filter taps are shown.

[0038] Fig.24 and FIG. 25A to FIG. 25G Other examples of different filter shapes and numbers of filter taps are shown.

[0039] Fig.26 is a flowchart illustrating a video decoding method according to some embodiments of the present disclosure.

[0040] Fig. 27 is a flowchart illustrating a video encoding method according to some embodiments of the present disclosure.

[0041] Fig.28 is a diagram illustrating a computing environment coupled with a user interface according to some implementations of the present disclosure. DETAILED DESCRIPTION

[0042] Reference will now be made in detail to specific embodiments, examples of which are illustrated in the accompanying drawings. In the following detailed description, numerous non-limiting specific details are set forth to aid in understanding the subject matter presented herein. However, it will be apparent to one of ordinary skill in the art that various alternatives may be used without departing from the scope of the claims, and that the subject matter may be practiced without these specific details. For example, it will be apparent to one of ordinary skill in the art that the subject matter presented herein may be implemented on many types of electronic devices having digital video capabilities.

[0043] It should be noted that the terms "first", "second", etc. used in the specification, claims, and drawings of the present disclosure are used to distinguish objects rather than to describe any particular order or sequence. It should be understood that the terms used in this manner can be interchanged under appropriate conditions, so that the embodiments of the present disclosure described herein can be implemented in an order other than those shown in the drawings or described in the present disclosure.

[0044] Various video codec technologies can be used to compress video data. Video coding is performed according to one or more video codec standards. For example, video codec standards include Versatile Video Codec (VVC), High Efficiency Video Codec (H.265 / HEVC), Advanced Video Codec (H.264 / AVC), Moving Picture Experts Group (MPEG) codec, etc. Video codecs typically use prediction methods (e.g., inter-frame prediction, intra-frame prediction, etc.) that exploit redundancy present in video images or sequences. An important goal of video coding technology is to compress video data into a form that uses a lower bit rate while avoiding or minimizing video quality degradation.

[0045] The first version of the VVC standard was completed in July 2020, providing approximately 50% bitrate savings or equivalent perceptual quality compared to the previous generation video codec standard HEVC. Although the VVC standard provides significant codec improvements over its predecessor, there is evidence that superior codec efficiency can be achieved using additional codec tools. Recently, the Joint Video Exploration Team (JVET), in collaboration with ITU-T VCEG and ISO / IEC MPEG, began exploring advanced technologies that can significantly improve codec efficiency compared to VVC. In April 2021, a software code base called the Enhanced Compression Model (ECM) was established for future video codec exploration work. The ECM reference software is based on the VVC Test Model (VTM) developed by JVET for VVC, and further extends and / or improves several existing modules (e.g., intra / inter prediction, transform, in-loop filter, etc.). In the future, any new codec tools beyond the VVC standard will need to be integrated into the ECM platform and tested using the JVET Common Test Conditions (CTC).

[0046] Similar to all previous video codec standards, ECM is built on a block-based hybrid video codec framework. The input video signal is processed block by block, known as coding units (CUs). In ECM-1.0, a CU can be up to 128×128 pixels. However, like VVC, a coding tree unit (CTU) is divided into multiple CUs to accommodate different local characteristics based on quad / bin / ternary trees. In the multi-type tree structure, a CTU is first partitioned according to a quadtree structure. Then, each quadtree leaf node can be further partitioned according to a binary tree structure and a ternary tree structure.

[0047] Figure 1 is a block diagram illustrating an example system 10 for encoding and decoding video blocks in parallel according to some embodiments of the present disclosure. Figure 1 As shown, system 10 includes a source device 12 that generates and encodes video data to be later decoded by a destination device 14. Source device 12 and destination device 14 may include any of a variety of electronic devices, including a desktop or laptop computer, a tablet computer, a smart phone, a set-top box, a digital television, a camera, a display device, a digital media player, a video game console, a video streaming device, etc. In some implementations, source device 12 and destination device 14 are equipped with wireless communication capabilities.

[0048] In some embodiments, the target device 14 may receive the encoded video data to be decoded via the link 16. The link 16 may include any type of communication medium or device capable of moving the encoded video data from the source device 12 to the target device 14. In one example, the link 16 may include a communication medium for enabling the source device 12 to directly transmit the encoded video data to the target device 14 in real time. The encoded video data may be modulated and transmitted to the target device 14 according to a communication standard such as a wireless communication protocol. The communication medium may include any wireless or wired communication medium, such as a radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium may form a portion of a packet-based network such as a local area network, a wide area network, or a global network such as the Internet. The communication medium may include a router, a switch, a base station, or any other device that may be used to facilitate communication from the source device 12 to the target device 14.

[0049] In some embodiments, the encoded video data may be transferred from the output interface 22 to the storage device 32. Subsequently, the target device 14 may access the encoded video data in the storage device 32 via the input interface 28. The storage device 32 may include any of a variety of distributed or locally accessed data storage media, such as a hard drive, a Blu-ray disc, a digital versatile disc (DVD), a compact disc read-only memory (CD-ROM), a flash memory, a volatile memory or a non-volatile memory, or any other suitable digital storage medium for storing encoded video data. In a further example, the storage device 32 may correspond to a file server or another intermediate storage device that can store the encoded video data generated by the source device 12. The target device 14 may access the video data stored in the storage device 32 via streaming or downloading. The file server may be any type of computer capable of storing the encoded video data and transmitting the encoded video data to the target device 14. Example file servers include a web server (e.g., for a website), a file transfer protocol (FTP) server, a network attached storage (NAS) device, or a local disk drive. The target device 14 can access the encoded video data through any standard data connection suitable for accessing the encoded video data stored on the file server, including a wireless channel suitable for accessing the encoded video data stored on the file server (e.g., a wireless fidelity (Wi-Fi) connection), a wired connection (e.g., a digital subscriber line (DSL), a cable modem, etc.), or a combination of both. The transmission of the encoded video data from the storage device 32 can be a streaming transmission, a download transmission, or a combination of both.

[0050] like Figure 1As shown in , source device 12 includes video source 18, video encoder 20 and output interface 22. Video source 18 may include a source such as a video capture device (e.g., a camera), a video archive containing previously captured video, a video feed interface for receiving video from a video content provider, and / or a computer graphics system for generating computer graphics data as computer graphics data of the source video. As an example, if video source 18 is a camera of a security monitoring system, source device 12 and target device 14 may form a camera phone or a video phone. However, the embodiments described in the present application may be generally applicable to video encoding and decoding and may be applied to wireless and / or wired applications.

[0051] Captured, pre-captured, or computer-generated video may be encoded by video encoder 20. The encoded video data may be transmitted directly to target device 14 via output interface 22 of source device 12. The encoded video data may also (additionally or alternatively) be stored on storage device 32 for later access by target device 14 or other devices for decoding and / or playback. Output interface 22 may further include a modem and / or a transmitter.

[0052] Target device 14 includes input interface 28, video decoder 30, and display device 34. Input interface 28 may include a receiver and / or a modem and receives encoded video data via link 16. The encoded video data transmitted via link 16 or provided on storage device 32 may include various syntax elements generated by video encoder 20 for use in decoding the video data by video decoder 30. Such syntax elements may be included within the encoded video data transmitted over a communication medium, stored on a storage medium, or stored on a file server.

[0053] In some implementations, the target device 14 may include a display device 34, which may be an integrated display device or an external display device configured to communicate with the target device 14. The display device 34 displays the decoded video data to a user and may include any of a variety of display devices, such as a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, or another type of display device.

[0054] The video encoder 20 and the video decoder 30 can operate according to proprietary or industry standards (such as VVC, HEVC, MPEG-4 Part 10, AVC, or an extension of such a standard). It should be understood that the present application is not limited to a specific video encoding / decoding standard and can be applied to other video encoding / decoding standards. It is generally believed that the video encoder 20 of the source device 12 can be configured to encode the video data according to any standard in these current or future standards. Similarly, it is also generally believed that the video decoder 30 of the target device 14 can be configured to decode the video data according to any standard in these current or future standards.

[0055] The video encoder 20 and the video decoder 30 can each be implemented as any of a variety of suitable encoder and / or decoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or any combination thereof. When implemented partially in software, the electronic device can store instructions for the software in a suitable non-transitory computer-readable medium and use one or more processors to execute the instructions in the hardware to perform the video encoding / decoding operations disclosed in the present disclosure. Each of the video encoder 20 and the video decoder 30 can be included in one or more encoders or decoders, either of which can be integrated as part of a combined encoder / decoder (CODEC) in a corresponding device.

[0056] Figure 2 is a block diagram illustrating an example video encoder 20 according to some embodiments described in the present application. The video encoder 20 can perform intra-frame prediction coding and inter-frame prediction coding on video blocks within a video frame. Intra-frame prediction coding relies on spatial prediction to reduce or remove spatial redundancy of video data within a given video frame or picture. Inter-frame prediction coding relies on temporal prediction to reduce or remove temporal redundancy of video data within adjacent video frames or pictures of a video sequence. It should be noted that in the field of video coding and decoding, the term "frame" can be used as a synonym for the term "image" or "picture".

[0057] like Figure 2As shown in , the video encoder 20 includes a video data memory 40, a prediction processing unit 41, a decoded picture buffer (DPB) 64, an adder 50, a transform processing unit 52, a quantization unit 54, and an entropy coding unit 56. The prediction processing unit 41 further includes a motion estimation unit 42, a motion compensation unit 44, a partition unit 45, an intra-frame prediction processing unit 46, and an intra-frame block copy (BC) unit 48. In some embodiments, the video encoder 20 also includes an inverse quantization unit 58 for video block reconstruction, an inverse transform processing unit 60, and an adder 62. An in-loop filter 63 such as a deblocking filter can be located between the adder 62 and the DPB 64 to filter the block boundaries to remove blocking artifacts from the reconstructed video. In addition to the deblocking filter, another in-loop filter (such as a sample adaptive offset (SAO) filter, a cross-component sample adaptive offset (CCSAO) filter, and / or an adaptive in-loop filter (ALF)) can also be used to filter the output of the adder 62. It should be noted that, for the CCSAO technology, the present application is not limited to the embodiments described herein, but the present application can be applied to the case where an offset is selected for any other component of the luminance component, the Cb chrominance component, and the Cr chrominance component according to any of the luminance component, the Cb chrominance component, and the Cr chrominance component to modify any component based on the selected offset. Further, it should be noted that the first component mentioned herein can be any one of the luminance component, the Cb chrominance component, and the Cr chrominance component, the second component mentioned herein can be any other one of the luminance component, the Cb chrominance component, and the Cr chrominance component, and the third component mentioned herein can be the remaining component of the luminance component, the Cb chrominance component, and the Cr chrominance component. In some examples, the in-loop filter can be omitted, and the decoded video block can be directly provided to the DPB 64 by the adder 62. The video encoder 20 can be in the form of a fixed or programmable hardware unit, or can be dispersed in one or more of the illustrated fixed or programmable hardware units.

[0058] Video data memory 40 may store video data to be encoded by components of video encoder 20. For example, video data may be stored in memory 40. Figure 1 The video source 18 shown obtains video data in the video data memory 40. The DPB 64 is a buffer that stores reference video data (e.g., reference frames or pictures) for encoding the video data by the video encoder 20 (e.g., in an intra-frame prediction coding mode or an inter-frame prediction coding mode). The video data memory 40 and the DPB 64 can be formed by any of a variety of memory devices. In various examples, the video data memory 40 can be on-chip with other components of the video encoder 20, or off-chip relative to those components.

[0059] like Figure 2 As shown in , after receiving the video data, the partition unit 45 within the prediction processing unit 41 partitions the video data into video blocks. The partitioning may also include partitioning the video frame into strips, tiles (e.g., video block sets), or other larger coding units (CUs) according to a predefined partitioning structure associated with the video data, such as a quadtree (QT) structure. A video frame is or may be viewed as a two-dimensional array or matrix of samples having sample values. The samples in the array may also be referred to as pixels (pixel or pel). The multiple samples in the horizontal and vertical directions (or axes) of the array or picture define the size and / or resolution of the video frame. For example, a video frame may be divided into multiple video blocks by using QT partitioning. A video block is again or may be viewed as a two-dimensional array or matrix of samples having sample values, but its size is smaller than a video frame. The multiple samples in the horizontal and vertical directions (or axes) of a video block define the size of the video block. The video block can be further divided into one or more block partitions or sub-blocks (which can again form blocks) by, for example, iteratively using QT partitioning, binary tree (BT) partitioning, or ternary tree (TT) partitioning, or any combination thereof. It should be noted that the term "block" or "video block" used herein can be a part of a frame or picture, such as a rectangular (square or non-square) part. For example, with reference to HEVC and VVC, a block or video block can be or correspond to a coding tree unit (CTU), a CU, a prediction unit (PU), or a transform unit (TU), and / or can be or correspond to a corresponding block (e.g., a coding tree block (CTB), a coding block (CB), a prediction block (PB), or a transform block (TB)) and / or correspond to a sub-block.

[0060] Prediction processing unit 41 may select one of a plurality of possible prediction coding modes for the current video block based on the error results (e.g., coding rate and distortion level), such as one of a plurality of intra-frame prediction coding modes or one of a plurality of inter-frame prediction coding modes. Prediction processing unit 41 may provide the resulting intra-frame prediction coding block or inter-frame prediction coding block to adder 50 to generate a residual block, and to adder 62 to reconstruct the coded block for subsequent use as part of a reference frame. Prediction processing unit 41 also provides syntax elements such as motion vectors, intra-frame mode indicators, partition information, and other such syntax information to entropy coding unit 56.

[0061] In order to select an appropriate intra-frame prediction coding mode for the current video block, intra-frame prediction processing unit 46 within prediction processing unit 41 may perform intra-frame prediction coding of the current video block relative to one or more neighboring blocks in the same frame as the current block to be encoded to provide spatial prediction. Motion estimation unit 42 and motion compensation unit 44 within prediction processing unit 41 perform inter-frame prediction coding of the current video block relative to one or more prediction blocks in one or more reference frames to provide temporal prediction. Video encoder 20 may perform multiple encoding passes, for example, to select an appropriate coding mode for each block of video data.

[0062] In some embodiments, motion estimation unit 42 determines the inter-frame prediction mode of the current video frame by generating a motion vector according to a predetermined pattern within the sequence of video frames, the motion vector indicating the displacement of a video block within the current video frame relative to a prediction block within a reference video frame. Motion estimation performed by motion estimation unit 42 is the process of generating motion vectors that estimates the motion of video blocks. For example, a motion vector may indicate the displacement of a video block within a current video frame or picture relative to a prediction block within a reference frame associated with a current block encoded within the current frame. The predetermined pattern may designate video frames in the sequence as P frames or B frames. Intra BC unit 48 may determine a vector (e.g., a block vector) for intra BC encoding in a manner similar to the manner in which motion estimation unit 42 determines motion vectors for inter-frame prediction, or may utilize motion estimation unit 42 to determine the block vector.

[0063] The prediction block of the video block may be or may correspond to a block or reference block of a reference frame, which is considered to be closely matched to the video block to be encoded in terms of pixel difference values, which may be determined by the sum of absolute differences (SAD), the sum of squared differences (SSD), or other difference metrics. In some embodiments, the video encoder 20 may calculate the values ​​of sub-integer pixel positions of the reference frame stored in the DPB 64. For example, the video encoder 20 may interpolate a quarter pixel position, an eighth pixel position, or other fractional pixel position of the reference frame. Therefore, the motion estimation unit 42 may perform a motion search relative to the full pixel position and the fractional pixel position and output a motion vector with fractional pixel accuracy.

[0064] Motion estimation unit 42 calculates a motion vector for a video block in an inter-prediction coded frame by comparing the position of the video block to the position of a prediction block of a reference frame selected from a first reference frame list (List 0) or a second reference frame list (List 1), each of which identifies one or more reference frames stored in DPB 64. Motion estimation unit 42 sends the calculated motion vector to motion compensation unit 44 and then to entropy encoding unit 56.

[0065] The motion compensation performed by the motion compensation unit 44 may involve obtaining or generating a prediction block based on the motion vector determined by the motion estimation unit 42. After receiving the motion vector of the current video block, the motion compensation unit 44 may locate the prediction block pointed to by the motion vector in one of the reference frame lists, obtain the prediction block from the DPB 64 and forward the prediction block to the adder 50. The adder 50 then forms a residual video block with pixel difference values ​​by subtracting the pixel values ​​of the prediction block provided by the motion compensation unit 44 from the pixel values ​​of the current video block being encoded. The pixel difference values ​​forming the residual video block may include luminance component difference values ​​or chrominance component difference values ​​or both. The motion compensation unit 44 may also generate syntax elements associated with the video block of the video frame for use by the video decoder 30 when decoding the video block of the video frame. The syntax elements may include, for example, syntax elements defining a motion vector for identifying the prediction block, any flag indicating a prediction mode, or any other syntax information described herein. Note that the motion estimation unit 42 and the motion compensation unit 44 may be highly integrated, but are illustrated separately for conceptual purposes.

[0066] In some embodiments, the intra BC unit 48 may generate a vector and obtain a prediction block in a manner similar to that described above in conjunction with the motion estimation unit 42 and the motion compensation unit 44, but wherein the prediction block is in the same frame as the current block being encoded, and wherein the vector is referred to as a block vector relative to the motion vector. In some embodiments, the intra BC unit 48 may determine an intra prediction mode for encoding the current block. In some examples, the intra BC unit 48 may encode the current block using various intra prediction modes, for example, during separate encoding passes, and test its performance through rate-distortion analysis. Next, the intra BC unit 48 may select an appropriate intra prediction mode to use among the various tested intra prediction modes and generate an intra mode indicator accordingly. For example, the intra BC unit 48 may calculate rate-distortion values ​​for various tested intra prediction modes using rate-distortion analysis, and select an intra prediction mode with the best rate-distortion characteristic among the tested modes as an appropriate intra prediction mode to use. The rate-distortion analysis generally determines the amount of distortion (or error) between a coded block and the original uncoded block that was coded to generate the coded block, and the bit rate (i.e., the number of bits) used to produce the coded block. Intra BC unit 48 may calculate a ratio based on the distortion and rate for each coded block to determine which intra prediction mode exhibits the best rate-distortion value for the block.

[0067] In some examples, intra BC unit 48 may use motion estimation unit 42 and motion compensation unit 44 in whole or in part to perform such functions for intra BC prediction in accordance with embodiments described herein. In either case, for intra block copying, the prediction block may be a block that is considered to closely match the block to be encoded in terms of pixel differences, which may be determined by SAD, SSD, or other difference metrics, and identification of the prediction block may include calculating values ​​for sub-integer pixel positions.

[0068] Regardless of whether the prediction block is from the same frame according to intra-frame prediction or from a different frame according to inter-frame prediction, the video encoder 20 can form a residual video block by subtracting the pixel values ​​of the prediction block from the pixel values ​​of the current video block being encoded to form pixel difference values. The pixel difference values ​​forming the residual video block may include luminance component differences and chrominance component differences.

[0069] As described above, the intra-prediction processing unit 46 may perform intra-prediction on the current video block as an alternative or supplement to the inter-prediction performed by the motion estimation unit 42 and the motion compensation unit 44, or the intra-block copy prediction performed by the intra BC unit 48. In some embodiments, the intra-prediction processing unit 46 may determine an intra-prediction mode for encoding the current block. To this end, the intra-prediction processing unit 46 may encode the current block using various intra-prediction modes, for example, during a separate encoding pass, and the intra-prediction processing unit 46 (or a mode selection unit in some examples) may select an appropriate intra-prediction mode from the tested intra-prediction modes to use. The intra-prediction processing unit 46 may provide information indicating the selected intra-prediction mode of the block to the entropy encoding unit 56. The entropy encoding unit 56 may encode the information indicating the selected intra-prediction mode into the bitstream.

[0070] After prediction processing unit 41 determines a prediction block for the current video block via inter-prediction or intra-prediction, adder 50 forms a residual video block by subtracting the prediction block from the current video block. The residual video data in the residual block may be included in one or more TUs and provided to transform processing unit 52. Transform processing unit 52 transforms the residual video data into residual transform coefficients using a transform such as a discrete cosine transform (DCT) or a conceptually similar transform.

[0071] Transform processing unit 52 may send the resulting transform coefficients to quantization unit 54. Quantization unit 54 quantizes the transform coefficients to further reduce the bit rate. The quantization process may also reduce the bit depth associated with some or all of the coefficients. The degree of quantization may be modified by adjusting a quantization parameter. In some examples, quantization unit 54 may then perform a scan of the matrix including the quantized transform coefficients. Alternatively, entropy encoding unit 56 may perform the scan.

[0072] After quantization, entropy coding unit 56 entropy encodes the quantized transform coefficients into a video bitstream using, for example, context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), syntax-based context adaptive binary arithmetic coding (SBAC), probability interval partitioned entropy (PIPE) coding, or other entropy coding methods or techniques. The encoded bitstream may then be transmitted to a video bitstream such as Figure 1 The video decoder 30 shown, or archive it to Figure 1 The data is stored in the storage device 32 for later transmission to the video decoder 30 or retrieval by the video decoder. The entropy coding unit 56 may also entropy encode the motion vectors and other syntax elements of the current video frame being encoded.

[0073] Inverse quantization unit 58 and inverse transform processing unit 60 apply inverse quantization and inverse transform, respectively, to reconstruct the residual video block in the pixel domain to generate a reference block for predicting other video blocks. As described above, motion compensation unit 44 may generate a motion compensated prediction block from one or more reference blocks of a frame stored in DPB 64. Motion compensation unit 44 may also apply one or more interpolation filters to the prediction block to calculate sub-integer pixel values ​​for use in motion estimation.

[0074] Adder 62 adds the reconstructed residual block to the motion compensated prediction block produced by motion compensation unit 44 to produce a reference block for storage in DPB 64. The reference block may then be used as a prediction block by intra BC unit 48, motion estimation unit 42, and motion compensation unit 44 to inter-predict another video block in a subsequent video frame.

[0075] Figure 3 1 is a block diagram illustrating an example video decoder 30 according to some embodiments of the present application. The video decoder 30 includes a video data memory 79, an entropy decoding unit 80, a prediction processing unit 81, an inverse quantization unit 86, an inverse transform processing unit 88, an adder 90, and a DPB 92. The prediction processing unit 81 further includes a motion compensation unit 82, an intra-frame prediction unit 84, and an intra-frame BC unit 85. The video decoder 30 may perform the above-mentioned Figure 2 The decoding process is the reverse of the encoding process described with respect to video encoder 20. For example, motion compensation unit 82 may generate prediction data based on motion vectors received from entropy decoding unit 80, and intra-prediction unit 84 may generate prediction data based on intra-prediction mode indicators received from entropy decoding unit 80.

[0076] In some examples, units of the video decoder 30 may be assigned to perform embodiments of the present application. Likewise, in some examples, embodiments of the present disclosure may be divided between one or more units of the video decoder 30. For example, the intra BC unit 85 may perform embodiments of the present application alone or in combination with other units of the video decoder 30 (such as the motion compensation unit 82, the intra prediction unit 84, and the entropy decoding unit 80). In some examples, the video decoder 30 may not include the intra BC unit 85, and the functions of the intra BC unit 85 may be performed by other components of the prediction processing unit 81 (such as the motion compensation unit 82).

[0077] The video data memory 79 may store video data to be decoded by other components of the video decoder 30, such as an encoded video bitstream. For example, the video data stored in the video data memory 79 may be obtained from a storage device 32, a local video source (such as a camera) via a wired or wireless network transmission of the video data or by accessing a physical data storage medium (e.g., a flash drive or hard disk). The video data memory 79 may include a coded picture buffer (CPB) that stores encoded video data from an encoded video bitstream. The DPB 92 of the video decoder 30 stores reference video data for use in decoding the video data by the video decoder 30 (e.g., in an intra-frame prediction coding mode or an inter-frame prediction coding mode). The video data memory 79 and the DPB 92 may be formed by any of a variety of memory devices, such as a dynamic random access memory (DRAM), including synchronous DRAM (SDRAM), magnetoresistive RAM (MRAM), resistive RAM (RRAM), or other types of memory devices. For illustrative purposes, the ... Figure 3 9 depicts the video data memory 79 and the DPB 92 as two different components of the video decoder 30. However, it will be apparent to those skilled in the art that the video data memory 79 and the DPB 92 can be provided by the same memory device or a separate memory device. In some examples, the video data memory 79 can be on-chip with other components of the video decoder 30, or off-chip relative to those components.

[0078] During the decoding process, the video decoder 30 receives an encoded video bitstream representing a video block of an encoded video frame and associated syntax elements. The video decoder 30 may receive syntax elements at the video frame level and / or the video block level. The entropy decoding unit 80 of the video decoder 30 entropy decodes the bitstream to generate quantized coefficients, motion vectors or intra-frame prediction mode indicators, and other syntax elements. The entropy decoding unit 80 then forwards the motion vectors or intra-frame prediction mode indicators and other syntax elements to the prediction processing unit 81.

[0079] When a video frame is encoded as an intra-frame prediction coded (I) frame or for intra-frame coded prediction blocks in other types of frames, the intra-frame prediction unit 84 of the prediction processing unit 81 can generate prediction data for a video block of the current video frame based on the signaled intra-frame prediction mode and reference data from a previously decoded block of the current frame.

[0080] When the video frame is encoded as an inter-frame prediction coded (i.e., B or P) frame, the motion compensation unit 82 of the prediction processing unit 81 generates one or more prediction blocks of the video block of the current video frame based on the motion vectors and other syntax elements received from the entropy decoding unit 80. Each prediction block can be generated from a reference frame in one of the reference frame lists. The video decoder 30 can construct the reference frame lists: List 0 and List 1 based on the reference frames stored in the DPB 92 using a default construction technique.

[0081] In some examples, when a video block is encoded according to the intra BC mode described herein, intra BC unit 85 of prediction processing unit 81 generates a prediction block for the current video block based on the block vector and other syntax elements received from entropy decoding unit 80. The prediction block may be within a reconstructed region of the same picture as the current video block defined by video encoder 20.

[0082] The motion compensation unit 82 and / or the intra BC unit 85 determine the prediction information of the video block of the current video frame by parsing the motion vector and other syntax elements, and then use the prediction information to generate the prediction block of the decoded current video block. For example, the motion compensation unit 82 uses some of the received syntax elements to determine the prediction mode (e.g., intra prediction or inter prediction) used to encode the video block of the video frame, the inter prediction frame type (e.g., B or P), the construction information of one or more reference frame lists in the reference frame list of the frame, the motion vector of each inter prediction encoded video block of the frame, the inter prediction state of each inter prediction encoded video block of the frame, and other information for decoding the video block in the current video frame.

[0083] Similarly, the intra BC unit 85 may use some of the received syntax elements (e.g., flags) to determine that the current video block is predicted using the intra BC mode, construction information that video blocks of the frame are within the reconstructed region and should be stored in the DPB 92, block vectors for each intra BC predicted video block of the frame, intra BC prediction status for each intra BC predicted video block of the frame, and other information for decoding the video blocks in the current video frame.

[0084] Motion compensation unit 82 may also perform interpolation using interpolation filters to calculate interpolated values ​​for sub-integer pixels of a reference block, as used by video encoder 20 during encoding of the video block. In this case, motion compensation unit 82 may determine the interpolation filters used by video encoder 20 from the received syntax elements and use the interpolation filters to produce a prediction block.

[0085] Inverse quantization unit 86 inverse quantizes the quantized transform coefficients provided in the bitstream and entropy decoded by entropy decoding unit 80, using the same quantization parameters calculated by video encoder 20 for each video block in the video frame to determine the degree of quantization. Inverse transform processing unit 88 applies an inverse transform (e.g., an inverse DCT, an inverse integer transform, or a conceptually similar inverse transform process) to the transform coefficients to reconstruct the residual block in the pixel domain.

[0086] After the motion compensation unit 82 or the intra BC unit 85 generates a prediction block for the current video block based on the vector and other syntax elements, the adder 90 reconstructs the decoded video block of the current video block by summing the residual block from the inverse transform processing unit 88 and the corresponding prediction block generated by the motion compensation unit 82 and the intra BC unit 85. An in-loop filter 91 (such as a deblocking filter, an SAO filter, a CCSAO filter, and / or an ALF) can be positioned between the adder 90 and the DPB 92 to further process the decoded video block. In some examples, the in-loop filter 91 can be omitted and the decoded video block can be provided directly to the DPB 92 by the adder 90. The decoded video blocks in a given frame are then stored in the DPB 92, which stores reference frames for subsequent motion compensation of the next video block. The DPB 92 or a memory device separate from the DPB 92 can also store the decoded video for later presentation on a display such as a Figure 1 On a display device such as a display device 34.

[0087] In a typical video encoding and decoding process, a video sequence typically includes an ordered set of frames or pictures. Each frame may include three sample arrays, denoted as SL, SCb, and SCr. SL is a two-dimensional array of luma samples. SCb is a two-dimensional array of Cb chroma samples. SCr is a two-dimensional array of Cr chroma samples. In other examples, a frame may be monochrome and therefore include only a two-dimensional array of luma samples.

[0088] like Figure 4AAs shown, the video encoder 20 (or more specifically, the partition unit 45) generates an encoded representation of a frame by first partitioning the frame into a set of CTUs. A video frame may include an integer number of CTUs sequentially ordered from left to right and from top to bottom in a raster scan order. Each CTU is the largest logical coding unit, and the width and height of the CTU are signaled by the video encoder 20 in a sequence parameter set so that all CTUs in a video sequence have the same size, namely one of 128×128, 64×64, 32×32, and 16×16. However, it should be noted that the present application is not necessarily limited to a specific size. Figure 4B As shown, each CTU may include one CTB for luma samples, two corresponding coding tree blocks for chroma samples, and syntax elements for encoding and decoding samples of the coding tree blocks. The syntax elements describe the properties of different types of units of pixel coding blocks and how the video sequence can be reconstructed at the video decoder 30, including inter-frame prediction or intra-frame prediction, intra-frame prediction mode, motion vectors, and other parameters. In a monochrome picture or a picture with three separate color planes, a CTU may include a single coding tree block and syntax elements for encoding and decoding samples of the coding tree block. The coding tree block may be an N×N sample block.

[0089] To achieve better performance, the video encoder 20 may recursively perform tree partitioning (such as binary tree partitioning, ternary tree partitioning, quadtree partitioning, or a combination thereof) on the coding tree block of the CTU and divide the CTU into smaller CUs. Figure 4C As shown, a 64×64 CTU 400 is first divided into four smaller CUs, each with a block size of 32×32. Among the four smaller CUs, CU 410 and CU 420 are each divided into four 16×16 CUs by block size. Two 16×16 CUs 430 and 440 are each further divided into four 8×8 CUs by block size. Figure 4D Depicted diagram Figure 4C The quadtree data structure of the final result of the partitioning process of the CTU 400 depicted in FIG. 4 is shown in FIG. 4 , with each leaf node of the quadtree corresponding to a CU with a corresponding size ranging from 32×32 to 8×8. Figure 4B For a CTU depicted, each CU may include a CB of luma samples and two corresponding coding blocks of chroma samples of the same size frame, and syntax elements for encoding and decoding the samples of the coding blocks. In a monochrome picture or a picture with three separate color planes, a CU may include a single coding block and syntax structures for encoding and decoding the samples of the coding block. It should be noted that Figure 4C and Figure 4DThe quadtree partitioning depicted in FIG is for illustration purposes only, and a CTU can be divided into multiple CUs to accommodate different local characteristics based on quadtree / ternary tree / binary tree partitioning. In the multi-type tree structure, a CTU is partitioned according to a quadtree structure, and each quadtree leaf CU can be further partitioned according to a binary tree structure and a ternary tree structure. Figure 4E As shown, there are five possible partition types for a coding block with width W and height H, namely, quadrifugal partition, horizontal binary partition, vertical binary partition, horizontal trifugal partition and vertical trifugal partition.

[0090] In some embodiments, the video encoder 20 may further partition the coding block of the CU into one or more M×N PBs. A PB is a rectangular (square or non-square) sample block to which the same prediction (inter or intra) is applied. The PU of a CU may include a PB for luma samples, two corresponding PBs for chroma samples, and syntax elements for predicting the PBs. In a monochrome picture or a picture with three separate color planes, a PU may include a single PB and a syntax structure for predicting the PB. The video encoder 20 may generate predicted luma, Cb, and Cr blocks for the luma, Cb, and Cr PBs of each PU of the CU.

[0091] Video encoder 20 may use intra prediction or inter prediction to generate a prediction block for a PU. If video encoder 20 uses intra prediction to generate a prediction block for a PU, video encoder 20 may generate the prediction block for the PU based on decoded samples of a frame associated with the PU. If video encoder 20 uses inter prediction to generate a prediction block for a PU, video encoder 20 may generate the prediction block for the PU based on decoded samples of one or more frames other than the frame associated with the PU.

[0092] After the video encoder 20 generates the predicted luma, Cb, and Cr blocks of one or more PUs of a CU, the video encoder 20 may generate a luma residual block of the CU by subtracting the predicted luma block of the CU from its original luma coding block, so that each sample in the luma residual block of the CU indicates the difference between a luma sample in one of the predicted luma blocks of the CU and a corresponding sample in the original luma coding block of the CU. Similarly, the video encoder 20 may generate a Cb residual block and a Cr residual block of the CU, respectively, so that each sample in the Cb residual block of the CU indicates the difference between a Cb sample in one of the predicted Cb blocks of the CU and a corresponding sample in the original Cb coding block of the CU, and each sample in the Cr residual block of the CU may indicate the difference between a Cr sample in one of the predicted Cr blocks of the CU and a corresponding sample in the original Cr coding block of the CU.

[0093] In addition, if Figure 4CAs illustrated, the video encoder 20 may use quadtree partitioning to decompose the luma, Cb, and Cr residual blocks of a CU into one or more luma, Cb, and Cr transform blocks, respectively. A transform block is a rectangular (square or non-square) block of samples to which the same transform is applied. A TU of a CU may include a transform block of luma samples, two corresponding transform blocks of chroma samples, and syntax elements for transforming transform block samples. Thus, each TU of a CU may be associated with a luma transform block, a Cb transform block, and a Cr transform block. In some examples, the luma transform block associated with a TU may be a sub-block of the luma residual block of the CU. The Cb transform block may be a sub-block of the Cb residual block of the CU. The Cr transform block may be a sub-block of the Cr residual block of the CU. In a monochrome picture or a picture with three separate color planes, a TU may include a single transform block and a syntax structure for transforming samples of a transform block.

[0094] The video encoder 20 may apply one or more transforms to the luma transform block of the TU to generate a luma coefficient block of the TU. The coefficient block may be a two-dimensional array of transform coefficients. The transform coefficient may be a scalar. The video encoder 20 may apply one or more transforms to the Cb transform block of the TU to generate a Cb coefficient block of the TU. The video encoder 20 may apply one or more transforms to the Cr transform block of the TU to generate a Cr coefficient block of the TU.

[0095] After generating a coefficient block (e.g., a luma coefficient block, a Cb coefficient block, or a Cr coefficient block), the video encoder 20 may quantize the coefficient block. Quantization generally refers to the process of quantizing transform coefficients to possibly reduce the amount of data used to represent the transform coefficients, thereby providing further compression. After the video encoder 20 quantizes the coefficient block, the video encoder 20 may entropy encode the syntax elements indicating the quantized transform coefficients. For example, the video encoder 20 may perform CABAC on the syntax elements indicating the quantized transform coefficients. Ultimately, the video encoder 20 may output a bitstream including a sequence of bits that form a representation of a coded frame and associated data, and the bitstream is stored in the storage device 32 or transmitted to the target device 14.

[0096] After receiving the bitstream generated by the video encoder 20, the video decoder 30 can parse the bitstream to obtain syntax elements from the bitstream. The video decoder 30 can reconstruct a frame of video data based at least in part on the syntax elements obtained from the bitstream. The process of reconstructing video data is generally the opposite of the encoding process performed by the video encoder 20. For example, the video decoder 30 can perform an inverse transform on a coefficient block associated with a TU of the current CU to reconstruct a residual block associated with the TU of the current CU. The video decoder 30 also reconstructs the coding block of the current CU by adding the samples of the prediction block of the PU of the current CU to the corresponding samples of the transform block of the TU of the current CU. After reconstructing the coding block of each CU of the frame, the video decoder 30 can reconstruct the frame.

[0097] As mentioned above, video codecs mainly use two modes, i.e., intra-frame prediction (or intra-prediction) and inter-frame prediction (or inter-prediction) to achieve video compression. It should be noted that IBC can be considered as intra-frame prediction or a third mode. Between these two modes, inter-frame prediction contributes more to codec efficiency than intra-frame prediction because it uses motion vectors to predict the current video block from a reference video block.

[0098] However, with the continuous improvement of video data capture technology and finer video block sizes for retaining details in video data, the amount of data required to represent the motion vector of the current frame has also increased significantly. One way to overcome this challenge is to benefit from the fact that not only a group of neighboring CUs in the spatial and temporal domains have similar video data for prediction purposes, but also the motion vectors between these neighboring CUs are similar. Therefore, by exploring the spatial and temporal correlation of CUs, the motion information of spatially neighboring CUs and / or temporally co-located CUs can be used as an approximation of the motion information (e.g., motion vector) of the current CU, which is also called the "motion vector predictor" (MVP) of the current CU.

[0099] Instead, it will be combined as above Figure 2 The actual motion vector of the current CU determined by the motion estimation unit 42 is encoded into the video bitstream, and the motion vector prediction value of the current CU is subtracted from the actual motion vector of the current CU to generate the motion vector difference (MVD) of the current CU. In this way, the motion vector determined by the motion estimation unit 42 for each CU of the frame does not need to be encoded into the video bitstream, and the amount of data used to represent the motion information in the video bitstream can be significantly reduced.

[0100] Like the process of selecting a prediction block in a reference frame during inter-frame prediction of a coding block, both the video encoder 20 and the video decoder 30 need to adopt a set of rules to construct a motion vector candidate list (also called a "merge list") of the current CU using those potential candidate motion vectors associated with the spatially neighboring CUs and / or the temporally co-located CUs of the current CU, and then select a member from the motion vector candidate list as the motion vector prediction value of the current CU. In doing so, the motion vector candidate list itself does not need to be transmitted from the video encoder 20 to the video decoder 30, and the index of the selected motion vector prediction value within the motion vector candidate list is sufficient for the video encoder 20 and the video decoder 30 to use the same motion vector prediction value within the motion vector candidate list to encode and decode the current CU.

[0101] The main problem to be solved in this application is to further improve the coding efficiency of the cross-component prediction codec tool (i.e., cross-component linear model (CCLM)) used in ECM. This application briefly reviews some related codec tools in ECM. Subsequently, some deficiencies in the existing CCLM design are discussed. Finally, a solution to improve the existing CCLM prediction design is proposed.

[0102] Cross-component linear model prediction

[0103] To reduce cross-component redundancy, the cross-component linear model (CCLM) prediction mode is used in VVC, where the chroma samples are predicted based on the reconstructed luma samples of the same CU using the following linear model:

[0104] pred C (i,j)=α·rec L ′(i,j)+β(1)

[0105] Among them, pred C (i, j) represents the predicted chroma sample in the CU, and rec L (i, j) represents the downsampled reconstructed luminance sample of the same CU.

[0106] The CCLM parameters (α and β) are derived from up to four neighboring chroma samples and their corresponding downsampled luma samples. Assuming the current chroma block size is W×H, W' and H' are set to:

[0107] When the LM mode is applied, W'=W, H'=H;

[0108] When the LM-A mode is applied, W'=W+H;

[0109] When the LM-L mode is applied, H'=H+W;

[0110] The upper neighboring position is denoted as S[0,-1]…S[W'-1,-1], and the left neighboring position is denoted as S[-1,0]…S[-1,H'-1]. Then the four sample points are selected as:

[0111] When LM mode is applied and both upper and left neighboring samples are available, S[W' / 4,-1], S[3*W' / 4,-1], S[-1,H' / 4], S[-1,3*H' / 4];

[0112] When LM-A mode is applied or only upper neighboring samples are available, S[W' / 8,-1], S[3*W' / 8,-1], S[5*W' / 8,-1], S[7*W' / 8,-1];

[0113] When LM-L mode is applied or only left neighbor samples are available, S[-1,H' / 8], S[-1,3*H' / 8], S[-1,5*H' / 8], S[-1,7*H' / 8];

[0114] These four adjacent brightness samples at the selected position are downsampled and compared four times to find the two larger values: x 0 A and x 1 A , and two smaller values: x 0 B and x 1 B Their corresponding chrominance sample values ​​are represented by y 0 A ,y 1 A ,y 0 B and 1 B Then x A 、x B ,y A and B is derived as:

[0115] X a =(x 0 A +x 1 A +1)>>1;X b =(x 0 B +x 1 B +1)>>1

[0116] Y a =(y 0 A +y 1A +1)>>1;Y b =(y 0 B +y 1 B +1)>>1(2)

[0117] Finally, the linear model parameters α and β are obtained according to the following equations.

[0118]

[0119] β=Y b -α·X b (4)

[0120] Figure 5 An example of the positions of left and upper samples involved in the CCLM mode and samples of the current block is shown.

[0121] The division operation used to calculate the parameter α is implemented using a lookup table. To reduce the memory required to store the table, the diff value (the difference between the maximum and minimum values) and the parameter α are expressed in exponential notation. For example, diff is approximated using a 4-bit significant part and an exponent. Therefore, the table for 1 / diff is reduced to 16 elements for representing 16 significant values, as shown below:

[0122] DivTable[]={0,7,6,5,5,4,4,3,3,2,2,1,1,1,1,0}(5)

[0123] This has the benefit of reducing both the complexity of the computation and the amount of memory required to store the required tables.

[0124] In addition to being used together to calculate linear model coefficients, the upper template and the left template may also be used alternatively or additionally in the other two LM modes (referred to as LM_A and LM_L modes).

[0125] In LM_A (also called LM_T) mode, only the upper template is used to calculate the linear model coefficients. To get more samples, the upper template is expanded to (W+H) samples. In LM_L mode, only the left template is used to calculate the linear model coefficients. To get more samples, the left template is expanded to (H+W) samples.

[0126] In LM_LT mode, both the left template and the upper template are used to calculate the linear model coefficients.

[0127] To match the chroma sample positions of a 4:2:0 video sequence, two types of downsampling filters are applied to the luma samples to achieve a 2 to 1 downsampling ratio in both horizontal and vertical directions. The choice of downsampling filter is specified by the SPS level flag. The two downsampling filters are as follows, corresponding to the contents of "type-0" and "type-2", respectively.

[0128]

[0129] Note that when the upper reference line is at a CTU boundary, only one luma line (common line buffer in intra prediction) is used to derive the downsampled luma samples.

[0130] This parameter calculation is performed as part of the decoding process, not just as an encoder search operation. Therefore, no syntax is used to convey the values ​​of α and β to the decoder.

[0131] For chroma intra mode encoding and decoding, a total of 8 intra modes are allowed for chroma intra mode encoding and decoding. These modes include five traditional intra modes and three cross-component linear model modes (CCLM, LM_A, and LM_L). The chroma mode signal transmission and derivation process are shown in Table 1. Chroma mode encoding and decoding directly depends on the intra prediction mode of the corresponding luminance block. Since separate block partition structures are enabled for luminance components and chrominance components in the I strip, a chroma block may correspond to multiple luminance blocks. Therefore, for the chroma DM mode, the intra prediction mode of the corresponding luminance block covering the center position of the current chroma block will be directly inherited.

[0132] Table 1 Derivation of chrominance prediction mode from luma mode when CCLM is enabled

[0133]

[0134]

[0135] Regardless of the value of sps_cclm_enabled_flag, a single binarization table is used, as shown in Table 2.

[0136] Table 2 Unified binarization table of chroma prediction mode

[0137] Value of intra_chroma_pred_mode Binary bit string 4 00 0 0100 1 0101 2 0110 3 0111 5 10 6 110 7 111

[0138] In Table 2, the first binary bit indicates whether it is normal mode (0) or LM mode (1). If it is LM mode, the next binary bit indicates whether it is LM_CHROMA (0). If it is not LM_CHROMA, the next 1 binary bit indicates whether it is LM_L (0) or LM_A (1). For this case, when sps_cclm_enabled_flag is 0, the first binary bit of the binarization table corresponding to intra_chroma_pred_mode can be discarded before entropy coding and decoding. Or in other words, the first binary bit is inferred to be 0 and therefore not coded and decoded. For the cases where sps_cclm_enabled_flag is equal to 0 and 1, this single binarization table is used. The first two binary bits in Table 2 use their own context model for context coding and decoding, and the remaining binary bits use bypass coding and decoding.

[0139] In addition, to reduce the luma-chroma delay in the dual tree, when the 64×64 luma coding tree node is partitioned using Not Split (and the 64×64 CU is not partitioned using intra sub-partitioning (ISP)) or QT, the chroma CU in the 32×32 / 32×16 chroma coding tree node can use CCLM as follows:

[0140] If a 32×32 chroma node is an unpartitioned or partitioned QT split, all chroma CUs within the 32×32 node can use CCLM.

[0141] If a 32×32 chroma node is partitioned using horizontal BT, and the 32×16 child node is not partitioned or uses vertical BT partitioning, all chroma CUs within the 32×16 chroma node can use CCLM.

[0142] Under all other luma and chroma coding tree split conditions, CCLM is not allowed for chroma CUs.

[0143] During the development of the ECM, the simplified derivation (min-max approximation) of α and β was removed. Instead, the model parameters α and β are derived using a linear least squares solution between the causal reconstructed data of the downsampled luma samples and the causal chroma samples.

[0144]

[0145] Among them, Rec C (i) and Rec' L (i) indicates the reconstructed chroma samples and downsampled luma samples around the target block, and I indicates the total number of samples of neighboring data.

[0146] The LM_A and LM_L models are also called multidirectional linear models (MDLM).

[0147] Fig. 6A An example of the MDLM in action is shown when the block content cannot be predicted from the L-shaped reconstructed region.

[0148] Figure 6B An example is shown where MDLM_L uses only the left-side reconstruction samples to derive CCLM parameters.

[0149] Figure 6C An example is shown where MDLM_T uses only the top reconstruction samples to derive CCLM parameters.

[0150] Integerization

[0151] After the initial integer design of the least mean square (LMS) CCLM was proposed, the method was simplified and improved to reduce the accuracy of α to n. α The maximum bit width of the multiplier was reduced from 13 to 7, and the division LUT entries were reduced from 64 to 32, resulting in the ECM LMS version.

[0152] Basic Algorithm

[0153] In some embodiments, a linear relationship is used to model the correlation of the luminance signal and the chrominance signal. The chrominance value is predicted based on the reconstructed luminance value of the co-located block as shown below.

[0154] Pred C [x,y]=α·Rec L [x,y]+β (6)

[0155] Among them, Pred C Indicates the prediction of the chroma samples in the block, and Rec L Indicates the reconstructed luma samples in the block. Parameters α and β are derived from the causal reconstructed samples around the current block.

[0156] In YUV420 sampling, the sampling rates of luma and chroma components are different. The sampling rate of chroma components is half of luma components and has a phase difference of 0.5 pixels in the vertical direction. Reconstructing luma requires downsampling in the vertical direction and subsampling in the horizontal direction to match the size of chroma signals, as shown below.

[0157] Rec L '[x,y]=(Rec L [2x,2y]+Rec L [2x,2y+1])>>1 (7)

[0158] In this paper, the model parameters α and β are derived using a linear least squares solution between the causal reconstructed data of the downsampled luminance and chrominance components.

[0159]

[0160] Integer Implementation

[0161] Floating point operations are required to calculate the linear model parameter α in equation (8) to maintain high data accuracy. When α is represented by a floating point value, floating point multiplication is involved in equation (6). This section designs an integer implementation of the algorithm.

[0162] In some embodiments, the fractional part of parameter a is expressed as n α The value of parameter a is represented by the enlarged and rounded integer value α', and a'=a×(1<<n α ). Therefore, the linear model of equation (1) becomes

[0163] pred C [x,y]=(α'·Rec L '[x,y]>>n α )+β' (10)

[0164] Where β' is the rounded value of the floating point number β, and α' can be calculated as follows.

[0165]

[0166] This paper proposes to replace the division operation in equation (11) with table lookup and multiplication. First, A2 is descaled to reduce the table size. A1 is also descaled to avoid product overflow.

[0167] Then, in A2, only the The value of A2' is defined as the most significant bit, while the other bits are set to zero. The approximate value A2' can be calculated as

[0168]

[0169] where [...] represents a rounding operation, and It can be calculated as

[0170]

[0171] Here, bdepth(A2) represents the bit depth of value A2.

[0172] Then do the same for A1.

[0173]

[0174] Considering the quantized representation of A1 and A2, formula (11) can be rewritten as follows.

[0175]

[0176] in, Expressed as length to avoid division.

[0177] During simulation, the constant parameters are set as:

[0178] n α Equal to 13, this value is a trade-off between data accuracy and computational cost.

[0179] Equal to 6, resulting in a lookup table size of 64. When bdepth(A2)<6 (for example, A2<32), the table size can be further reduced to 32 by enlarging A2.

[0180] n tabl e Equal to 15, resulting in a 16-bit data representation of the table elements.

[0181] Set to 15 to avoid product overflow and maintain 16-bit multiplication.

[0182] Finally, α' is clipped to [-2 -15 ,2 15 -1], retaining 16-bit multiplication in equation (5). By this clipping, when n α When it is equal to 13, the actual a value is limited to [-4,4), which helps prevent error amplification.

[0183] With the calculated parameter α', the parameter β' is calculated as follows.

[0184]

[0185] The division in the above equation can be simply replaced by a shift since the value I is a power of 2.

[0186] Simplified parameter calculation

[0187] Introduction

[0188] In HM6.0, an intra prediction mode called LM is applied to predict the chroma PU based on a linear model using the reconstruction of the co-located luma PU. The parameters of the linear model consist of the slope (a>>k) and the y intercept (b), which are derived from the neighboring luma and chroma pixels using the least mean square solution. The value of the prediction sample predSamples[x,y] (where x,y=0…nS-1, where nS specifies the block size of the current chroma PU) is derived as follows:

[0189] predSamples[x,y]=Clip1C (((p Y '[x,y]*a)>>k)+b)(12)

[0190] Among them, x, y = 0…nS-1.

[0191] Among them, P Y '[x,y] is the pixel reconstructed from the corresponding brightness component. When the coordinates x and y are greater than or equal to 0, P Y ' is the pixel reconstructed from the same brightness PU. When x or y is less than 0, P Y ' is the reconstructed neighboring pixel of the same brightness PU.

[0192] Some intermediate variables L, C, LL, LC, k2 and k3 in the derivation process are derived as follows:

[0193]

[0194] k2=Log2((2*nS)>>k3)(17)

[0195] k3=Max(0,BitDepth C +Log2(nS)-14)(18)

[0196] Therefore, the variables a, b, and k can be derived as:

[0197] a1=(LC< <k2)–L*C(19)

[0198] a2=(LL< <k2)–L*L(20)

[0199] k1=Max(0,Log2(abs(a2))-5)–Max(0,Log2(abs(a1))-14)+2(21)

[0200] a1s=a1>>Max(0,Log2(abs(a1))-14)(22)

[0201] a2s=abs(a2>>Max(0,Log2(abs(a2))-5))(23)

[0202] a3=a2s<1?0:Clip3(-2 15 ,2 15 -1,a1s*lmDiv+(1<<(k1-1))>>k1)(24)

[0203] a=a3>>Max(0,Log2(abs(a3))-6)(25)

[0204] k=13–Max(0,Log2(abs(a))-6)(26)

[0205] b=(L–((a*C)>>k1)+(1<<(k2-1)))>>k2(27)

[0206] where lmDiv is specified in a 63-entry lookup table (i.e., Table 3) and is generated by:

[0207] lmDiv(a2s)=((1<<15)+a2s / 2) / a2s(28)

[0208] Table 3lmDiv Specification

[0209]

[0210]

[0211] In equation (24), a1s is a 16-bit signed integer and lmDiv is a 16-bit unsigned integer. Therefore, a 16-bit multiplier and 16-bit storage are required. In this paper, we propose to reduce the bit depth of the multiplier to the internal bit depth and reduce the size of the lookup table.

[0212] Reduce the bit depth of the multiplier

[0213] The bit depth of a1s is reduced to the internal bit depth by changing equation (22) to the following:

[0214] a1s=a1>>Max(0,Log2(abs(a1))–(BitDepth C –2))(29)

[0215] The lmDiv value with internal bit depth is implemented as follows:

[0216] lmDiv(a2s)=((1<<(BitDepth C -1))+a2s / 2) / a2s(30)

[0217] Table 4 shows an example where the internal bit depth is 10.

[0218] Table 4 lmDiv specifications for internal bit depth 10

[0219] a2s 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 lmDiv 512 256 171 128 102 85 73 64 57 51 47 43 39 37 34 32 a2s 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 lmDiv 30 28 27 26 24 23 22 21 20 20 19 18 18 17 17 16 a2s 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 lmDiv 16 15 15 14 14 13 13 13 12 12 12 12 11 11 11 11 a2s 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 lmDiv 10 10 10 10 10 9 9 9 9 9 9 9 8 8 8

[0220] The following modifications are also made to equation (21) and equation (26):

[0221] k1=Max(0,Log2(abs(a2))-5)–Max(0,Log2(abs(a1))–(BitDepth C –2))(31)

[0222] k = BitDepth C –1–Max(0,Log2(abs(a))-6)(32)

[0223] Reduce the number of entries in the lookup table

[0224] In some embodiments, the number of entries is reduced from 63 to 32, and the bits per entry is reduced from 16 to 10, as shown in Table 3. This can save almost 70% of memory. The corresponding changes to equation (24), equation (28), and equation (26) are as follows.

[0225] a3=a2s<32?0:Clip3(-2 15 ,2 15 -1,a1s*lmDiv+(1<<(k1-1))>>k1)(33)

[0226] lmDiv(a2s)=((1<<(BitDepth C +4))+a2s / 2) / a2s(34)

[0227] k = BitDepth C +4–Max(0,Log2(abs(a))-6)(35)

[0228] Table 5 lmDiv specifications for internal bit depth 10

[0229] a2s 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 lmDiv 512 496 482 468 455 443 431 420 410 400 390 381 372 364 356 349 a2s 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 lmDiv 341 334 328 321 315 309 303 298 293 287 282 278 273 269 264 260

[0230] Multi-model linear model prediction

[0231] In ECM-1.0, a multi-model LM (MMLM) prediction mode is proposed, in which the chroma samples are predicted based on the reconstructed luminance samples of the same CU by using the following two linear models:

[0232]

[0233] Among them, pred C (i,j),pred C (i, j) represents the predicted chroma sample in the CU, and rec L ′(i,j) represents the downsampled reconstructed luma sample of the same CU. The threshold is calculated as the average of the neighboring reconstructed luma samples. Figure 7An example of dividing neighboring samples into two groups based on the value of the threshold is shown. For each group, the parameters αi and βi (where i is equal to 1 and 2 respectively) are derived based on the linear relationship between the luminance and chrominance values ​​of the two samples, which are the minimum luminance samples A(X) in the group. A ,Y A ) and the maximum brightness sample B(X B ,Y B ). Here X A , Y A is the x-coordinate value (i.e., brightness value) and y-coordinate value (i.e., chrominance value) of sample point A, and X B , Y B are the x- and y-coordinate values ​​of sample point B.

[0234] The linear model parameters α and β are obtained according to the following equations.

[0235]

[0236] β=y A -αx A

[0237] This method is also called min-max method. The division in the above equation can be avoided and replaced by multiplication and shift.

[0238] For square coding blocks, the above two equations are applied directly. For non-square coding blocks, the neighboring samples of the longer boundary are first subsampled to have the same number of samples as the shorter boundary.

[0239] In addition to the scenario where the upper template and the left template are used together to calculate the linear model coefficients, these two templates can also be used alternatively in two other MMLM modes (referred to as MMLM_A and MMLM_L modes).

[0240] In MMLM_A mode, only pixel samples in the upper template are used to calculate the linear model coefficients. To get more samples, the upper template is expanded to a size of (W+W). In MMLM_L mode, only pixel samples in the left template are used to calculate the linear model coefficients. To get more samples, the left template is expanded to a size of (H+H).

[0241] Note that when the upper reference line is located at a CTU boundary, only one luma line (stored in the intra-predicted line buffer) is used to derive the downsampled luma samples.

[0242] For chroma intra mode encoding and decoding, a total of 11 intra modes are allowed for chroma intra mode encoding and decoding. These modes include five traditional intra modes and six cross-component linear model modes (CCLM, LM_A, LM_L, MMLM, MMLM_A, and MMLM_L). The chroma mode signal transmission and derivation process are shown in Table 6. Chroma mode encoding and decoding directly depends on the intra prediction mode of the corresponding luminance block. Since separate block partition structures are enabled for luminance components and chrominance components in the I strip, a chroma block may correspond to multiple luminance blocks. Therefore, for the chroma DM mode, the intra prediction mode of the corresponding luminance block covering the center position of the current chroma block will be directly inherited.

[0243] Table 6 Derivation of chroma prediction mode from luma mode when MMLM_ is enabled

[0244]

[0245] Adaptively enable LM and MMLM for prediction

[0246] MMLM and LM modes can also be used together in an adaptive manner. For MMLM, the two linear models are as follows:

[0247]

[0248] Among them, pred C (i, j) represents the predicted chroma sample in the CU, and rec L ′(i,j) represents the downsampled reconstructed luma sample of the same CU. The threshold can be simply determined based on the average value of luma and chroma and their minimum and maximum values. Figure 8 An example of dividing neighboring sample points into two groups based on the inflection point T indicated by the arrow is shown. The linear model parameters α1 and β1 are derived based on the linear relationship between the brightness value and the chromaticity value of the two sample points, which are the minimum brightness sample point A(X A ,Y A ) and threshold (X T ,Y T The linear model parameters α2 and β2 are derived from the linear relationship between the brightness and chromaticity values ​​of two sample points, which are the maximum brightness sample points B(X B ,Y B ) and threshold (X T ,Y T ). Here X A , Y A is the x-coordinate value (i.e., brightness value) and y-coordinate value (i.e., chrominance value) of sample point A, and X B , Y B is the x-coordinate value and y-coordinate value of sample point B. The linear model parameter α of each groupi and β i (where i is equal to 1 and 2 respectively) is obtained according to the following equation.

[0249]

[0250] β1=Y A -α1X A

[0251]

[0252] β2=Y T -α2X T

[0253] For square coding blocks, the above equation is applied directly. For non-square coding blocks, the neighboring samples of the longer boundary are first subsampled to have the same number of samples as the shorter boundary.

[0254] In addition to the scenario where the upper template and the left template are used together to determine the linear model coefficients, these two templates can also be used alternatively or additionally in two other MMLM modes (referred to as MMLM_A and MMLM_L modes, respectively).

[0255] In MMLM_A mode, only pixel samples in the upper template are used to calculate the linear model coefficients. To get more samples, the upper template is expanded to a size of (W+W). In MMLM_L mode, only pixel samples in the left template are used to calculate the linear model coefficients. To get more samples, the left template is expanded to a size of (H+H).

[0256] Note that when the upper reference line is located at a CTU boundary, only one luma line (stored in the intra-predicted line buffer) is used to derive the downsampled luma samples.

[0257] For chroma intra mode codec, there is a conditional check to select LM mode (CCLM, LM_A and LM_L) or multi-model LM mode (MMLM, MMLM_A and MMLM_L). The conditional check is as follows:

[0258]

[0259] Among them, BlkSizeThres LM Indicates the minimum block size for LM mode, and BlkSizeThres MM represents the minimum block size of the MMLM mode. The symbol d represents a predetermined threshold. In some embodiments, d can take a value of 0. In some embodiments, d can take a value of 8.

[0260] For chroma intra mode encoding and decoding, a total of 8 intra modes are allowed for chroma intra mode encoding and decoding. These modes include five traditional intra modes and three cross-component linear model modes. The chroma mode signal transmission and derivation process are shown in Table 1. It is worth noting that for a given CU, if it is encoded and decoded in the linear model mode, it is determined whether it is a traditional single model LM mode or an MMLM mode based on the above conditional check. Unlike the case shown in Table 6, there is no need to transmit a separate MMLM mode with a signal. Chroma mode encoding and decoding depends directly on the intra prediction mode of the corresponding luminance block. Since separate block partition structures are enabled for luminance components and chrominance components in the I stripe, one chroma block may correspond to multiple luminance blocks. Therefore, for the chroma DM mode, the intra prediction mode of the corresponding luminance block covering the center position of the current chroma block will be directly inherited.

[0261] Slope adjustment for CCLM

[0262] During the development of the ECM, a slope adjustment for the CCLM was proposed.

[0263] Rationale

[0264] CCLM uses a 2-parameter model to map luminance values ​​to chrominance values. The slope parameter "a" and the bias parameter "b" define the mapping as follows:

[0265] chromaVal=a*lumaVal+b

[0266] It is proposed to signal an adjustment "u" to the slope parameter to update the model to the following form:

[0267] chromaVal=a'*lumaVal+b'

[0268] in,

[0269] a'=a+u

[0270] b'=bu*y r

[0271] With this selection, the mapping function will be centered around the light intensity value y r It is recommended to use the average value of the reference brightness samples used when creating the model as the y r , in order to provide meaningful modifications to the model.

[0272] FIG. 9A to FIG. 9B An example process for performing slope adjustment on a CCLM is illustrated. Fig.9A A model created using the current CCLM is shown. Fig. 9B The model updated as suggested is shown.

[0273] Implementation

[0274] The slope adjustment parameter is provided as an integer between -4 and 4 (inclusive) and is signaled in the bitstream. The units of the slope adjustment parameter are 1 / 8 chroma sample value per luma sample value (for 10-bit content).

[0275] Adjustments apply to CCLM models that use reference samples above and to the left of the block ("LM_CHROMA_IDX" and "MMLM_CHROMA_IDX"), but not to "one-sided" mode. This choice is based on a trade-off between codec efficiency and complexity.

[0276] When slope adjustment is applied to a multi-mode CCLM model, both models may be adjusted, so up to two slope updates may be signaled for a single chroma block.

[0277] Encoder Methods

[0278] The proposed encoder method performs a SATD-based search for Cr to find the optimal value for the slope update, and performs a similar SATD-based search for Cb. If either result is a non-zero slope adjustment parameter, the combined slope adjustment pair (SATD-based update for Cr, SATD-based update for Cb) will be included in the RD check list of the TU.

[0279] Fusion of Chroma Intra Prediction Modes

[0280] During the development of ECM, the fusion of chroma intra modes was proposed.

[0281] Introduction

[0282] The intra prediction modes enabled for chroma components in ECM-4.0 include six cross-component linear model (LM) modes (including CCLM_LT, CCLM_L, CCLM_T, MMLM_LT, MMLM_L, and MMLM_T modes), direct mode (DM), and four default chroma intra prediction modes. These four default modes are given by the list {0, 50, 18, 1}, and if the DM mode already belongs to the list, the mode in the list will be replaced by mode 66.

[0283] ECM-4.0 includes a decoder-side intra mode derivation (DIMD) method for luma intra prediction. First, the horizontal and vertical gradients are calculated for each reconstructed luma sample of the L-shaped template of the second adjacent row and column of the current block to build a gradient histogram (HoG). Then, the two intra prediction modes with the largest and second largest histogram magnitude values ​​are mixed with the planar mode to generate the final prediction value of the current luma block.

[0284] In order to improve the encoding and decoding efficiency of chroma intra prediction, two methods were proposed at the last JVET meeting and studied in EE2 test 1.2. These two methods include the chroma intra prediction mode derived on the decoder side (DIMD chroma) and the fusion of non-LM mode and MMLM_LT mode.

[0285] Test 1.2a: DIMD Chroma Mode

[0286] Fig.10 An example of co-located reconstructed luma samples of the current chroma block is shown.

[0287] The DIMD chroma mode is proposed in test 1.2a. The proposed DIMD chroma mode uses the DIMD derivation method to derive the chroma intra prediction mode of the current block based on the co-located reconstructed luma samples. For example, the horizontal gradient and vertical gradient are calculated for each co-located reconstructed luma sample of the current chroma block to establish HoG, such as Fig.10 Then, the intra prediction mode with the largest histogram amplitude value is adopted to perform chroma intra prediction on the current chroma block.

[0288] When the intra prediction mode derived from the DIMD chroma mode is the same as the intra prediction mode derived from the DM mode, the intra prediction mode with the second largest histogram amplitude value is used as the DIMD chroma mode.

[0289] A CU level flag is signaled to indicate whether the proposed DIMD chroma mode is applied, as shown in Table 7.

[0290] Table 7. Binarization process of intra_chroma_pred_mode in some embodiments

[0291]

[0292]

[0293] Test 1.2b: Fusion of Chroma Intra Prediction Modes

[0294] In test 1.2b, it is proposed that the DM mode and these four default modes can be merged with the MMLM_LT mode as follows:

[0295] pred=(w0*pred0+w1*pred1+(1<<(shift-1)))>>shift

[0296] Among them, pred0 is the prediction value obtained by applying the non-LM mode, pred1 is the prediction value obtained by applying the MMLM_LT mode, and pred is the final prediction value of the current chroma block. The two weights w0 and w1 are determined by the intra-frame prediction mode of the adjacent chroma blocks, and the shift is set to be equal to 2. Specifically, when the upper and left adjacent blocks are both encoded and decoded in LM mode, {w0, w1} = {1, 3}; when the upper and left adjacent blocks are both encoded and decoded in non-LM mode, {w0, w1} = {3, 1}; otherwise, {w0, w1} = {2, 2}.

[0297] For syntax design, if non-LM mode is selected, a flag is signaled to indicate whether fusion is applied, and the proposed fusion is only applicable to I slices.

[0298] Test 1.2c: Test 1.2a + Test 1.2b

[0299] In test 1.2c, the DIMD chroma mode is combined with the chroma intra prediction mode. In some embodiments, the DIMD chroma mode described in test 1.2a is applied, and for I-slices, the DM mode, the four default modes, and the DIMD chroma mode can be combined with the MMLM_LT mode using the weights described in test 1.2b; for non-I-slices, only the DIMD chroma mode and the MMLM_LT mode can be combined using equal weights. Test 1.2d: Simplified Test 1.2a + Test 1.2b

[0300] Fig.11 An example of selecting neighboring reconstructed luma and chroma samples is shown.

[0301] In test 1.2d, the simplified DIMD chroma mode is combined with the chroma intra prediction mode fusion. Fig.11 As shown, the simplified DIMD chrominance mode derives the intra mode based on the adjacent reconstructed Y, Cb and Cr samples in the second adjacent row and column. The rest is the same as test 1.2c.

[0302] Decoder-side intra mode derivation (DIMD)

[0303] FIG. 12A to FIG. 12D An example process of DIMD is shown.

[0304] When DIMD is applied, two intra modes are derived from the reconstructed neighboring samples and these two predictions are combined with the planar mode prediction using weights derived from the gradients, as FIG. 12A to FIG. 12D As shown. Fig. 12A In , the gradient is estimated per sample point (for shaded samples). Fig. 12B In , the gradient value is mapped to the nearest predicted direction in [2, 66]. Fig. 12C In , for each prediction direction, all absolute gradients Gx and Gy of the neighboring pixels in that direction are added together, and the first two directions (M1 and M2) are selected. Fig.12D In , weighted intra prediction is performed using the selected direction. The division operation in the weight derivation is performed using the same lookup table (LUT) based integerization scheme used by CCLM. For example, the division operation in the orientation calculation:

[0305] Orientation = G y / G x

[0306] This is calculated using the following LUT-based scheme:

[0307] x=Floor(Log2(Gx))

[0308] normDiff=((Gx<<4)>>x)&15

[0309] x+=(3+(normDiff!=0)?1:0)

[0310] Orientation = (Gy*(DivSigTable[normDiff]|8)+(1<<(x-1)))>>x

[0311] in,

[0312] DivSigTable

[16] ={0,7,6,5,5,4,4,3,3,2,2,1,1,1,1,0}

[0313] The derived intra modes are included in the master list of intra most probable modes (MPMs), so the DIMD process is performed before building the MPM list. The master derived intra modes of a DIMD block are stored with the block and used for MPM list construction for neighboring blocks.

[0314] Multiple Reference Line (MRL) Intra Prediction

[0315] Fig.13 An example of four reference rows adjacent to a prediction block is shown.

[0316] Multiple reference line (MRL) intra prediction uses more reference lines for intra prediction. Fig.13 In Figure 1, an example of 4 reference lines is depicted, where the samples of segments A and F are not taken from the reconstructed neighboring samples, but are filled with the nearest samples from segments B and E, respectively. HEVC intra-picture prediction uses the nearest reference line (i.e., reference line 0). In MRL, 2 additional lines are used (reference line 1 and reference line 3).

[0317] The index of the selected reference row (mrl_idx) is signaled and used to generate intra prediction values. For reference row idx greater than 0, only additional reference row modes are included in the MPM list and only the mpm index is signaled, not the remaining modes. The reference row index is signaled before the intra prediction mode, and planar mode is excluded from the intra prediction mode if a non-zero reference row index is signaled.

[0318] MRL is disabled for the first row of blocks within a CTU to prevent the use of extended reference samples outside the current CTU row. Similarly, PDPC is disabled when additional rows are used. For MRL mode, the derivation of the DC value in the DC intra prediction mode for non-zero reference row index is aligned with the derivation of reference row index 0. MRL requires storage of 3 neighboring luma reference rows with a CTU to generate predictions. The Cross Component Linear Model (CCLM) tool also requires 3 neighboring luma reference rows for its downsampling filter. The definition of MRL using the same 3 rows is aligned with CCLM to reduce the storage requirements of the decoder.

[0319] Convolutional Cross Component Model (CCCM) for Intra Prediction

[0320] During the development of ECM, the Convolutional Cross-Component Model (CCCM) for Chroma Intra Mode was proposed.

[0321] Introduction

[0322] It is proposed to apply a convolutional cross-component model (CCCM) to predict chroma samples from reconstructed luma samples, similar in spirit to the current CCLM mode. Like CCLM, when chroma subsampling is used, the reconstructed luma samples are downsampled to match the lower resolution chroma grid.

[0323] In addition, similar to CCLM, you can choose to use a single model or a multi-model variant of CCCM. The multi-model variant uses two models, one model is derived for samples above the average brightness reference value, and the other model is derived for the remaining samples (following the spirit of CCLM design). For PUs with at least 128 available reference samples, the multi-model CCCM mode can be selected.

[0324] Convolutional Filters

[0325] The proposed convolutional 7-tap filter consists of a 5-tap plus-shaped spatial component, a nonlinear term, and a bias term. The input of the 5-tap spatial component of the filter consists of the center (C) luminance sample co-located with the chrominance sample to be predicted and its upper / north (N), lower / south (S), left / west (W), and right / east (E) neighboring samples, as shown in Fig.14 shown.

[0326] Fig.14 An example of the location of luma samples in the convolution filter is shown.

[0327] The nonlinear term P is expressed as a power of two of the center luminance sample C and scaled to the sample value range of the content:

[0328] P = (C*C+midVal) >> bit depth

[0329] That is, for 10 bits of content, it is calculated as follows:

[0330] P=(C*C+512)>>10

[0331] The bias term B represents a scalar offset between the input and output (similar to the offset term in CCLM) and is set to the intermediate chrominance value (512 for 10-bit content).

[0332] The output of the filter is calculated as the filter coefficient c i Convolution with the input value and clipped to the valid chroma sample range:

[0333] predChromaVal=c0C+c1N+c2S+c3E+c4W+c5P+c6B

[0334] Calculation of filter coefficients

[0335] Filter coefficient c i It is calculated by minimizing the MSE between the predicted chrominance samples and the reconstructed chrominance samples in the reference region. Fig.15 An example of a reference region for deriving filter coefficients is shown. Fig.15 The diagram shows a reference region consisting of 6 rows of chroma samples above and to the left of the PU. The reference region extends one PU width to the right of the PU boundary and one PU height below. The region is adjusted to include only available samples. The shaded region needs to be extended to support the "side samples" of the plus-shaped spatial filter and fill in the unavailable area.

[0336] MSE minimization is performed by computing the autocorrelation matrix of the luma input and the cross-correlation vector between the luma input and the chroma output. The autocorrelation matrix is ​​subjected to an LDL decomposition and the final filter coefficients are calculated using back substitution. This procedure roughly follows the calculation of the ALF filter coefficients in ECM, but the LDL decomposition is chosen instead of the Cholesky decomposition to avoid the use of square root operations. The proposed method uses only integer operations.

[0337] Bitstream signal transmission

[0338] The usage of this mode is signaled via a PU level flag for the CABAC codec. This is supported by including a new CABAC context. In terms of signaling, CCCM is treated as a sub-mode of CCLM. That is, the CCCM flag is signaled only when the intra prediction mode is LM_CHROMA_IDX (to enable single-model CCCM) or MMLM_CHROMA_IDX (to enable multi-model CCCM).

[0339] Encoder Operation

[0340] The encoder performs two new RD checks in the chroma prediction mode loop, one for checking single-model CCCM mode and one for checking multi-model CCCM mode.

[0341] Inefficiency of video codecs

[0342] For the existing MMLM design, the adjacent reconstructed luma / chroma sample pairs are divided into two groups based on the value of the threshold (which only considers the value of the luma DC). That is, the luma / chroma sample pairs are classified by considering only the intensity of one luma sample. However, the luma component usually retains rich textures, and the current sample may be highly correlated with the neighboring samples. This inter-sample correlation (AC correlation) may be beneficial to the classification of luma / chroma sample pairs and can bring additional coding efficiency.

[0343] also, FIG. 16A to FIG. 16B An example is shown where one chroma sample is associated with multiple luma samples simultaneously.

[0344] like Fig.16A As shown in , CCLM assumes that a given chrominance sample is only related to the corresponding luma sample (L0.5, which can be regarded as the fractional luma sample position), and uses simple linear regression (SLR) and ordinary least squares (OLS) estimation to predict the given chrominance sample. Fig. 16B As shown, in some video contents, a chrominance sample may be simultaneously correlated with multiple luminance samples (AC or DC correlation), so the multiple linear regression (MLR) model can further improve the prediction accuracy.

[0345] Although CCCM mode can improve intra-frame prediction efficiency, there is still room for further improvement. At the same time, some parts of the existing CCCM mode also need to be simplified to achieve efficient codec hardware implementation, or need to be improved to improve codec efficiency. In addition, the trade-off between its implementation complexity and its codec efficiency benefits needs to be further improved.

[0346] Marginal Linear Model (ELM)

[0347] The present disclosure improves the encoding and decoding efficiency of luminance and chrominance components, has a design spirit similar to MMLM, but introduces a classifier that takes into account luminance edge / AC information. In addition to the existing band-based MMLM, the present disclosure also provides examples of the proposed classifier. The process of generating predicted chrominance samples is the same as that of MMLM (original least squares method, simplified min-max method, etc.), but the classification method is different.

[0348] Note that although the existing CCLM design in the VVC standard is adopted as the basic CCLM method in the following description, for those skilled in the art of video encoding and decoding, the proposed cross-component method described in the present disclosure can also be applied to other predictive encoding and decoding tools with a similar design spirit. For example, for Chroma from Luma (CfL) in the AV1 standard, the proposed ELM can also be applied by dividing luminance / chrominance sample pairs into multiple groups.

[0349] It should be noted that Y / Cb / Cr can also be represented as Y / U / V in the field of video encoding and decoding.

[0350] Note that if the video is in RGB format, the proposed ELM can also be applied by simply mapping the YUV representation to GBR (as shown in the following paragraphs).

[0351] Note that the drawings in the present disclosure can be combined with all the examples mentioned in the present disclosure.

[0352] In some embodiments, a method for decoding a video signal is provided, the method comprising: receiving an encoded block of luminance samples of a first video signal block; decoding the encoded block of luminance samples to obtain reconstructed luminance samples; dividing the reconstructed luminance samples into multiple sample groups based on the direction and intensity of edge information; applying different linear prediction models to the reconstructed luminance samples in different sample groups; and predicting chrominance samples of the first video signal block based on the applied linear prediction models.

[0353] Classification

[0354] Classifier C0: Represent the existing threshold-based MMLM classifier as C0, which produces 2 classes.

[0355] Classifier C1: Local Binary Pattern (LBP)

[0356] First, compare the current sample Y0 with N adjacent samples Yi.

[0357] Second, if Y0>Yi, then score += 1; otherwise, if Y0<Yi, then score -= 1.

[0358] Third, quantize the score to form K classes.

[0359] Fourth, classify the current sample point into K classes.

[0360] Take classifier C1 as an example:

[0361] First, compare the current sample point Y0 with the 4 neighboring sample points Yi (without diagonals).

[0362] Second, if Y0 > Yi, then score += 1; otherwise, if Y0 < Yi, then score -= 1.

[0363] Third, quantize the score to form 3 classes: (score > 0, = 0, < 0).

[0364] Fourth, classify the current sample point into 3 classes.

[0365] Classifier C2:

[0366] First, select a direction to calculate the edge intensity. This direction is composed of the current sample point and N neighboring sample points along this direction. Calculate an edge intensity by subtracting the current sample point and a neighboring sample point.

[0367] Second, quantize the edge intensity into M segments through M - 1 thresholds Ti.

[0368] Third, classify the current sample point into K classes.

[0369] Take classifier C2 as an example:

[0370] First, define a direction according to the MMLM mode. For example, MMLM_L: ver, MMLM_A: hor, MMLM: use C0. This direction is composed of the current sample point and 1 neighboring sample point along this direction. Calculate the edge intensity by subtracting the current sample point and the neighboring sample point.

[0371] Second, quantize the edge intensity into 2 segments through 1 simple threshold 0. (> 0, <= 0).

[0372] Third, classify the current sample point into 2 classes.

[0373] Classifier C3:

[0374] First, as Fig.21 shown, select an edge detection filter shape (e.g., 1 - tap) to calculate the edge intensity. This direction is composed of the current sample point and N neighboring sample points along this direction. Calculate an edge intensity through the filter value.

[0375] Secondly, the edge strength is quantized into M segments by M-1 thresholds Ti. (Or use a mapping table). The filter shape, filter taps and mapping table can be predefined or signaled / switched at SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level.

[0376] Third, classify the current sample point using K classes (for example, K=M).

[0377] The above classifiers can be combined to form a joint classifier. For example, C0 is combined with C2 to produce 2*2 classes. For example, C2 and C2 are combined, but the directions of definition are different (MMLM_L: hor, MMLM_A: ver), to produce 2*2 classes.

[0378] The brightness samples to be classified may be first downsampled to conform to the CCLM design.

[0379] Sample processing

[0380] like Figure 5 As shown, for a chrominance block to be predicted with a co-located luminance block:

[0381] First, the co-located luminance block samples are reconstructed.

[0382] Secondly, the co-located adjacent brightness samples (gray) are downsampled.

[0383] Third, the adjacent luminance / chrominance sample pairs are classified based on the classifier described in the embodiments of the present disclosure.

[0384] Fourth, different linear models are derived for different classes.

[0385] Fifth, different linear models are applied to the reconstructed brightness samples in different classes.

[0386] Sixth, the chrominance samples are predicted based on the applied linear prediction model.

[0387] Filter-based Linear Model (FLM)

[0388] For the chroma samples to be predicted, the reconstructed co-located and adjacent luma samples can be used to predict the chroma samples to capture the inter-sample correlation between the co-located luma samples, adjacent luma samples, and chroma samples. The reconstructed luma samples are linearly weighted and combined with an "offset" to generate predicted chroma samples (C: predicted chroma sample, L: the i-th reconstructed co-located or adjacent luma sample, α i: filter coefficient, β: offset, N: filter tap). Note that the linear weighting plus the offset value directly forms the predicted chroma sample (which can be adaptively low-passed or high-passed according to the video content), which is then added to the residual to form the reconstructed chroma sample.

[0389]

[0390] For a given CU, the top and left reconstructed luma / chroma samples can be used to derive / train the FLM parameters (α i , β). Like CCLM, α can be derived via OLS i The top and left training samples are collected and a pseudo-inverse matrix is ​​calculated on the encoder / decoder side to derive the parameters, which are then used to predict the chroma samples in a given CU. Let N denote the number of filter taps applied to the luma samples, M denote the total top and left reconstructed luma / chroma sample pairs used for training the parameters, represents the brightness sample of the i-th sample pair and the j-th filter tap, C i Denotes the chrominance sample of the i-th sample pair, the pseudo-inverse matrix A is shown in the following equation + and the derivation of parameters. Fig.17 An example is shown where N is 6 (6 taps), M is 8, top 2 rows / left 3 columns of luma samples, and top 1 row / left 1 column of chroma samples are used to derive / train parameters.

[0391]

[0392] b = Ax

[0393] x=(A T A) -1 A T b=A + b

[0394] In some embodiments, only α i Instead of using the offset β to predict the chrominance samples, this is a subset of the above embodiment.

[0395] In some embodiments, although the existing CCLM design in the VVC standard is used as the basic CCLM method in the following description, for those skilled in the art of video coding and decoding, the proposed cross-component method described in the present disclosure can also be applied to other predictive coding tools with similar design spirits. For example, for the chrominance based on luma (CfL) in the AV1 standard, the proposed FLM can also be applied by incorporating multiple luma samples into the MLR model.

[0396] In some embodiments, the proposed ELM / FLM / GLM can be directly extended to the CfL design in the AV1 standard, which explicitly transmits the model parameters (α, β). For example, (1-tap case) α and / or β are derived at the encoder at the SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level and signaled to the decoder for CfL mode.

[0397] In some examples, Y / Cb / Cr may also be expressed as Y / U / V in the field of video coding and decoding.

[0398] In some examples, if the video is in RGB format, the proposed FLM can also be applied by simply mapping the YUV representation to GBR (as shown in the following paragraphs).

[0399] In some variations, the drawings in this disclosure may be combined with all of the examples mentioned in this disclosure.

[0400] In some embodiments, a method of decoding a video signal is provided, the method comprising: receiving an encoded block of luma samples of a first video signal block;

[0401] The invention discloses a method for predicting chrominance samples of a first video signal block by decoding a coded block of luminance samples to obtain reconstructed luminance samples; determining a luminance sample region and a chrominance sample region to derive a multivariate linear regression (MLR) model; deriving the MLR model by pseudo-inverse matrix calculation; applying the MLR model to reconstruct the luminance samples; and predicting chrominance samples of a first video signal block based on the applied MLR model.

[0402] Filter shape

[0403] Fig.17 Examples of luma and chroma samples used to derive parameters of the prediction model are shown.

[0404] like Fig.17 As shown, a 6-tap luma filter is used for FLM prediction. However, although the multi-tap filter can fit the training data (i.e., the reconstructed luma / chroma samples adjacent to the top / left) well, in some cases, the training data does not capture all the characteristics of the test data, which may lead to overfitting and may not predict the test data (i.e., the chroma block samples to be predicted) well. In addition, different filter shapes can adapt well to different video block contents, thereby achieving more accurate prediction.

[0405] To address this issue, the filter shape / number of filter taps can be predefined or signaled / switched at the SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level. A set of filter shape candidates can be predefined or signaled / switched at the SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level. Different components (U / V) may have different filter switch controls. For example, a set of filter shape candidates (idx=0~5) are predefined, and the filter shape (1,2) represents a 2-tap luminance filter, and (1,2,4) represents a 3-tap luminance filter, such as Fig.17 As shown ... and so on, the filter shape selection for the U / V component can be switched at the PH or CU / CTU level. Note that the N-tap can represent the N-tap with or without offset β, as described above with respect to the embodiment of FLM.

[0406]

[0407] Fig.18 Examples of luma and chroma samples used to derive parameters of the prediction model are shown.

[0408] Different chroma types / color formats can have different predefined filter shapes / taps. For example, using predefined filter shapes, for 420 type-0: (1, 2, 4, 5), 420 type-2: (0, 1, 2, 4, 7), 422: (1, 4), 444: (0, 1, 2, 3, 4, 5), such as Fig.18 shown.

[0409] The unavailable luminance / chrominance samples used to derive the MLR model can be filled in from the available reconstructed samples. Fig.18 The 6-tap (0, 1, 2, 3, 4, 5) filter shown, for the CU located at the left picture boundary, the left column including (0, 3) is unavailable (outside the picture boundary), so (0, 3) is repeatedly padded from (1, 4) to apply the 6-tap filter. Note that the padding process is applied in both the training data (top / left neighboring reconstructed luma / chroma samples) and the test data (luma / chroma samples in the CU).

[0410] Fig. 22 Examples of different filter shapes and numbers of filter taps are shown. It should be understood that Fig. 22 , each cluster of solid blocks labeled with letters in alphabetical order represents a separate filter, and for ease of illustration, different filters are shown together in this figure.

[0411] Fig.23Other examples of different filter shapes and numbers of filter taps are shown.

[0412] Fig.24 and FIG. 25A to FIG. 25G Other examples of different filter shapes and numbers of filter taps are shown.

[0413] One or more shapes / numbers of filter taps may be used for FLM prediction, for example Fig. 22 , Fig.23 , Fig.24 One or more sets of filter taps can be used for FLM prediction, for example FIG. 25A to FIG. 25G shown.

[0414] Matrix derivation

[0415] As described above with respect to the FLM embodiment, the MLR model (linear equation) must be derived at both the encoder and decoder. This section proposes several methods for deriving the pseudo-inverse matrix A + Or a method that directly solves the linear equation. Other known methods can also be applied, such as Newton's method, Cayley-Hamilton method, and eigendecomposition as mentioned in https: / / en.wikipedia.org / wiki / Invertible_matrix.

[0416] Note that in this section, for simplicity, A + Indicated as A -1 .

[0417] In some embodiments, A is solved by a closed-form analytical solution of the adjoint matrix (adjA) -1 In some examples, an n×n general form, a 2×2, and a 3×3 case are shown below. If FLM uses 3×3, then 2 scalars need to be solved plus an offset.

[0418] b=Ax,x=(A T A) -1 A T b=A + b, denoted as A in this section -1 b

[0419]

[0420] The (n-1)×(n-1) submatrix obtained by deleting the jth row and the ith column

[0421]

[0422] In some embodiments, Gauss-Jordan elimination can be used to augment the matrix [AI nand a series of elementary row operations to obtain the reduced row echelon form [I|X] to solve these linear equations. Examples of 2×2 and 3×3 are shown below.

[0423]

[0424]

[0425] In some embodiments, to solve Ax = b, first decompose A using the Cholesky-Crout algorithm to obtain an upper triangular matrix and a lower triangular matrix, and then perform one forward substitution followed by one backward substitution in sequence to obtain the solution. An example of 3×3 is shown below.

[0426]

[0427] In some embodiments, if certain conditions are met and it becomes impossible to solve the linear equations, default values can be used to fill the chrominance prediction values. The default values can be predefined or signaled / switched at the SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level. For example, predefine 1<<(bit depth - 1), meanC, meanL, or meanC - meanL (average current chrominance or other chrominance, available luminance values, or a subset of the FLM reconstructed neighboring regions). Default α i can be 0.

[0428] In some examples, first solve A through the adjoint matrix -1 , but A is a singular value and detA is 0.

[0429] Second, A cannot be Cholesky decomposed, g jj <REG_SQR, where REG_SQR is a small value, which can be predefined or signaled / switched at the SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level.

[0430] The applied region

[0431] Fig.17 shows a typical case of deriving FLM parameters using the top 2 / left 3 luminance rows and the top 1 / left 1 chrominance row. However, as explained in this disclosure, due to different block contents and the reconstruction quality of different neighboring samples, using different regions for parameter derivation may bring coding and decoding advantages. Several ways are proposed to select the applied region for parameter derivation.

[0432] First, similar to MDLM, FLM derivation can only use top or left luma / chroma samples to derive parameters. Whether to use FLM, FLM_L or FLM_T can be predefined or signaled / switched at SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / subblock / sample level.

[0433] When the FLM mode is applied, W'=W, H'=H;

[0434] When FLM_T mode is applied, W'=W+We; where We represents the extended top luminance / chrominance sample. When FLM_L mode is applied, H'=H+He; where He represents the extended left luminance / chrominance sample.

[0435] The number of extended luma / chroma samples (We, He) may be predefined or signaled / switched at SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level.

[0436] For example, (We, He)=(H, W) is predefined as a VVC CCLM, or (W, H) is predefined as an ECM CCLM. Unavailable (We, He) luma / chroma samples can be repeatedly filled from the nearest (horizontal, vertical) luma / chroma samples.

[0437] Fig.19 An example is shown where the top or left reconstruction sample is used for FLM.

[0438] Fig.19 A diagram of FLM_L / FLM_T is shown (eg, at 4 taps). When FLM_L or FLM_T is applied, only H' or W' luma / chroma samples, respectively, are used for parameter derivation.

[0439] Second, similar to MRL, different row indices can be predefined or signaled / switched at SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / subblock / sample level to indicate the selected rows of luma / chroma sample pairs. This can benefit from different reconstruction qualities of samples in different rows.

[0440] Fig. 20 Another example in which the reconstruction samples are used for FLM is shown.

[0441] Fig. 20 It is shown that, similar to MRL, FLM can use different rows for parameter derivation (e.g., at 4 taps). For example, index 1: use shadow luma / chroma samples.

[0442] Third, the CCLM area is expanded and all top N rows / left M rows are taken for parameter derivation. Fig. 20 It shows that all dark / shadow regions can use luma and chroma samples at once. Using a larger region (data) for training may result in a more robust MLR model.

[0443] grammar

[0444] FLC: Fixed Length Code

[0445] TU: Truncated Unary Code

[0446] EGk: k-th order Exponential Golomb code, where k can be fixed or signaled / switched at SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level.

[0447] SVLC: Signed EG0

[0448] UVLC: Unsigned EG0

[0449] Table 8. Example of FLM syntax. Note that the binarization of each syntax element can be changed.

[0450]

[0451]

[0452] Gradient Linear Model (GLM)

[0453] Although the above embodiments of FLM provide the best flexibility (and thus the best performance), if the number of filter taps increases, many unknown parameters need to be solved. When the inverse matrix is ​​larger than 3×3, the closed-form derivation is not suitable (too many multipliers), and iterative methods such as Cholesky need to be used, which increases the burden on the decoder processing cycle. In this section, pre-operation is proposed before applying the MLR model, including using sample point gradients to develop the correlation between luminance AC information and chrominance intensity. With the help of gradients, the number of filter taps can be effectively reduced. In general, GLM is simplified from FLM. We focus on examples with unknown parameters <= 3 (2 taps + 1 offset, or 3 taps without offset).

[0454] In some embodiments, the described methods / examples can be combined / reused with the methods mentioned in other embodiments, including but not limited to classification, filter shape, matrix derivation (through special processing), applied regions, and syntax. In addition, the methods / examples listed in this section can also be applied to other embodiments (e.g., with more taps) to obtain better performance at a certain complexity trade-off.

[0455] In this disclosure, reference samples / training templates / reconstructed neighborhoods generally refer to luma samples used to derive MLR model parameters, which are then applied to internal luma samples in a CU to predict chroma samples in the CU.

[0456] Filter shape

[0457] Instead of using luma sample intensity values ​​directly as input to MLR, pre-operations (e.g., pre-linear weighting, sign, scaling / absolute value, thresholding, ReLU) can be applied to reduce the dimensionality of unknown parameters. For example, instead of applying a 2-tap on 2 luma samples, the 2 luma samples can be pre-linearly weighted and then a simpler 1-tap can be applied to reduce complexity. Fig.21 Some examples of 1-tap / 2-tap (with offset) pre-operation are shown, where the 2-tap coefficients are represented as (a, b) and each circle represents the position of the corresponding co-located chroma sample. Different 1-tap modes are designed for different gradient directions, and different "interpolated" luma samples are used (weighting different luma positions) for gradient calculation. The pre-operation parameters (coefficient, sign, scale / absolute value, threshold, ReLU) can be fixed or signaled / switched at SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level. Note that in the example, if multiple coefficients apply to one sample (e.g., -1, 4), they can be merged (e.g., 3) to reduce operations.

[0458] Pre-operations can be based on gradient, edge direction (detection), pixel intensity, pixel change, pixel variance, Roberts / Prewitt / compass / Sobel / Laplacian operators, high pass filter, low pass filter, etc. The edge direction detector listed in the example can be extended to different edge directions. For example, 1 tap (1, -1) or 2 taps (a, b) are applied along different directions to detect different edge gradients. The filter shape / coefficients can be symmetric about the chrominance position, such as Fig.21 As shown (420 type-0 case).

[0459] The pre-operation can be applied repeatedly. For example, a template filter is applied once to the template using a low-pass smoothing FIR filter [1, 2, 1] / 4 or [1, 2, 1; 1, 2, 1] / 8 to remove outliers. Then, a 1-tap GLM is applied to derive the MLR model.

[0460] Power of 2 constraint: Pre-operation coefficients (coefficients applied eventually (e.g., 3) or intermediately (e.g., -1, 4) to each luma sample) can be constrained to values ​​that are powers of 2 to save multipliers.

[0461] An example of a 1-tap GLM. The notation is similar to that in the above embodiment for FLM. Note that L here represents the "pre-operated" luminance sample. For example, Fig.21 As shown, the 1-tap GLM [-1, 0, 1; -1, 0, 1]. The parameter derivation of the 1-tap GLM can reuse the CCLM design (described later), but the directional gradient must be considered (possibly using a high-pass filter). 2-tap or multi-tap GLMs require additional MLR parameter derivation (which cannot be reused).

[0462]

[0463]

[0464] b = Ax

[0465] x=(A T A) -1 A T b=A + b

[0466]

[0467] Implicit filter shape derivation

[0468] In some embodiments, rather than explicitly signaling the selected filter shape index, the used directionally-steered filter shape may be derived at the decoder to save bit overhead.

[0469] First, N directional gradient filters are applied to each reconstructed luminance sample of the L-shaped template in the i-th neighboring row and column of the current block.

[0470] Second, the filtered values ​​(gradients) are accumulated by SAD, SSD or SATD.

[0471] Third, the Histogram of Gradients (HoG) is constructed.

[0472] Fourth, the maximum value in HoG is the derived (brightness) gradient direction.

[0473] For example, the decoder-side intra mode derivation (DIMD) method for luma intra prediction included in ECM-4.0 is reused.

[0474] First, two directional gradient filters (3×3hor / ver Sobel) are applied to each reconstructed luminance sample of the L-shaped template in the second adjacent row and column of the current block.

[0475] Second, the filtered values ​​(gradients) are accumulated via SAD.

[0476] Third, the Histogram of Gradients (HoG) is constructed.

[0477] Fourth, the maximum value in HoG is the derived (brightness) gradient direction.

[0478] Shape candidates: [-1, 0, 1; -1, 0, 1], [1, 2, 1; -1, -2, -1]. For example, the maximum value is hor, then the shape [-1, 0, 1; -1, 0, 1] is used for GLM

[0479] The gradient filter used to derive the gradient direction can have the same or different shape as the GLM. For example, both use the levels [-1, 0, 1; -1, 0, 1].

[0480] Classification

[0481] FLM / GLM can be combined with MMLM or ELM. Take GLM as an example (1 tap or 2 taps). When combined with classification, each group can share or have its own filter shape, and the shape of each group is indicated by the syntax. For example, combined with C0':

[0482] Group 0: grad_hor, model 0, Group 1: grad_ver, model 1.

[0483] Group 0: grad_hor, model 0, Group 1: grad_hor, model 1, only generate hor brightness pattern once.

[0484] In some embodiments, combined with the MMLM classifier C0:

[0485] Divide adjacent reconstructed luminance / chrominance sample pairs into two groups based on the threshold;

[0486] Derive different MLR models for different groups (which can be GLM simplifications);

[0487] Divide the luminance / chrominance sample pairs inside the CU into 2 groups;

[0488] Apply different MLR models to the reconstructed brightness samples in different groups;

[0489] The chroma samples in the CU are predicted based on MLR models of different classifications.

[0490]

[0491] rec L ′(i,j): downsampled reconstructed brightness sample,

[0492] rec C (i,j): reconstructed chrominance samples (note that only adjacent samples are available),

[0493] Threshold: The average value of neighboring reconstructed brightness samples.

[0494] Note that the number of classes can be extended to multiple classes by increasing the number of thresholds (e.g., equally divided based on min / max of neighboring reconstructed (downsampled) luma samples, either fixed or signaled / switched at SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level).

[0495] In some embodiments, in combination with MMLM classifier variant C0':

[0496] Instead of using the MMLM luminance DC intensity, the FLM / GLM filter values ​​applied to neighboring luminance samples are used for classification. For example, if a 1-tap (1, -1) GLM is applied, the average AC value (physical meaning) is used. This process can be similar to the above embodiment combined with the MMLM classifier C0.

[0497] Divide adjacent reconstructed luma / chroma sample pairs into K groups based on one or more filter shapes, one or more filter values, and K-1 thresholds Ti;

[0498] Derive different MLR models for different groups (which can be GLM simplifications);

[0499] Divide the luminance / chrominance sample pairs inside the CU into K groups;

[0500] Apply different MLR models to the reconstructed brightness samples in different groups;

[0501] The chroma samples in the CU are predicted based on MLR models of different classifications.

[0502] The threshold may be predefined (e.g., 0 or may be a table), or signaled / switched at the SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / subblock / sample level. For example, the threshold may be the average AC value (filtered value) of neighboring reconstructed (possibly downsampled) luma samples (2 groups), or equally divided based on min / max AC (K groups).

[0503] In some embodiments, combined with the ELM classifier C3:

[0504] like Fig.21 As shown, a filter shape (e.g., 1 tap) is selected to calculate the edge strength. The direction is composed of the current sample point and N neighboring sample points (e.g., all 6) along the direction. An edge strength is calculated by filtering the value (e.g., equivalent value).

[0505] The edge strength is quantized into M segments by M-1 thresholds Ti.

[0506] Classify the current sample point using K classes (for example, K == M).

[0507] Derive different MLR models for different groups (which can be GLM simplifications);

[0508] Divide the luminance / chrominance sample pairs inside the CU into K groups;

[0509] Apply different MLR models to the reconstructed brightness samples in different groups;

[0510] The chroma samples in the CU are predicted based on MLR models of different classifications.

[0511] The filter shape used for classification can be the same as or different from the filter shape used for MLR prediction. The number of thresholds M-1 and the value Ti of the thresholds can be fixed or signaled / switched at the SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level.

[0512] In some embodiments, other classifiers / combined classifiers in the embodiments regarding ELM can also be used for FLM / GLM.

[0513] In some embodiments, if the number of classification samples in a group is less than a certain number (e.g., a predefined number of 4), the default value mentioned in the embodiment of FLM may be applied to the group parameters (α i , β). If the corresponding neighboring reconstruction samples are not available with respect to the selected LM mode, a default value may be applied. For example, the MMLM_L mode is selected but the samples on the left are invalid.

[0514] Simplify and unify

[0515] This section provides a simplification of the GLM. The matrix / parameter derivation in the embodiments of the FLM requires floating-point operations (e.g., closed-form division), which is costly for the decoder hardware and therefore requires a fixed-point design. For the case of a 1-tap GLM, it can be viewed as a modified luminance reconstruction sample generation of the CCLM (e.g., horizontal gradient direction, from CCLM [1, 2, 1; 1, 2, 1] / 8 to GLM [-1, 0, 1; -1, 0, 1]), and the original CCLM process can be reused for the GLM, including fixed-point operations, MDLM downsampling, division table, application size restrictions, min-max approximation, and slope adjustment. For all items, the 1-tap GLM can have its own configuration or share the same design with the CCLM. For example, a simplified min-max method is used to derive parameters (instead of LMS) and the slope adjustment is combined after the GLM model is derived. In this case, the center point (luminance value y r) becomes the average value of the reference luma sample "gradient". Another example, when GLM is turned on for this CU, CCLM slope adjustment is inferred to be off, and no syntax related to slope adjustment needs to be signaled.

[0516] This section takes the reference sample points of typical cases (top row, left column) as an example. Fig. 20 As shown, the extended reconstruction region can also use simplifications with the same spirit, and can adopt a syntax (such as MDLM, MRL) to indicate a specific region.

[0517] In some embodiments, the following aspects may be combined and applied together: For example, combining reference sample point downsampling and a division table to perform a division process.

[0518] When applying classification (MMLM / ELM), the same or different simplification operations can be applied to each group. For example, fill the sample points of each group to the target number of sample points and then right shift, and then apply the same derivation process and the same division table.

[0519] Fixed-point implementation

[0520] The CCLM design can be reused for the 1-tap case, and the division by n is implemented by right shift, and the division by A2 is implemented by LUT. α , ) can be the same as CCLM or have different values ​​for higher precision. The integerization parameters can be predefined or signaled / switched at SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level and can be conditional on the sequence bit depth. For example, n table = bit depth + 4.

[0521] MDLM downsampling

[0522] When GLM is combined with MDLM, the total samples available for parameter derivation may not be a power of 2 value and need to be padded to a power of 2 to replace the division with a right shift operation. For example, for an 8×4 chroma CU, MDLM requires W+H=12 samples, but MDLM_T has only 8 samples available (reconstructed), padded with 4 samples (0, 2, 4, 6) that are equally downsampled.

[0523]

[0524] Other filling methods may also be applied, such as repeating / mirroring the filling for the last neighboring sample point (rightmost / bottommost).

[0525] The filling method of GLM can be the same as or different from CCLM.

[0526] Note that in the ECM version, the 8×4 chroma CU MDLM_T / MDLM_L requires 2T / 2L=16 / 8 samples respectively, in which case the same padding method can be applied to meet the target number of samples that is a power of 2.

[0527] Division LUT

[0528] The division LUT proposed for CCLM / LIC (local illumination compensation) in the development of known standards such as AVC / HEVC / AV1 / VVC / AVS can be used for GLM division. For example, for the case of bit depth = 10, the LUT in the above embodiment is reused (Table 5). The division LUT can be different from CCLM. For example, CCLM uses the min-max algorithm with DivTable as described in the above CCLM section of this disclosure, but GLM uses the 32-entry LMS division LUT as described in the above section of this disclosure.

[0529] When a GLM is combined with an MMLM, the meanL value may not always be positive (e.g., using filter / gradient values ​​to classify groups), so sgn(meanL) needs to be extracted and abs(meanL) used to look up the division LUT. Note that the division LUT used for classification and parameter derivation for the MMLM may be different. For example, a lower precision LUT (such as the LUT in min-max) is used for mean classification, while a higher precision LUT (such as the LUT in LMS) is used for parameter derivation.

[0530] Size and latency limits

[0531] Similar to the CCLM design, some size restrictions may be applied to the ELM / FLM / GLM. For example, as described in the CCLM section above of this disclosure, there are the same constraints on the luma-chroma delays in the dual tree.

[0532] Size restrictions can be based on CU area / width / height / depth. Disabling thresholds can be predefined or signaled at SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level. For example, the disabling threshold is predefined as: chroma CU area < 128.

[0533] Reduce line buffer

[0534] Similar to the CCLM design, if the co-located luma region of the current chroma CU contains the first row within a CTU, the top template sample generation can be limited to 1 row to reduce the row buffer storage of CTU rows. Note that when the upper reference row is at a CTU boundary, only one luma row (common row buffer in intra prediction) is used to derive the downsampled luma samples.

[0535] For example, in Fig.19 In , if the co-located luma region of the current chroma CU contains the 1st row within a CTU, the top template can be restricted to use only 1 row (instead of 2 rows) for parameter derivation (other CUs can still use 2 rows). This saves luma sample row buffer storage when processing CTUs row by row on the decoder hardware. Several methods can be used to achieve the reduction of the row buffer. Note that examples that are restricted to "1" row can be extended to N rows by similar operations. Such operations can also be applied to 2 taps or multiple taps. When multiple taps are used, chroma samples may also need to have operations applied.

[0536] For example, Fig.21 The 1-tap [1, 0, -1; 1, 0, -1].

[0537] First, reduce the shape: we can reduce it to [0, 0, 0; 1, 0, -1], using only the coefficients of the lower row.

[0538] Secondly, filling: the limited upper row brightness samples can be filled with the lower row brightness samples (repeat, mirror, 0, meanL, meanC... etc.).

[0539] Fusion of Chroma Intra Prediction Modes

[0540] Similar to the fusion design described above, since GLM can be viewed as a special CCLM mode, the fusion design can be reused or have its own way. Multiple weights (>=2) can be applied to generate the final prediction value. For example,

[0541] pred=(w0*pred0+w1*pred1+(1<<(shift-1)))>>shift

[0542] pred0 is non-LM, fused with pred1 GLM predictions.

[0543] pred0 is one of the CCLMs (including all MDLM / MMLMs) fused with the pred1 GLM predictions.

[0544] pred0 is a GLM, fused with the predicted values ​​of pred1GLM.

[0545] The weights w0 and w1 of different I / P / B stripes may be designed differently depending on whether the neighboring blocks are encoded using CCLM / GLM / other coding modes and the block size / width / height.

[0546] For example, the I slice can be determined by the intra prediction mode of the adjacent chroma block, and the shift is set to be equal to 2. For example, when the upper and left adjacent blocks are both coded and decoded in LM mode, {w0, w1} = {1, 3}; when the upper and left adjacent blocks are both coded and decoded in non-LM mode, {w0, w1} = {3, 1}; otherwise, {w0, w1} = {2, 2}. For non-I slices, w0 and w1 are both set to be equal to 2.

[0547] For syntax design, if non-LM mode is selected, a flag is signaled to indicate whether fusion is applied.

[0548] Extension: 1-tap linear model

[0549] The 1-tap GLM has a good gain-complexity trade-off because it can reuse the existing CCLM module without introducing additional derivations. This 1-tap design can be expanded (generalized) as follows:

[0550] First, for the chrominance sample to be predicted, a corresponding luminance sample L is generated by combining the co-located and adjacent luminance samples.

[0551] Second, where the combination can be:

[0552] Linear filters, such as high-pass gradient filter (GLM), low-pass smoothing filter (CCLM),

[0553] n-th power nonlinear filter, such as L n , where n can be a positive number, a negative number, or a positive or negative fraction, such as +1 / 2, which is the square root, which can be rounded and rescaled to the bit depth dynamic range, such as +3, which is the cube, which can be rounded and rescaled to the bit depth dynamic range.

[0554] Third, the combination in point 2 can be repeatedly applied. For example, [1, 2, 1; 1, 2, 1] / 8 FIR smoothing is applied to the reconstructed luminance samples, and a nonlinear power of 1 / 2 is applied.

[0555] Fourth, the nonlinear filter can be implemented as a LUT, for example, for bit depth = 10, nth power, n = 1 / 2, LUT[i] = (int)(sqrt(i)+0.5)<<5, i = 0~1023, where 5 represents the dynamic range scaled to bit depth = 10.

[0556] Non-linear filters provide an option when linear filters cannot handle the luma-chroma relationship efficiently. Whether to use non-linear terms can be predefined or signaled / switched at SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level.

[0557] In the above case, GLM can refer to the generalized linear model (generate a brightness sample linearly or nonlinearly and feed into the CCLM linear model), and the linear / nonlinear generation is called the general mode.

[0558] In some embodiments, different gradient / general patterns can be combined. Some examples of forming another pattern:

[0559] For example, combine 1 gradient mode with CCLM downsampled values.

[0560] For example, combining a gradient mode with a nonlinear L 2 Value combination.

[0561] For example, combine 1 gradient pattern with another gradient pattern, either with different or the same direction.

[0562] In some embodiments, the combination may be additive, subtractive, or linearly weighted.

[0563] grammar

[0564] FLC: Fixed Length Code

[0565] TU: Truncated Unary Code

[0566] EGk: k-th order Exponential Golomb code, where k can be fixed or signaled / switched at SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level.

[0567] SVLC: Signed EG0

[0568] UVLC: Unsigned EG0

[0569] Table 9. Example of GLM syntax. Note that the binarization of each syntax element can be varied.

[0570]

[0571]

[0572] The GLM on / off control of the Cb / Cr components can be controlled together or individually. For example, at the CU level,

[0573] First, a flag indicates whether GLM is activated for this CU.

[0574] Second, if active, 1 flag indicates whether both Cb / Cr are active.

[0575] Third, 1 flag indicates that Cb or Cr is active, if not simultaneously active.

[0576] Fourth, when Cb and / or Cr are activated, the filter index / gradient (normal) mode is transmitted using the signal alone.

[0577] Fifth, all flags can have their own context model or be bypassed for encoding and decoding.

[0578] Whether to signal the GLM on / off flag may depend on the luma / chroma coding mode and the CU size.

[0579] For example, in the ECM5 chroma intra mode syntax, GLM can be inferred to be off in the following cases:

[0580] First, MMLM / MMLM_L / MMLM_T.

[0581] Secondly, the CU area is smaller than A, where A can be predefined or signaled / switched at the SPS / DPS / VPS / SEI / APS / PPS / PH / SH / area / CTU / CU / sub-block / sample level.

[0582] Third, if combined with CCCM, it is inferred to be off when CCCM is on.

[0583] In some embodiments, when a GLM is combined with an MMLM, the different models may share the same or have their own gradients / general patterns.

[0584]

[0585] Note that the drawings in this disclosure can be combined with all the examples mentioned in this disclosure.

[0586] Note that the disclosed methods can be applied independently or in combination.

[0587] CCCM without downsampling

[0588] CCCM requires processing of downsampled luma reference values ​​before computing model parameters and applying the CCCM model, which increases the burden on the decoder processing cycle. In this section, CCCM without downsampling is proposed, including utilizing non-downsampled luma reference values ​​and / or different choices for non-downsampled luma reference. As described below, one or more filter shapes can be used to achieve this.

[0589] Please note that the methods / examples in this section can be combined / reused with the methods mentioned in other embodiments, including but not limited to classification, filter shape, matrix derivation (through special processing), applied regions, and syntax. In addition, the methods / examples listed in this section can also be applied to other embodiments (e.g., with more taps) to obtain better performance at a certain complexity trade-off.

[0590] In this disclosure, reference samples / training templates / reconstructed neighborhoods generally refer to luma samples used to derive MLR model parameters, which are then applied to internal luma samples in a CU to predict chroma samples in the CU.

[0591] Filter shape

[0592] One or more shapes / numbers of filter taps may be used for CCCM prediction, such as Fig. 22 , Fig.23 , Fig.24 One or more sets of filter taps can be used for FLM prediction, for example FIG. 25A to FIG. 25G As shown. The selected luminance reference value is non-downsampled. CCCM prediction can be performed using one or more predefined shapes / numbers of filter taps based on previously decoded information at TB / CB / slice / picture / sequence level.

[0593] Although multi-tap filters can fit the training data (i.e., the top / left neighboring reconstructed luminance / chrominance samples) well, in some cases, the training data does not capture all the characteristics of the test data, which may lead to overfitting and may not predict the test data (i.e., the chrominance block samples to be predicted) well. In addition, different filter shapes can adapt well to different video block contents, thereby achieving more accurate prediction. To address this issue, the filter shape / number of filter taps can be predefined or signaled / switched at the SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level. A set of filter shape candidates can be predefined or signaled / switched at the SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level. Different components (U / V) may have different filter switch controls. For example, a set of filter shape candidates (idx=0-5) is predefined, and the filter shape (1, 2) represents a 2-tap luminance filter, and (1, 2, 4) represents a 3-tap luminance filter, such as Fig.17 As shown ... and so on, the filter shape selection for the U / V component can be switched at the PH or CU / CTU level. Note that the N-tap can represent the N-tap with or without offset β, as described above with respect to the embodiment of FLM.

[0594]

[0595] Different chroma types / color formats can have different predefined filter shapes / taps. For example, using predefined filter shapes, for 420 type-0: (1, 2, 4, 5), 420 type-2: (0, 1, 2, 4, 7), 422: (1, 4), 444: (0, 1, 2, 3, 4, 5), such as Fig.18 shown.

[0596] The unavailable luminance / chrominance samples used to derive the MLR model can be filled in from the available reconstructed samples. Fig.18 The 6-tap (0, 1, 2, 3, 4, 5) filter shown, for the CU located at the left picture boundary, the left column including (0, 3) is unavailable (outside the picture boundary), so (0, 3) is repeatedly padded from (1, 4) to apply the 6-tap filter. Note that the padding process is applied in both the training data (top / left neighboring reconstructed luma / chroma samples) and the test data (luma / chroma samples in the CU).

[0597] According to one or more embodiments of the present disclosure, the unavailable luma / chroma samples may be skipped and not used to derive the MLR model. Then, no filling process is required for the unavailable luma / chroma samples.

[0598] Note that the drawings in this disclosure can be combined with all the examples mentioned in this disclosure.

[0599] Note that the disclosed methods can be applied independently or in combination.

[0600] CCLM / MMLM with LDL decomposition

[0601] CCCM requires LDL decomposition to calculate the model parameters of the CCCM model, which avoids the use of square root operations and only requires integer operations. In this section, CCLM / MMLM with LDL decomposition is proposed. LDL decomposition can also be used for ELM / FLM / GLM, as described in other embodiments.

[0602] Please note that the methods / examples in this section can be combined / reused with the methods mentioned in other embodiments, including but not limited to classification, filter shape, matrix derivation (through special processing), applied regions, and syntax. In addition, the methods / examples listed in this section can also be applied to other embodiments (e.g., with more taps) to obtain better performance at a certain complexity trade-off.

[0603] In this disclosure, reference samples / training templates / reconstructed neighborhoods generally refer to luma samples used to derive MLR model parameters, which are then applied to internal luma samples in a CU to predict chroma samples in the CU. CCLM / MMLM with extended range

[0604] One or more reference points can be used for CCLM / MMLM prediction, i.e. Fig.15 As shown, the reference region can be the same as the reference region in CCCM. Different reference regions can be used for CCLM / MMLM prediction based on previous decoding information at TB / CB / slice / picture / sequence level.

[0605] Although the training data of multiple reference regions can fit the calculation of model parameters well, in some cases, the training data does not capture all the characteristics of the test data, which may lead to overfitting and may not predict the test data (i.e., the chroma block samples to be predicted) well. In addition, different reference regions can adapt well to different video block contents, thereby achieving more accurate prediction. To solve this problem, the reference shape / number of reference regions can be predefined or signaled / switched at the SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level. A set of reference region candidates can be predefined or signaled / switched at the SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level. Different components (U / V) may have different reference region switch controls. For example, a set of reference region candidates (idx=0~4) is predefined as shown in the following table. The reference region selection of the U / V component can be switched at the PH or CU / CTU level. Different chroma types / color formats can have different predefined reference regions.

[0606]

[0607] The unavailable luma / chroma samples used to derive the MLR model can be filled in from the available reconstructed samples. Note that the filling process is applied in both the training data (top / left neighboring reconstructed luma / chroma samples) and the test data (luma / chroma samples in the CU).

[0608] According to one or more embodiments of the present disclosure, the unavailable luma / chroma samples may be skipped and not used to derive the MLR model. Then, no filling process is required for the unavailable luma / chroma samples.

[0609] Note that the drawings in this disclosure can be combined with all the examples mentioned in this disclosure.

[0610] Note that the disclosed methods can be applied independently or in combination.

[0611] FLM / GLM / ELM / CCCM with minimum sample point restriction

[0612] FLM needs to process the downsampled luma reference and calculate the model parameters, which increases the burden on the decoder processing cycle, especially for smaller blocks. In this section, FLM with minimum sample restriction is proposed, for example, FLM is only used for samples greater than a predefined number (such as 64, 128). To this end, one or more different restrictions can be used, for example, in a single model, FLM is only used for samples greater than a predefined number (such as 256), while in a multi-model, FLM is only used for samples greater than a predefined number (such as 128).

[0613] According to one or more embodiments of the present disclosure, the number of predefined minimum sample points of a single model may be greater than or equal to the number of predefined minimum sample points of a multi-model. For example, in a single model, FLM / GLM / ELM / CCCM is only used for sample points greater than or equal to a predefined number (e.g., 128), while in a multi-model, FLM / GLM / ELM / CCCM is only used for sample points greater than or equal to a predefined number (e.g., 256).

[0614] According to one or more embodiments of the present disclosure, the number of predefined minimum sample points of FLM / GLM / ELM may be greater than or equal to the number of predefined minimum sample points of CCCM. For example, in a single model, CCCM is only used for sample points greater than or equal to a predefined number (such as 0), while in a multi-model, CCCM is only used for sample points greater than or equal to a predefined number (such as 128). In a single model, FLM is only used for sample points greater than or equal to a predefined number (such as 128), while in a multi-model, FLM is only used for sample points greater than or equal to a predefined number (such as 256).

[0615] The methods / examples in this section can be combined / reused with the methods mentioned in other embodiments, including but not limited to classification, filter shape, matrix derivation (through special processing), applied area, syntax. In addition, the methods / examples listed in this section can also be applied to other embodiments (e.g., with more taps) to obtain better performance at a certain complexity trade-off.

[0616] Fig.26 2 is a flowchart illustrating a video decoding method 2600 according to some embodiments of the present disclosure. The method 2600 may be performed by a video decoder (eg, the video decoder 30). Fig.26 As shown, method 2600 includes steps 2602 , 2604 , 2606 , and 2608 .

[0617] In step 2602, the video decoder obtains a coding unit in the current picture from a bitstream. In some embodiments, the coding unit includes a luma block and a chroma block. In some embodiments, the bitstream includes encoded video information and is stored in a computer-readable storage medium. In some embodiments, the bitstream includes data associated with the coding unit in the current picture.

[0618] In step 2604, the video decoder obtains the reconstructed luma samples in the luma block. In some embodiments, the reconstructed luma samples in the luma block may be obtained from adder 90, in-loop filter 91 or DPB 92.

[0619] In step 2606, the video decoder determines one or more cross-component prediction models based on the block size. In some embodiments, the block size is the size of the luma block or the size of a reconstructed neighboring block located on top of or to the left of the luma block. In some embodiments, determining (2606) one or more cross-component prediction models based on the block size includes: comparing the block size to a size limit; and determining the one or more cross-component prediction models based on the comparison result.

[0620] In step 2608, the video decoder applies at least one of the one or more cross-component prediction models to at least the reconstructed luma samples to predict chroma samples in the chroma block. In some embodiments, the reconstructed luma samples are co-located with the chroma samples.

[0621] In some embodiments, one or more cross-component prediction models include at least one model selected from the group consisting of a filter linear model (FLM), a gradient linear model (GLM), an edge classification linear model (ELM), and a convolutional cross-component model (CCCM). For example, the ELM includes the embodiments described above. The FLM includes the embodiments described above. The GLM includes the embodiments described above.

[0622] In some embodiments, the size limit includes a first value and a second value greater than the first value. In some embodiments, determining one or more cross-component prediction models based on the result of the comparison may include: in response to the block size being greater than or equal to the first value, selecting a single cross-component prediction model from multiple cross-component prediction models as one or more cross-component prediction models. Multiple cross-component prediction models may include at least one model selected from the group consisting of a filter linear model (FLM), a gradient linear model (GLM), an edge classification linear model (ELM), and a convolution cross-component model (CCCM). In some embodiments, determining one or more cross-component prediction models based on the result of the comparison may include: in response to the block size being greater than or equal to the second value, determining multiple cross-component prediction models as one or more cross-component prediction models. Multiple cross-component prediction models may include at least one model selected from the group consisting of FLM, GLM, ELM, and CCCM. The first value and the second value represent the minimum sample point limit for a single model and multiple models, respectively. For example, for a single model, FLM / GLM / ELM / CCCM is only used for samples greater than or equal to a predefined number (such as a first value). For another example, for multiple models, FLM / GLM / ELM / CCCM is only used for sample points greater than a predefined number (such as the second value).

[0623] In some embodiments, the second value is greater than the first value. For example, the second value is 256 and the first value is 128. In some variants, the second value is less than the first value. In some variants, the second value is equal to the first value. By setting the minimum sample point limit, the burden of the codec processing cycle can be reduced, thereby improving the codec efficiency.

[0624] In some embodiments, when one or more cross-component prediction models include a single cross-component prediction model, the size limit includes a third value and a fourth value greater than the third value. In some embodiments, determining one or more cross-component prediction models based on the result of the comparison may include: in response to the block size being greater than or equal to the third value, selecting CCCM as a single cross-component prediction model. In some embodiments, determining one or more cross-component prediction models based on the result of the comparison may include: in response to the block size being greater than or equal to the fourth value, selecting one of FLM, GLM or ELM as a single cross-component prediction model. The third value and the fourth value represent the minimum sample point limits of CCCM and FLM / GLM / ELM, respectively. For example, CCCM is only used for samples greater than or equal to a predefined number (such as the third value). For example, FLM / GLM / ELM is only used for samples greater than or equal to a predefined number (such as the fourth value).

[0625] In some embodiments, the fourth value is greater than the third value. In some variations, the fourth value may be less than or equal to the third value.

[0626] In some embodiments, when the one or more cross-component prediction models include multiple cross-component prediction models, the size limit includes a fifth value and a sixth value greater than the fifth value, the fifth value is greater than the third value, and the sixth value is greater than the fourth value. In some embodiments, determining the one or more cross-component prediction models based on the result of the comparison may further include: in response to the block size being greater than or equal to the fifth value, selecting CCCM as one of the one or more cross-component prediction models. In some embodiments, determining the one or more cross-component prediction models based on the result of the comparison may further include: in response to the block size being greater than or equal to the sixth value, selecting one or more of FLM, GLM, or ELM as the one or more cross-component prediction models.

[0627] In some examples, when the one or more cross-component prediction models include only one cross-component prediction model, the third value is 0 and the fourth value is 128. For example, in a single model, CCCM is only used for samples greater than or equal to 0. In a single model, FLM / GLM / ELM is only used for samples greater than or equal to 128.

[0628] In some examples, when the one or more cross-component prediction models include multiple cross-component prediction models, the fifth value is 128 and the sixth value is 256. For example, in the multi-model, CCCM is only used for samples greater than or equal to 128. In the multi-model, FLM / GLM / ELM is only used for samples greater than or equal to 256.

[0629] In some variations, the sixth value may be less than or equal to the fifth value.

[0630] In some variations, the fifth value is less than or equal to the third value, and the sixth value is less than or equal to the fourth value.

[0631] In some embodiments, the size limit is predefined or signaled at the sequence parameter set (SPS), decoding parameter set (DPS), video parameter set (VPS), supplemental enhancement information (SEI), adaptation parameter set (APS), picture parameter set (PPS), picture header (PH), slice header (SH), region, coding tree unit (CTU), coding unit (CU), sub-unit or sample level.

[0632] In some embodiments, determining (2606) one or more cross-component prediction models includes: deriving a classifier based on local information of the luma block, the local information including at least one of a local binary pattern or edge information of the luma block; dividing neighboring samples located at the top or left of the luma block into a plurality of groups based on the classifier; and deriving different cross-component prediction models for different groups in the plurality of groups based on the block size. For example, the classifier may include classifiers C0, C1, C2, C3, or a combination thereof described in the embodiments of the ELM in the present disclosure.

[0633] In some embodiments, applying (2608) at least one of the one or more cross-component prediction models to at least the reconstructed luma samples includes: classifying the reconstructed luma samples into a first group of a plurality of groups based on a classifier; and applying a corresponding cross-component prediction model for the first group to the reconstructed luma samples. For example, after determining the classifier and the corresponding model for each group, the reconstructed luma samples are classified using the classifier. The corresponding model is then applied to the reconstructed luma samples to predict co-located chroma samples.

[0634] In some embodiments, determining (2606) one or more cross-component prediction models includes: determining at least one filter parameter of a luma filter, the filter parameters including a filter shape and a number of filter taps of the luma filter; obtaining multiple groups of neighboring samples of the coding unit based on the at least one filter parameter, wherein each of the multiple groups of neighboring samples is located at the top or left side of the coding unit, each of the multiple groups of neighboring samples includes neighboring chroma samples and at least one neighboring luma sample corresponding to the neighboring chroma samples, and the at least one neighboring luma sample is arranged in the current picture according to the filter shape of the luma filter; and further deriving one or more cross-component prediction models based on the multiple groups of neighboring samples.

[0635] In some embodiments, deriving one or more cross-component prediction models based on multiple sets of neighboring samples includes: constructing a linear equation based on multiple sets of neighboring samples, wherein the linear equation describes a mapping from a sample value of a luma sample to a sample value of a chroma sample; and solving the linear equation by at least one of the following algorithms to derive coefficients of the one or more cross-component prediction models: pseudo-inverse matrix calculation, adjoint matrix calculation, Gauss-Jordan elimination, or Cholesky decomposition. For example, the linear equation may be constructed according to the method described in the embodiments regarding FLM.

[0636] In some embodiments, determining (2606) one or more cross-component prediction models further includes adjusting the filter shape and reducing the number of taps based on a pre-operation; and determining one or more cross-component prediction models based on the adjusted filter shape and reduced number of taps of the luma filter. For example, the pre-operations described in the embodiments of the GLM (e.g., pre-linear weighting, sign, scaling / absolute value, threshold, ReLU) can be used to reduce the number of filter taps or adjust the filter shape. For example, instead of applying 2 taps on 2 luma samples, the 2 luma samples can be pre-linearly weighted and then a simpler 1 tap can be applied to reduce complexity.

[0637] In some embodiments, applying (2608) at least one of the one or more cross-component prediction models to at least the reconstructed luma samples includes applying at least one of the one or more cross-component prediction models to the reconstructed luma samples and to reconstructed neighboring samples located at the top or to the left of the luma block. For example, instead of using only the reconstructed co-located luma samples, the co-located luma samples and the neighboring luma samples may be used to predict the chroma samples to capture the inter-sample correlation between the co-located luma samples, the neighboring luma samples, and the chroma samples.

[0638] Fig. 27 2 is a flowchart illustrating a video encoding method 2700 according to some embodiments of the present disclosure. The method 2700 may be performed by a video encoder (eg, the video encoder 20). Fig. 27 As shown, method 2700 includes steps 2702 , 2704 , 2706 , and 2708 .

[0639] In step 2702, the video encoder partitions the video frame into a plurality of coding units. In some embodiments, a coding unit in the plurality of coding units includes a luma block and a chroma block.

[0640] In step 2704, the video encoder obtains the reconstructed luma samples in the luma block. In some embodiments, the reconstructed luma samples in the luma block may be obtained from adder 62, in-loop filter 63, or DPB 64.

[0641] In step 2706, the video encoder determines one or more cross-component prediction models based on the block size. In some embodiments, the block size is the size of the luma block or the size of a reconstructed neighboring block located on top of or to the left of the luma block. In some embodiments, determining (2706) one or more cross-component prediction models based on the block size includes: comparing the block size to a size limit; and determining the one or more cross-component prediction models based on the comparison result.

[0642] In step 2708, the video encoder applies at least one of the one or more cross-component prediction models to at least the reconstructed luma samples to predict chroma samples in the chroma block.

[0643] In some embodiments, the size limit includes a first value and a second value greater than the first value. In some embodiments, determining one or more cross-component prediction models based on the result of the comparison may include: in response to the block size being greater than or equal to the first value, selecting a single cross-component prediction model from multiple cross-component prediction models as one or more cross-component prediction models. Multiple cross-component prediction models may include at least one model selected from the group consisting of a filter linear model (FLM), a gradient linear model (GLM), an edge classification linear model (ELM), and a convolutional cross-component model (CCCM). In some embodiments, determining one or more cross-component prediction models based on the result of the comparison may include: in response to the block size being greater than or equal to the second value, determining multiple cross-component prediction models as one or more cross-component prediction models. Multiple cross-component prediction models may include at least one model selected from the group consisting of FLM, GLM, ELM, and CCCM.

[0644] In some embodiments, when the one or more cross-component prediction models include a single cross-component prediction model, the size limit includes a third value and a fourth value greater than the third value. In some embodiments, determining the one or more cross-component prediction models based on the result of the comparison may include: in response to the block size being greater than or equal to the third value, selecting a convolutional cross-component model (CCCM) as the single cross-component prediction model. In some embodiments, determining the one or more cross-component prediction models based on the result of the comparison may include: in response to the block size being greater than or equal to the fourth value, selecting one of a filter linear model (FLM), a gradient linear model (GLM), or an edge classification linear model (ELM) as the single cross-component prediction model.

[0645] In some embodiments, when the one or more cross-component prediction models include multiple cross-component prediction models, the size limit includes a fifth value and a sixth value greater than the fifth value, the fifth value is greater than the third value, and the sixth value is greater than the fourth value. In some embodiments, determining the one or more cross-component prediction models based on the result of the comparison may further include: in response to the block size being greater than or equal to the fifth value, selecting CCCM as one of the one or more cross-component prediction models. In some embodiments, determining the one or more cross-component prediction models based on the result of the comparison may further include: in response to the block size being greater than or equal to the sixth value, selecting one or more of FLM, GLM, or ELM as the one or more cross-component prediction models.

[0646] Fig.28 A computing environment 2810 coupled to a user interface 2850 is shown. The computing environment 2810 may be part of a data processing server. The computing environment 2810 includes a processor 2820, a memory 2830, and an input / output (I / O) interface 2840.

[0647] The processor 2820 generally controls the overall operation of the computing environment 2810, such as operations associated with display, data acquisition, data communication, and image processing. The processor 2820 may include one or more processors for executing instructions to perform all or some steps in the above method. In addition, the processor 2820 may include one or more modules that facilitate the interaction between the processor 2820 and other components. The processor may be a central processing unit (CPU), a microprocessor, a single-chip microcomputer, a graphics processing unit (GPU), etc.

[0648] The memory 2830 is configured to store various types of data to support the operation of the computing environment 2810. The memory 2830 may include predetermined software 2832. Examples of such data include instructions for any application or method operating on the computing environment 2810, video data sets, image data, etc. The memory 2830 may be implemented by using any type of volatile or non-volatile memory device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0649] I / O interface 2840 provides an interface between processor 2820 and peripheral interface modules (such as keyboard, click wheel, buttons, etc.). Buttons may include but are not limited to a home button, a start scan button, and a stop scan button. I / O interface 2840 may be coupled to an encoder and a decoder.

[0650] In an embodiment, a non-transitory computer-readable storage medium is also provided. The non-transitory computer-readable storage medium includes multiple programs in the memory 2830, and the multiple programs can be executed by the processor 2820 in the computing environment 2810 to perform the above method. In one example, the multiple programs can be executed by the processor 2820 in the computing environment 2810 to receive (for example, from Figure 2 The video encoder 20 in the embodiment of the present invention includes a bit stream or data stream of encoded video information (e.g., video blocks representing encoded video frames and / or one or more associated syntax elements, etc.), and can also be executed by the processor 2820 in the computing environment 2810 to perform the above-mentioned decoding method according to the received bit stream or data stream. In some example embodiments, multiple programs can be executed by the processor 2820 in the computing environment 2810 to perform the above-mentioned encoding method to encode video information (e.g., video blocks representing video frames and / or one or more associated syntax elements, etc.) into a bit stream or data stream, and can also be executed by the processor 2820 in the computing environment 2810 to transmit the bit stream or data stream (e.g., to Figure 3Alternatively, a non-transitory computer-readable storage medium may have stored therein a bitstream or data stream, the bitstream or data stream comprising the video decoder 30 in the encoder (e.g., Figure 2 The video encoder 20 in FIG. 1 uses the encoded video information (e.g., video blocks representing encoded video frames and / or one or more associated syntax elements, etc.) generated by the encoding method described above for use by a decoder (e.g., Figure 3 The video decoder 30 in the video decodes the video data. The non-transitory computer-readable storage medium may be, for example, a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0651] In an embodiment, a computing device is also provided, which includes: one or more processors (e.g., processor 2820); and a non-transitory computer-readable storage medium or memory 2830, in which a plurality of programs executable by the one or more processors are stored, wherein the one or more processors are configured to perform the above-mentioned method when executing the plurality of programs.

[0652] In an embodiment, a computer program product is also provided, which includes multiple programs in the memory 2830, for example, and the multiple programs can be executed by the processor 2820 in the computing environment 2810 to perform the above method. For example, the computer program product may include a non-transitory computer-readable storage medium.

[0653] In an embodiment, the computing environment 2810 may be implemented with one or more ASICs, DSPs, digital signal processing devices (DSPDs), programmable logic devices (PLDs), FPGAs, GPUs, controllers, microcontrollers, microprocessors, or other electronic components to perform the above methods.

[0654] The description of the present disclosure has been presented for illustrative purposes and is not intended to be exhaustive or limited to the present disclosure. Many modifications, variations, and alternative embodiments will be apparent to one of ordinary skill in the art having the benefit of the teachings presented in the foregoing description and the associated drawings.

[0655] Unless otherwise specifically stated, the order of steps of the method according to the present disclosure is intended to be illustrative only, and the steps of the method according to the present disclosure are not limited to the order specifically described above, but can be changed according to actual conditions. In addition, at least one step of the method according to the present disclosure can be adjusted, combined or deleted according to actual needs.

[0656] Examples are chosen and described in order to explain the principles of the present disclosure and to enable other persons skilled in the art to understand the various embodiments of the present disclosure and to best utilize the basic principles and various embodiments with various modifications suitable for the specific use contemplated. Therefore, it should be understood that the scope of the present disclosure should not be limited to the specific examples of embodiments described herein, and modifications and other embodiments are intended to be included within the scope of the present disclosure.

Claims

1. A video decoding method, comprising: Obtaining a coding unit in a current picture from a bitstream, wherein the coding unit includes a luminance block and a chrominance block; Obtaining reconstructed luminance samples in the luminance block; determining one or more cross-component prediction models based on a block size, wherein the block size is a size of the luma block or a size of a reconstructed neighboring block located on top or to the left of the luma block; and At least one of the one or more cross-component prediction models is applied to at least the reconstructed luma samples to predict chroma samples in the chroma block.

2. The method according to claim 1, wherein: Determining one or more cross-component prediction models based on the block size includes: comparing the block size to a size limit; and The one or more cross-component prediction models are determined based on a result of the comparing.

3. The method according to claim 2, wherein: The size limit includes a first value and a second value greater than the first value, and Wherein, determining the one or more cross-component prediction models based on the comparison result includes: In response to the block size being greater than or equal to the first value, selecting a single cross-component prediction model from the multiple cross-component prediction models as the one or more cross-component prediction models, wherein the multiple cross-component prediction models include at least one model selected from the group consisting of a filter linear model (FLM), a gradient linear model (GLM), an edge classification linear model (ELM), or a convolutional cross-component model (CCCM); or In response to the block size being greater than or equal to the second value, a plurality of cross-component prediction models are determined as the one or more cross-component prediction models, wherein the plurality of cross-component prediction models include at least one model selected from the group consisting of FLM, GLM, ELM, or CCCM.

4. The method according to claim 2, wherein: When the one or more cross-component prediction models include a single cross-component prediction model, the size limit includes a third value and a fourth value greater than the third value, and Wherein, determining the one or more cross-component prediction models based on the comparison result includes: In response to the block size being greater than or equal to the third value, selecting a convolutional cross-component model (CCCM) as the single cross-component prediction model; or In response to the block size being greater than or equal to the fourth value, one of a filter linear model (FLM), a gradient linear model (GLM), or an edge classification linear model (ELM) is selected as the single cross-component prediction model.

5. The method according to claim 4, wherein: When the one or more cross-component prediction models include a plurality of cross-component prediction models, the size limit includes a fifth value and a sixth value greater than the fifth value, the fifth value is greater than the third value, and the sixth value is greater than the fourth value, and Wherein, determining the one or more cross-component prediction models based on the comparison result further comprises: In response to the block size being greater than or equal to the fifth value, selecting CCCM as one of the one or more cross-component prediction models; or In response to the block size being greater than or equal to the sixth value, one or more of FLM, GLM, or ELM are selected as the one or more cross-component prediction models.

6. The method according to claim 1, wherein: Determining one or more cross-component prediction models includes: deriving a classifier based on local information of the luminance block, wherein the local information comprises at least one of a local binary pattern or edge information of the luminance block; dividing the neighboring samples located at the top or left of the luminance block into a plurality of groups based on the classifier; and Different cross-component prediction models are derived for different groups of the plurality of groups based on the block size.

7. The method according to claim 1, wherein: Determining one or more cross-component prediction models includes: determining at least one filter parameter of a luminance filter, wherein the filter parameter comprises a filter shape and a number of filter taps of the luminance filter; obtaining a plurality of groups of neighboring samples of the coding unit based on the at least one filter parameter, wherein each of the plurality of groups of neighboring samples is located at a top or a left side of the coding unit and includes neighboring chroma samples and at least one neighboring luma sample corresponding to the neighboring chroma samples, and the at least one neighboring luma sample is arranged in the current picture according to a filter shape of the luma filter; and The one or more cross-component prediction models are further derived based on the multiple groups of neighboring samples.

8. A video encoding method, comprising: Partitioning a video frame into a plurality of coding units, wherein a coding unit in the plurality of coding units includes a luma block and a chroma block; Obtaining reconstructed luminance samples in the luminance block; determining one or more cross-component prediction models based on a block size, wherein the block size is a size of the luma block or a size of a reconstructed neighboring block located on top or to the left of the luma block; and At least one of the one or more cross-component prediction models is applied to at least the reconstructed luma samples to predict chroma samples in the chroma block.

9. The method according to claim 1, wherein: Determining one or more cross-component prediction models based on the block size includes: comparing the block size to a size limit; and The one or more cross-component prediction models are determined based on a result of the comparing.

10. The method according to claim 9, wherein: The size limit includes a first value and a second value greater than the first value, and Wherein, determining the one or more cross-component prediction models based on the comparison result includes: In response to the block size being greater than or equal to the first value, selecting a single cross-component prediction model from the multiple cross-component prediction models as the one or more cross-component prediction models, wherein the multiple cross-component prediction models include at least one model selected from the group consisting of a filter linear model (FLM), a gradient linear model (GLM), an edge classification linear model (ELM), or a convolutional cross-component model (CCCM); or In response to the block size being greater than or equal to the second value, a plurality of cross-component prediction models are determined as the one or more cross-component prediction models, wherein the plurality of cross-component prediction models include at least one model selected from the group consisting of FLM, GLM, ELM, or CCCM.

11. The method according to claim 9, wherein: When the one or more cross-component prediction models include a single cross-component prediction model, the size limit includes a third value and a fourth value greater than the third value, and Wherein, determining the one or more cross-component prediction models based on the comparison result includes: In response to the block size being greater than or equal to the third value, selecting a convolutional cross-component model (CCCM) as the single cross-component prediction model; or In response to the block size being greater than or equal to the fourth value, one of a filter linear model (FLM), a gradient linear model (GLM), or an edge classification linear model (ELM) is selected as the single cross-component prediction model.

12. The method according to claim 11, wherein: When the one or more cross-component prediction models include a plurality of cross-component prediction models, the size limit includes a fifth value and a sixth value greater than the fifth value, the fifth value is greater than the third value, and the sixth value is greater than the fourth value, and Wherein, determining the one or more cross-component prediction models based on the comparison result further comprises: In response to the block size being greater than or equal to the fifth value, selecting CCCM as one of the one or more cross-component prediction models; or In response to the block size being greater than or equal to the sixth value, one or more of FLM, GLM, or ELM are selected as the one or more cross-component prediction models.

13. The method according to claim 8, wherein: Determining one or more cross-component prediction models includes: deriving a classifier based on local information of the luminance block, wherein the local information comprises at least one of a local binary pattern or edge information of the luminance block; dividing the neighboring samples located at the top or left of the luminance block into a plurality of groups based on the classifier; and Different cross-component prediction models are derived for different groups of the plurality of groups based on the block size.

14. The method according to claim 8, wherein: Determining one or more cross-component prediction models includes: determining at least one filter parameter of a luminance filter, wherein the filter parameter comprises a filter shape and a number of filter taps of the luminance filter; obtaining a plurality of groups of neighboring samples of the coding unit based on the at least one filter parameter, wherein each of the plurality of groups of neighboring samples is located at a top or a left side of the coding unit and includes neighboring chroma samples and at least one neighboring luma sample corresponding to the neighboring chroma samples, and the at least one neighboring luma sample is arranged in the current picture according to a filter shape of the luma filter; and The one or more cross-component prediction models are further derived based on the multiple groups of neighboring samples.

15. An electronic device comprising: one or more processors; a memory coupled to the one or more processors; as well as Multiple programs stored in the memory, when executed by the one or more processors, enable the electronic device to receive a video bitstream to execute the method as described in any one of claims 1 to 7, or enable the electronic device to execute the method as described in any one of claims 8 to 14 to generate a video bitstream.

16. A non-transitory computer-readable storage medium storing a plurality of programs for execution by an electronic device having one or more processors, wherein: When executed by the one or more processors, the multiple programs cause the electronic device to execute the method described in any one of claims 1 to 7 to process a video bitstream and store the processed video bitstream in the non-transitory computer-readable storage medium, or cause the electronic device to execute the method described in any one of claims 8 to 14 to generate a video bitstream and store the generated video bitstream in the non-transitory computer-readable storage medium.

17. A computer program product comprising instructions which, when executed by one or more processors of an electronic device, cause the electronic device to receive a video bitstream to execute the method of any one of claims 1 to 7, or cause the electronic device to execute the method of any one of claims 8 to 14 to generate a video bitstream.

Images