Method and apparatus for encoding video data

Abstract

A method and apparatus of encoding video data. The method includes determining whether an intra-chroma prediction mode for a coding unit is one of a multi-cross-component prediction (PMC) mode, deriving an intermediate prediction of a chroma component of the coding unit according to a linear model applied to a reconstruction of a luma component of the coding unit, in case that the intra-chroma prediction mode is one of the PMC mode, applying a clipping operation to an output of the intermediate prediction of the chroma component to limit values of the output of the intermediate prediction within a predetermined range, and signaling a semantic element, wherein the semantic element indicates whether the intra-chroma prediction mode is one of the PMC mode.

Description

[0001] This application is a divisional application of the invention patent application filed on June 3, 2021, with application number "202180003284.8" and titled "Cyclone Encoding and Decoding Enhancement in Multi-Span Component Prediction (PMC) Mode". Technical Field

[0002] This application generally relates to video encoding and decoding and compression, and more specifically, to methods and apparatus for improving the efficiency of chroma encoding and decoding. Background Technology

[0003] Various electronic devices (such as digital televisions, laptops or desktop computers, tablets, digital cameras, digital recording devices, digital media players, video game consoles, smartphones, video conferencing equipment, video streaming devices, etc.) support digital video. Electronic devices transmit, receive, encode, decode, and / or store digital video data by implementing video compression / decompression standards. Some well-known video codec standards include Universal Video Coding (VVC), High Efficiency Video Coding (HEVC, also known as H.265 or MPEG-H Part 2), and Advanced Video Coding (AVC, also known as H.264 or MPEG-4 Part 10), jointly developed by ISO / IEC MPEG and ITU-T TVCEG. AO Media Video 1 (AV1) was developed by the Alliance for Open Media (AOM) as a successor to its previous standard VP9. Audio and Video Coding (AVS) refers to digital audio and digital video compression standards, a series of video compression standards developed by the China Audio and Video Coding Standards Working Group.

[0004] Video compression typically involves performing spatial (intra-frame) prediction and / or temporal (inter-frame) prediction to reduce or remove redundancy inherent in video data. For block-based video coding, video frames are divided into one or more stripes, each strip containing multiple video blocks, also known as coding tree units (CTUs). Each CTU may contain one coding unit (CU) or be recursively split into smaller CUs until a predefined minimum CU size is reached. Each CU (also called a leaf CU) contains one or more transform units (TUs) and each CU also contains one or more prediction units (PUs). Each CU can be coded in intra-frame, inter-frame, or IBC modes. Video blocks in an intra-frame coded (I) strip of a video frame are coded using spatial prediction with respect to reference samples in neighboring blocks within the same video frame. Video blocks in an inter-frame coded (P or B) strip of a video frame can be coded using spatial prediction with respect to reference samples in neighboring blocks within the same video frame or temporal prediction with respect to reference samples in other previous and / or future reference video frames.

[0005] A prediction block for the current video block to be encoded is derived based on spatial or temporal prediction of previously encoded reference blocks (e.g., neighboring blocks). The process of finding the reference block can be accomplished using a block-matching algorithm. The residual data representing the pixel difference between the current block to be encoded and the prediction block is called the residual block or prediction error. Inter-frame coded blocks are encoded based on the residual block and the motion vector pointing to the reference block forming the prediction block in the reference frame. The process of determining the motion vector is typically called motion estimation. Intra-frame coded blocks are encoded based on the intra-frame prediction mode and the residual block. For further compression, the residual block is transformed from the pixel domain to the transform domain (e.g., the frequency domain) to obtain residual transform coefficients, which can then be quantized. The quantized transform coefficients, initially arranged in a two-dimensional array, can be scanned to generate a one-dimensional vector of transform coefficients, which is then entropy-encoded into the video bitstream for even greater compression.

[0006] The encoded video bitstream is then stored in a computer-readable storage medium (e.g., flash memory) for access by another electronic device with digital video capabilities or transmitted directly to the electronic device via wired or wireless means. The electronic device then performs video decompression (a process opposite to the video compression described above), for example, by parsing the encoded video bitstream to obtain semantic elements from the bitstream, and reconstructs the digital video data from the encoded video bitstream to its original format based at least in part on the semantic elements obtained from the bitstream, and the electronic device presents the reconstructed digital video data on its display.

[0007] As digital video quality evolves from high definition to 4K×2K or even 8K×4K, the amount of video data to be encoded / decoded grows exponentially. Maintaining image quality while efficiently encoding / decoding video data remains a long-standing challenge. Summary of the Invention

[0008] This application describes implementations related to video data encoding and decoding, and more specifically, describes implementations related to methods and apparatus for improving the encoding and decoding efficiency of chroma encoding and decoding, including: 1) reducing the complexity of multi-span component prediction (PMC) modes; 2) improving the encoding and decoding efficiency of PMC modes by exploring the sign and scaling relationship between the luminance prediction Cb residual and the luminance prediction Cr residual.

[0009] According to a first aspect of this application, a method for encoding video data includes: determining whether an intra-frame chroma prediction mode for a coding unit is one of the multi-span component prediction (PMC) modes; if the intra-frame chroma prediction mode is determined to be one of the PMC modes, deriving an intermediate prediction of the chroma component of the coding unit based on a linear model of the reconstruction of the luminance component applied to the coding unit; applying a cropping operation to the output of the intermediate prediction of the chroma component to limit the value of the output of the intermediate prediction within a predetermined range; and signaling a semantic element, wherein the semantic element indicates whether the intra-frame chroma prediction mode is one of the PMC modes.

[0010] In some embodiments, the intermediate prediction block of the chroma component of the coding unit is derived according to the following formula:

[0011] IPred = A·Rec Y +B

[0012] Where IPred represents the intermediate prediction of the chromaticity component having the same dimension as the luminance component, and Rec Y This represents the reconstruction of the luminance component, and A and B are the linear parameters of the linear model.

[0013] In some embodiments, the predetermined range is 0 to 2. BitDepth+1 -1, where BitDepth represents the bit depth of the chroma component.

[0014] In some embodiments, the predetermined range is 0 to 2. BitDepth -1, where BitDepth represents the bit depth of the chroma component.

[0015] In some embodiments, the predetermined range is 0 to 2. 15 -1.

[0016] In some embodiments, the intermediate prediction block of the chroma component of the coding unit is derived according to the following formula:

[0017] IPred = S × (A·Rec) Y +B)

[0018] Where IPred represents the intermediate prediction of the chromaticity component having the same dimension as the luminance component, and Rec Y The luminance component is reconstructed, A and B are the linear parameters of the linear model, and S represents the positive or negative sign.

[0019] In some embodiments, deriving intermediate prediction blocks of the chrominance components of the coding unit further includes: downsampling the output of IPred after the pruning operation; and obtaining the final prediction FPred of the Cr chrominance components according to the following formula. Cr :

[0020] FPred Cr =IPred′-S×w×Rec Cb

[0021] Where IPred' represents the downsampled output of IPred after the cropping operation, Rec Cb is the reconstructed Cb chromaticity component within the coding unit, and w is a positive weighting parameter.

[0022] In some embodiments, the predetermined range of the clipping operation is -2. BitDepth+1 to 2 BitDepth+1 -1, where BitDepth represents the bit depth of the chroma component.

[0023] In some embodiments, deriving intermediate prediction blocks of the chroma components of the coding unit further includes: determining whether there is a positive or negative correlation between the prediction residuals of chroma components Cb and Cr; signaling an intra-frame chroma PMC mode form flag, wherein the intra-frame chroma PMC mode form flag indicates a positive correlation between the prediction residuals of chroma components Cb and Cr by assigning a negative sign to S, or indicates a negative correlation between the prediction residuals of chroma components Cb and Cr by assigning a positive sign to S; determining the value of the positive weight parameter w based on the intra-frame chroma prediction mode; and signaling intra-frame chroma index semantics, wherein the intra-frame chroma index semantics indicates the value of the positive weight parameter w.

[0024] In some embodiments, the value of the positive weight parameter w is derived from a set of 2... n The value is selected from the table, where n is an integer.

[0025] According to a second aspect of this application, an electronic device includes one or more processing units, a memory, and a plurality of programs stored in the memory. When executed by one or more processing units, the programs cause the electronic device to perform the method of encoding video data as described above.

[0026] According to a third aspect of this application, a non-transitory computer-readable storage medium stores a plurality of programs for execution by an electronic device having one or more processor units. When executed by one or more processing units, the programs cause the electronic device to perform a method for encoding video data as described above. Attached Figure Description

[0027] The accompanying drawings, included to provide a further understanding of the embodiments and incorporated herein as a part of the specification, illustrate the described embodiments and, together with the description, serve to explain the underlying principles. Similar reference numerals denote corresponding parts.

[0028] Figure 1 This is a block diagram illustrating an exemplary video encoding and decoding system according to some embodiments of the present disclosure.

[0029] Figure 2 This is a block diagram illustrating an exemplary video encoder according to some embodiments of the present disclosure.

[0030] Figure 3 This is a block diagram illustrating an exemplary video decoder according to some embodiments of the present disclosure.

[0031] Figures 4A to 4E This is a block diagram illustrating how a frame is recursively divided into multiple video blocks of different sizes and shapes according to some embodiments of the present disclosure.

[0032] Figure 5 This is a block diagram depicting the basic process of a chromaticity prediction block generation process according to some embodiments of the present disclosure.

[0033] Figure 6 This is a block diagram illustrating an example of deriving four neighboring samples according to some embodiments of the present disclosure.

[0034] Figure 7 This is a block diagram illustrating an exemplary multi-span component prediction (PMC) process according to some embodiments of the present disclosure.

[0035] Figure 8 This is a flowchart illustrating an exemplary process simplified by using a trimming operation at the output of IPred, according to some embodiments of the present disclosure. Detailed Implementation

[0036] Referring now to the detailed description, examples of which are illustrated in the accompanying drawings. In the following detailed description, numerous non-limiting details are set forth to aid in understanding the subject matter presented herein. However, it will be apparent to those skilled in the art that various alternatives may be used without departing from the scope of the claims, and that the subject matter may be practiced without these specific details. For example, it will be apparent to those skilled in the art that the subject matter presented herein can be implemented on many types of electronic devices with digital video capabilities.

[0037] The first generation of AVS standards included the Chinese national standards "Information Technology, Advanced Audio and Video Coding, Part 2: Video" (referred to as AVS1) and "Information Technology, Advanced Audio and Video Coding, Part 16: Broadcast Television Video" (referred to as AVS+). Compared to the MPEG-2 standard, the first generation of AVS standards offered approximately 50% bitrate savings while maintaining the same perceived quality. The second generation of AVS standards included the Chinese national standard series "Information Technology, High-Efficiency Multimedia Coding" (referred to as AVS2), primarily targeting the transmission of Ultra HD TV programs. AVS2's coding efficiency was twice that of AVS+. Simultaneously, the video portion of the AVS2 standard was submitted by the Institute of Electrical and Electronics Engineers (IEEE) as an international standard for application. The AVS3 standard is a new generation of video codec standards for UHD video applications, designed to surpass the coding efficiency of the latest international standard HEVC, which offers approximately 30% bitrate savings. In March 2019, at the 68th AVS meeting, the AVS3-P2 baseline was completed, offering approximately 30% bitrate savings relative to the HEVC standard. Currently, there is a reference software called the High Performance Model (HPM), which is maintained by the AVS group to demonstrate a reference implementation of the AVS3 standard. Like HEVC, the AVS3 standard is built on a block-based hybrid video codec framework.

[0038] Figure 1 This is a block diagram illustrating an exemplary system 10 for encoding and decoding video blocks in parallel, according to some embodiments of the present disclosure. Figure 1 As shown, system 10 includes source device 12, which generates and encodes video data that will later be decoded by target device 14. Source device 12 and target device 14 can include any electronic device from a wide variety of electronic devices, including desktop or laptop computers, tablet computers, smartphones, set-top boxes, digital televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, etc. In some embodiments, source device 12 and target device 14 are equipped with wireless communication capabilities.

[0039] In some implementations, target device 14 may receive encoded video data to be decoded via link 16. Link 16 may include any type of communication medium or device capable of moving encoded video data from source device 12 to target device 14. In one example, link 16 may include a communication medium enabling source device 12 to transmit encoded video data directly to target device 14 in real time. The encoded video data may be modulated according to communication standards, such as wireless communication protocols, and transmitted to target device 14. The communication medium may include any wireless or wired communication medium, such as radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium may form part of a packet-based network, such as a local area network, a wide area network, or a global network such as the Internet. The communication medium may include a router, switch, base station, or any other means that may facilitate communication from source device 12 to target device 14.

[0040] In some other embodiments, encoded video data may be sent from output interface 22 to storage device 32. The encoded video data in storage device 32 may then be accessed by target device 14 via input interface 28. Storage device 32 may include any data storage medium of various distributed or locally accessed data storage media, such as hard disk drives, Blu-ray discs, DVDs, CD-ROMs, flash memory, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded video data. In another example, storage device 32 may correspond to a file server or another intermediate storage device that can hold the encoded video data generated by source device 12. Target device 14 may access the stored video data from storage device 32 via streaming or downloading. The file server may be any type of computer capable of storing and sending encoded video data to target device 14. Exemplary file servers include web servers (e.g., for websites), FTP servers, network attached storage (NAS) devices, or local disk drives. The target device 14 can access the encoded video data via any standard data connection suitable for accessing encoded video data stored on the file server, including wireless channels (e.g., Wi-Fi connections), wired connections (e.g., DSL, cable modems, etc.), or a combination of both. The transmission of encoded video data from storage device 32 can be streaming, downloading, or a combination of both.

[0041] like Figure 1As shown, source device 12 includes a video source 18, a video encoder 20, and an output interface 22. Video source 18 may include sources or combinations of such sources, such as: a video capture device (e.g., a camera), a video archive containing previously captured video, a video feed interface for receiving video from a video content provider, and / or a computer graphics system for generating computer graphics data as source video. As an example, if video source 18 is a camera in a security monitoring system, source device 12 and target device 14 may form a camera phone or video phone. However, the embodiments described in this application are generally applicable to video encoding / decoding and can be applied to wireless and / or wired applications.

[0042] Video captured, pre-captured, or computer-generated video can be encoded by video encoder 20. Encoded video data can be directly transmitted to target device 14 via output interface 22 of source device 12. Encoded video data can also (or optionally) be stored on storage device 32 for later access by target device 14 or other devices for decoding and / or playback. Output interface 22 may further include a modem and / or transmitter.

[0043] Target device 14 includes an input interface 28, a video decoder 30, and a display device 34. Input interface 28 may include a receiver and / or a modem, and receives encoded video data via link 16. The encoded video data transmitted via link 16 or provided on storage device 32 may include various semantic elements generated by video encoder 20 for use by video decoder 30 when decoding the video data. Such semantic elements may be included within the encoded video data transmitted on a communication medium, stored on a storage medium, or stored on a file server.

[0044] In some embodiments, the target device 14 may include a display device 34, which may be an integrated display device or an external display device configured to communicate with the target device 14. The display device 34 displays decoded video data to the user and may include any of a variety of display devices, such as a liquid crystal display (LCD), a plasma display, an organic light-emitting diode (OLED) display, or another type of display device.

[0045] The video encoder 20 and video decoder 30 may operate according to proprietary or industry standards (e.g., VVC, HEVC, MPEG-4, Part 10, Advanced Video Coding (AVC), AVS, or extensions of such standards). It should be understood that this application is not limited to any particular video encoding / decoding standard and is applicable to other video encoding / decoding standards. It is generally understood that the video encoder 20 of the source device 12 may be configured to encode video data according to any of these current or future standards. Similarly, it is generally understood that the video decoder 30 of the target device 14 may be configured to decode video data according to any of these current or future standards.

[0046] The video encoder 20 and video decoder 30 can be implemented as any circuit of a variety of suitable encoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or any combination thereof. When implemented in part in software, the electronic device may store instructions for software in a suitable non-transitory computer-readable medium and use one or more processors to execute the instructions in hardware to perform the video encoding / decoding operations disclosed in this disclosure. Each of the video encoder 20 and video decoder 30 may be included in one or more encoders or decoders, either of which may be integrated as part of a combined encoder / decoder (CODEC) in the respective device.

[0047] Figure 2 This is a block diagram illustrating an exemplary video encoder 20 according to some embodiments described in this application. The video encoder 20 can perform intra-frame predictive coding and inter-frame predictive coding of video blocks within a video frame. Intra-frame predictive coding relies on spatial prediction to reduce or remove spatial redundancy in the video data within a given video frame or picture. Inter-frame predictive coding relies on temporal prediction to reduce or remove temporal redundancy in the video data within adjacent video frames or pictures of a video sequence.

[0048] like Figure 2As shown, the video encoder 20 includes a video data memory 40, a prediction processing unit 41, a decoded picture buffer (DPB) 64, an adder 50, a transform processing unit 52, a quantization unit 54, and an entropy coding unit 56. The prediction processing unit 41 further includes a motion estimation unit 42, a motion compensation unit 44, a segmentation unit 45, an intra-frame prediction processing unit 46, and an intra-frame block copying (BC) unit 48. In some embodiments, the video encoder 20 also includes an inverse quantization unit 58, an inverse transform processing unit 60, and an adder 62 for video block reconstruction. A loop filter (such as a deblocking filter (not shown)) may be located between the adder 62 and the DPB 64 to filter block boundaries to remove block artifacts from the reconstructed video. In addition to the deblocking filter, a loop filter (not shown) may also be used to filter the output of the adder 62. Before placing the reconstructed CU into a reference image memory and using it as a reference for encoding and decoding future video blocks, loop filtering, such as Sample Adaptive Offset (SAO) and Adaptive Loop Filter (ALF), can be further applied to the reconstructed CU. The video encoder 20 can take the form of a fixed or programmable hardware unit, or can be distributed among one or more of the fixed or programmable hardware units described.

[0049] Video data memory 40 stores video data to be encoded by components of video encoder 20. The video data in video data memory 40 may be obtained, for example, from video source 18. DPB 64 is a buffer that stores reference video data for use by video encoder 20 (e.g., in intra-frame or inter-frame predictive coding modes) when encoding the video data. Video data memory 40 and DPB 64 may be formed from any of a variety of memory devices. In various examples, video data memory 40 may be on-chip along with other components of video encoder 20, or off-chip relative to those components.

[0050] like Figure 2As shown, after receiving video data, the segmentation unit 45 within the prediction processing unit 41 segments the video data into video blocks. This segmentation may also include segmenting the video frame into stripes, tiles, or other larger coding units (CUs) according to a predefined splitting structure (such as a quadtree structure) associated with the video data. The video frame may be divided into multiple video blocks (or a set of video blocks referred to as tiles). The prediction processing unit 41 may select one of several feasible predictive coding modes for the current video block based on error results (e.g., coding rate and distortion level), such as one or more inter-frame predictive coding modes among multiple intra-frame predictive coding modes. The prediction processing unit 41 may provide the resulting intra-frame predictive coding block or inter-frame predictive coding block to adder 50 to generate a residual block, and to adder 62 to reconstruct the coding block for subsequent use as part of a reference frame. The prediction processing unit 41 also provides semantic elements (such as motion vectors, intra-frame mode indicators, segmentation information, and other such semantic information) to entropy coding unit 56.

[0051] To select a suitable intra-predictive coding mode for the current video block, the intra-predictive processing unit 46 within the prediction processing unit 41 can perform intra-predictive coding of the current video block in relation to one or more neighboring blocks in the same frame as the current block to be encoded to provide spatial prediction. The motion estimation unit 42 and motion compensation unit 44 within the prediction processing unit 41 perform inter-predictive coding of the current video block in relation to one or more prediction blocks in one or more reference frames to provide temporal prediction. The video encoder 20 can perform multiple coding passes, for example, to select a suitable coding mode for each block of video data.

[0052] In some implementations, motion estimation unit 42 determines an inter-frame prediction mode for the current video frame by generating motion vectors based on a predetermined pattern within the video frame sequence. The motion vectors indicate the displacement of a prediction unit (PU) of a video block within the current video frame relative to a prediction block within a reference video frame. Motion estimation performed by motion estimation unit 42 is the process of generating motion vectors that estimate the motion for a video block. For example, the motion vectors may indicate the displacement of a PU of a video block within the current video frame or picture relative to a prediction block (or other coding unit) in a reference frame associated with the current block (or other coding unit) being encoded within the current frame. The predetermined pattern may designate video frames in the sequence as P-frames or B-frames. Intra-frame BC unit 48 may determine vectors (e.g., block vectors) for intra-frame BC coding in a similar manner to how motion estimation unit 42 determines motion vectors for inter-frame prediction, or the block vectors may be determined using motion estimation unit 42.

[0053] Regarding pixel differences, the predicted block is a block of a reference frame considered to closely match the PU of the video block to be encoded. Pixel differences can be determined by the sum of absolute differences (SAD), sum of squared differences (SSD), or other difference metrics. In some implementations, the video encoder 20 can compute values ​​for sub-integer pixel positions of the reference frame stored in the DPB 64. For example, the video encoder 20 can interpolate values ​​for quarter-pixel positions, eighth-pixel positions, or other fractional pixel positions of the reference frame. Therefore, the motion estimation unit 42 can perform motion search relative to full-pixel positions and fractional pixel positions and output a motion vector with fractional-pixel accuracy.

[0054] The motion estimation unit 42 calculates the motion vector of the PU for a video block in an inter-frame predictive coding frame by comparing the position of the PU with the position of the predicted block in a reference frame selected from a first reference frame list (list 0) or a second reference frame list (list 1), where each identifier in the first and second reference frame lists is stored in one or more reference frames in the DPB 64. The motion estimation unit 42 sends the calculated motion vector to the motion compensation unit 44, and then to the entropy coding unit 56.

[0055] Motion compensation performed by motion compensation unit 44 may involve extracting or generating prediction blocks based on motion vectors determined by motion estimation unit 42. Upon receiving the motion vector of the PU for the current video block, motion compensation unit 44 may locate the prediction block pointed to by the motion vector in a reference frame list within a reference frame list, retrieve the prediction block from DPB 64, and forward the prediction block to adder 50. Adder 50 then forms a residual video block of pixel differences by subtracting the pixel values ​​of the prediction block provided by motion compensation unit 44 from the pixel values ​​of the currently encoded video block. The pixel differences forming the residual video block may include a luminance difference component or a chrominance difference component, or both. Motion compensation unit 44 may also generate semantic elements associated with video blocks of a video frame for use by video decoder 30 when decoding video blocks of a video frame. Semantic elements may include, for example, semantic elements defining motion vectors for identifying prediction blocks, any markers indicating prediction modes, or any other semantic information described herein. It should be noted that motion estimation unit 42 and motion compensation unit 44 may be highly integrated, but are described separately for conceptual purposes.

[0056] In some implementations, the intra-BC unit 48 may generate vectors and extract prediction blocks in a manner similar to that described above in conjunction with the motion estimation unit 42 and the motion compensation unit 44; however, these prediction blocks are in the same frame as the current block being encoded, and these vectors are referred to as block vectors rather than motion vectors. Specifically, the intra-BC unit 48 may determine the intra-prediction mode to be used for encoding the current block. In some examples, the intra-BC unit 48 may, for example, use various intra-prediction modes to encode the current block during individual encoding passes and test their performance through rate-distortion analysis. Next, the intra-BC unit 48 may select a suitable intra-prediction mode from the various tested intra-prediction modes for use and generate an intra-mode indicator accordingly. For example, the intra-BC unit 48 may use rate-distortion analysis to calculate rate-distortion values ​​for the various tested intra-prediction modes and select the intra-prediction mode with the best rate-distortion characteristics from the tested modes as the suitable intra-prediction mode to use. Rate-distortion analysis generally determines the amount of distortion (or error) between the coded block and the original uncoded block that was encoded to generate the coded block, as well as the bit rate (i.e., the number of bits) used to generate the coded block. Intra-frame BC unit 48 can calculate the ratio from the distortion and rate for various coded blocks to determine which intra-frame prediction mode exhibits the optimal rate-distortion value for the block.

[0057] In other examples, the intra-frame BC unit 48 may use, in whole or in part, the motion estimation unit 42 and the motion compensation unit 44 to perform such functions for intra-frame BC prediction according to the embodiments described herein. In any case, for intra-frame block copying, in terms of pixel differences, the predicted block may be a block considered to closely match the block to be encoded, the pixel differences may be determined by the sum of absolute differences (SAD), the sum of squared differences (SSD), or other difference metrics, and the identification of the predicted block may include calculating values ​​for sub-integer pixel positions.

[0058] Regardless of whether the predicted block comes from the same frame predicted intra-frame or from different frames predicted inter-frame, the video encoder 20 can form a residual video block by subtracting the pixel values ​​of the predicted block from the pixel values ​​of the current video block being encoded. The pixel difference forming the residual video block can include both luma component difference and chroma component difference.

[0059] As an alternative to the inter-frame prediction performed by the motion estimation unit 42 and the motion compensation unit 44 as described above, or the intra-block copy prediction performed by the intra-BC unit 48, the intra-frame prediction processing unit 46 can perform intra-frame prediction on the current video block. Specifically, the intra-frame prediction processing unit 46 can determine an intra-frame prediction mode for encoding the current block. To this end, the intra-frame prediction processing unit 46 can, for example, use various intra-frame prediction modes to encode the current block during individual encoding passes, and the intra-frame prediction processing unit 46 (or, in some examples, a mode selection unit) can select a suitable intra-frame prediction mode from the tested intra-frame prediction modes for use. The intra-frame prediction processing unit 46 can provide information indicating the intra-frame prediction mode selected for the block to the entropy coding unit 56. The entropy coding unit 56 can encode the information indicating the selected intra-frame prediction mode into the bitstream.

[0060] After prediction processing unit 41 determines a prediction block for the current video block via inter-frame prediction or intra-frame prediction, adder 50 forms a residual video block by subtracting the prediction block from the current video block. The residual video data in the residual block may be included in one or more transform units (TUs) and provided to transform processing unit 52. Transform processing unit 52 uses a transform (such as discrete cosine transform (DCT) or a conceptually similar transform) to transform the residual video data into residual transform coefficients.

[0061] The transform processing unit 52 can send the resulting transform coefficients to the quantization unit 54. The quantization unit 54 quantizes the transform coefficients to further reduce the bit rate. The quantization process can also reduce the bit depth associated with some or all of the coefficients. The degree of quantization can be modified by adjusting the quantization parameters. In some examples, the quantization unit 54 can then perform a scan of the matrix including the quantized transform coefficients. Optionally, the entropy coding unit 56 can perform the scan.

[0062] After quantization, entropy coding unit 56 entropy-encodes the quantized transform coefficients into a video bitstream using, for example, context-adaptive variable-length coding (CAVLC), context-adaptive binary arithmetic coding (CABAC), semantically based context-adaptive binary arithmetic coding (SBAC), probabilistic interval segmented entropy (PIPE) coding, or another entropy coding method or technique. The encoded bitstream can then be sent to video decoder 30 or archived in storage device 32 for later transmission to or retrieval by video decoder 30. Entropy coding unit 56 can also entropy-encode motion vectors and other semantic elements used for the current video frame being encoded.

[0063] The inverse quantization unit 58 and the inverse transform processing unit 60 apply inverse quantization and inverse transform, respectively, to reconstruct residual video blocks in the pixel domain for generating reference blocks to predict other video blocks. As noted above, the motion compensation unit 44 can generate motion-compensated prediction blocks from one or more reference blocks of the frame stored in the DPB 64. The motion compensation unit 44 can also apply one or more interpolation filters to the prediction blocks to compute sub-integer pixel values ​​for use in motion estimation.

[0064] Adder 62 adds the reconstructed residual block to the motion-compensated prediction block generated by motion compensation unit 44 to generate a reference block to be stored in DPB 64. The reference block can then be used as a prediction block by intra-frame BC unit 48, motion estimation unit 42 and motion compensation unit 44 to perform inter-frame prediction of another video block in subsequent video frames.

[0065] Figure 3 This is a block diagram illustrating an exemplary video decoder 30 according to some embodiments of this application. The video decoder 30 includes a video data memory 79, an entropy decoding unit 80, a prediction processing unit 81, an inverse quantization unit 86, an inverse transform processing unit 88, an adder 90, and a DPB 92. The prediction processing unit 81 further includes a motion compensation unit 82, an intra-frame prediction unit 84, and an intra-frame BC unit 85. The video decoder 30 can perform the functions described above. Figure 2 The decoding process described for the video encoder 20 is essentially the inverse of the encoding process. For example, the motion compensation unit 82 can generate prediction data based on motion vectors received from the entropy decoding unit 80, while the intra-frame prediction unit 84 can generate prediction data based on intra-frame prediction mode indicators received from the entropy decoding unit 80.

[0066] In some examples, units of the video decoder 30 may be assigned tasks to perform embodiments of this application. Furthermore, in some examples, embodiments of this disclosure may be distributed across one or more units of the video decoder 30. For example, the intra-frame BC unit 85 may perform embodiments of this application individually or in combination with other units of the video decoder 30 (such as the motion compensation unit 82, the intra-frame prediction unit 84, and the entropy decoding unit 80). In some examples, the video decoder 30 may not include the intra-frame BC unit 85, and the functionality of the intra-frame BC unit 85 may be performed by other components of the prediction processing unit 81 (such as the motion compensation unit 82).

[0067] Video data memory 79 may store video data, such as encoded video bitstreams, to be decoded by other components of video decoder 30. The video data stored in video data memory 79 may be obtained, for example, from storage device 32, from a local video source (such as a camera), via wired or wireless network communication of video data, or by accessing a physical data storage medium (e.g., a flash drive or hard disk). Video data memory 79 may include an encoded picture buffer (CPB) that stores encoded video data from the encoded video bitstream. Decoded picture buffer (DPB) 92 of video decoder 30 stores reference video data for use by video decoder 30 (e.g., in intra-frame or inter-frame predictive coding modes) when decoding video data. Video data memory 79 and DPB 92 may be formed of any memory device from a variety of memory devices, such as dynamic random access memory (DRAM) (including synchronous DRAM (SDRAM)), magnetoresistive RAM (MRAM), resistive RAM (RRAM), or other types of memory devices. For illustrative purposes, video data memory 79 and DPB 92 are shown in... Figure 3 The two components are depicted as video decoder 30. However, it will be apparent to those skilled in the art that video data memory 79 and DPB 92 may be provided by the same memory device or separate memory devices. In some examples, video data memory 79 may be on-chip along with other components of video decoder 30, or off-chip relative to those components.

[0068] During the decoding process, the video decoder 30 receives an encoded video bitstream representing video blocks of encoded video frames and associated semantic elements. The video decoder 30 may receive semantic elements at the video frame level and / or the video block level. The entropy decoding unit 80 of the video decoder 30 performs entropy decoding on the bitstream to generate quantization coefficients, motion vectors or intra-frame prediction mode indicators, and other semantic elements. The entropy decoding unit 80 then forwards the motion vectors and other semantic elements to the prediction processing unit 81.

[0069] When a video frame is encoded as an intra-predictive coded (I) frame or used as an intra-coded prediction block in other types of frames, the intra-predictive unit 84 of the prediction processing unit 81 can generate prediction data for the video block of the current video frame based on the intra-predictive mode transmitted by the signal and reference data from the previous decoded block of the current frame.

[0070] When a video frame is encoded as an inter-frame predictive coded (i.e., B or P) frame, the motion compensation unit 82 of the prediction processing unit 81 generates one or more prediction blocks for the current video frame based on motion vectors and other semantic elements received from the entropy decoding unit 80. Each prediction block can be generated from a reference frame within a reference frame list. The video decoder 30 can construct the reference frame list, list 0 and list 1, based on the reference frames stored in the DPB 92 using a default construction technique.

[0071] In some examples, when a video block is encoded according to the intra-frame BC mode described herein, the intra-frame BC unit 85 of the prediction processing unit 81 generates a prediction block for the current video block based on the block vector and other semantic elements received from the entropy decoding unit 80. The prediction block can be located within the reconstructed region of the same image as the current video block, as defined by the video encoder 20.

[0072] Motion compensation unit 82 and / or intra-frame prediction (BC) unit 85 determine prediction information for video blocks in the current video frame by parsing motion vectors and other semantic elements, and then use this prediction information to generate prediction blocks for the current video block being decoded. For example, motion compensation unit 82 uses some of the received semantic elements to determine the prediction mode (e.g., intra-frame prediction or inter-frame prediction) for encoding video blocks in the video frame, the inter-frame prediction frame type (e.g., B or P), construction information for one or more reference frames in the reference frame list for the frame, motion vectors for each inter-frame prediction encoded video block in the frame, the inter-frame prediction state for each inter-frame prediction encoded video block in the frame, and other information for decoding video blocks in the current video frame.

[0073] Similarly, the intra-BC unit 85 may use some of the received semantic elements, such as tags, to determine which video blocks in the frame were predicted using the intra-BC mode, which video blocks in the frame are within the reconstruction region and should be stored in the DPB 92, the block vector for each intra-BC predicted video block in the frame, the intra-BC prediction state for each intra-BC predicted video block in the frame, and other information for decoding video blocks in the current video frame.

[0074] The motion compensation unit 82 may also perform interpolation using interpolation filters, such as those used by the video encoder 20 during the encoding of video blocks, to compute interpolated values ​​for sub-integer pixels of the reference block. In this case, the motion compensation unit 82 may determine the interpolation filters used by the video encoder 20 from the received semantic elements and use these interpolation filters to generate the prediction block.

[0075] The dequantization unit 86 dequantizes the quantized transform coefficients, which are provided in the bitstream and entropy decoded by the entropy decoding unit 80, using the same quantization parameters calculated by the video encoder 20 for each video block in the video frame to determine the degree of quantization. The inverse transform processing unit 88 applies an inverse transform (e.g., inverse DCT, inverse integer transform, or a conceptually similar inverse transform process) to the transform coefficients to reconstruct the residual block in the pixel domain.

[0076] After the motion compensation unit 82 or the intra-frame BC unit 85 generates a prediction block for the current video block based on vectors and other semantic elements, the adder 90 reconstructs the decoded video block for the current video block by adding the residual block from the inverse transform processing unit 88 to the corresponding prediction block generated by the motion compensation unit 82 and the intra-frame BC unit 85. A loop filter (not shown) may be located between the adder 90 and the DPB 92 for further processing of the decoded video block. Loop filtering, such as a deblocking filter, sample adaptive offset (SAO), and adaptive loop filter (ALF), may be applied to the reconstructed CU before it is placed in the reference picture memory. The decoded video block in a given frame is then stored in the DPB 92, which stores reference frames for subsequent motion compensation of the next video block. The DPB 92 or a separate memory device may also store the decoded video for later presentation on a display device (e.g., Figure 1 On the display device 34).

[0077] In a typical video coding process, a video sequence usually consists of an ordered set of frames or images. Each frame may include three sample arrays, denoted as SL, SCb, and SCr. SL is a two-dimensional array of luma samples. SCb is a two-dimensional array of chrominance samples (Cb). SCr is a two-dimensional array of chrominance samples (Cr). In other cases, a frame may be monochrome and therefore consist of only a two-dimensional array of luma samples.

[0078] Like HEVC, the AVS3 standard is built on a block-based hybrid video codec framework. The input video signal is processed block by block (called a coding unit (CU)). Unlike HEVC, which only partitions blocks based on quadtrees, in AVS3, a coding tree unit (CTU) is split into CUs based on quadtrees / binary trees / extended quadtrees to accommodate varying local characteristics. Furthermore, the concept of multiple partitioned unit types in HEVC is removed; that is, there is no distinction between CUs, prediction units (PUs), and transform units (TUs) in AVS3. Instead, each CU is used consistently as the basic unit for both prediction and transform without further partitioning. In AVS3's tree partitioning structure, a CTU is first partitioned based on a quadtree structure. Then, each quadtree leaf node can be further partitioned based on binary trees and extended quadtree structures.

[0079] like Figure 4A As shown, the video encoder 20 (or more specifically, the segmentation unit 45) generates a coded representation of a frame by first segmenting the frame into a set of coding tree units (CTUs). A video frame may include an integer number of CTUs ordered sequentially from left to right and top to bottom in raster scan order. Each CTU is the largest logical coding unit, and the width and height of the CTU are signaled by the video encoder 20 using a sequence parameter set, such that all CTUs in the video sequence have the same size, one of 128×128, 64×64, 32×32, and 16×16. However, it should be noted that this application is not necessarily limited to a specific size. Figure 4B As shown, each CTU may include a coding tree block (CTB) for luma samples, two corresponding coding tree blocks for chroma samples, and semantic elements for encoding the samples of the coding tree blocks. The semantic elements describe the properties of different types of units within the coded pixel block and how the video sequence can be reconstructed at the video decoder 30, including inter-frame prediction or intra-frame prediction, intra-frame prediction mode, motion vectors, and other parameters. In monochrome images or images with three separate color planes, the CTU may include a single coding tree block and semantic elements for encoding the samples of that coding tree block. The coding tree block may be an N×N sample block.

[0080] To achieve better performance, the video encoder 20 can recursively perform tree partitioning on the coding tree blocks of the CTU, such as binary tree partitioning, ternary tree partitioning, quadtree partitioning, or a combination of both, and divide the CTU into smaller coding units (CUs). Figure 4C As depicted, the 64×64 CTU 400 is first divided into four smaller CUs, each with a block size of 32×32. Among these four smaller CUs, CU 410 and CU 420 are each divided into four CUs with a block size of 16×16. The two 16×16 CUs, 430 and CU 440, are further divided into four CUs with a block size of 8×8. Figure 4D Depicting as shown Figure 4C The final result of the CTU 400 partitioning process described in the paper is a quadtree data structure, where each leaf node of the quadtree corresponds to a CU of various sizes ranging from 32×32 to 8×8. Similar to... Figure 4B The CTU depicted in the image may include, for each CU, a coded block (CB) of luminance samples of the same size frame and two corresponding coded blocks of chrominance samples, and semantic elements for encoding the samples of the coded blocks. In monochrome images or images with three separate color planes, the CU may include a single coded block and a semantic structure for encoding the samples of the coded block. It should be noted that... Figure 4C and Figure 4DThe quadtree partitioning depicted is for illustrative purposes only, and a CTU can be split into multiple CUs based on quadtree partitioning / ternary tree partitioning / binary tree partitioning to adapt to varying local characteristics. In multi-type tree structures, a CTU is partitioned according to a quadtree structure, and each quadtree leaf CU can be further partitioned according to binary and ternary tree structures. Figure 4E As shown, there are five split / segmentation types in AVS3: quadruple split, horizontal binary split, vertical binary split, horizontal extended quadruple split, and vertical extended quadruple split.

[0081] In some implementations, the video encoder 20 may further segment the coded blocks of the CU into one or more M×N prediction blocks (PBs). A prediction block is a rectangular (square or non-square) block of samples to which the same prediction (inter-frame or intra-frame) is applied. A prediction unit (PU) of the CU may include a prediction block for luma samples, two corresponding prediction blocks for chroma samples, and semantic elements for predicting the prediction blocks. In a monochrome image or an image with three separate color planes, a PU may include a single prediction block and a semantic structure for predicting the prediction blocks. The video encoder 20 may generate a predicted luma block, a predicted Cb block, and a predicted Cr block for each PU of the CU.

[0082] Video encoder 20 can use intra-frame prediction or inter-frame prediction to generate prediction blocks for the PU. If video encoder 20 uses intra-frame prediction to generate prediction blocks for the PU, then video encoder 20 can generate prediction blocks for the PU based on decoded samples of the frame associated with the PU. If video encoder 20 uses inter-frame prediction to generate prediction blocks for the PU, then video encoder 20 can generate prediction blocks for the PU based on decoded samples of one or more frames other than the frame associated with the PU.

[0083] After the video encoder 20 generates a predicted luminance block, a predicted Cb block, and a predicted Cr block for one or more PUs of the CU, the video encoder 20 can generate a luminance residual block for the CU by subtracting the predicted luminance block of the CU from the original luminance coding block of the CU, such that each sample in the luminance residual block of the CU indicates the difference between a luminance sample in one of the predicted luminance blocks of the CU and a corresponding sample in the original luminance coding block of the CU. Similarly, the video encoder 20 can generate Cb residual blocks and Cr residual blocks for the CU, respectively, such that each sample in the Cb residual block of the CU indicates the difference between a Cb sample in one of the predicted Cb blocks of the CU and a corresponding sample in the original Cb coding block of the CU, and each sample in the Cr residual block of the CU indicates the difference between a Cr sample in one of the predicted Cr blocks of the CU and a corresponding sample in the original Cr coding block of the CU.

[0084] In addition, such as Figure 4C As shown, the video encoder 20 can use quadtree partitioning to decompose the luminance residual block, Cb residual block, and Cr residual block of the CU into one or more luminance transform blocks, Cb transform blocks, and Cr transform blocks. A transform block is a rectangular (square or non-square) sample block to which the same transform is applied. A transform unit (TU) of the CU may include a transform block for luminance samples, two corresponding transform blocks for chrominance samples, and semantic elements for transforming the samples in the transform block. Therefore, each TU of the CU may be associated with a luminance transform block, a Cb transform block, and a Cr transform block. In some examples, the luminance transform block associated with a TU may be a sub-block of the CU's luminance residual block. A Cb transform block may be a sub-block of the CU's Cb residual block. A Cr transform block may be a sub-block of the CU's Cr residual block. In a monochrome image or an image with three separate color planes, a TU may include a single transform block and a semantic structure for transforming the samples in that transform block.

[0085] The video encoder 20 can apply one or more transforms to the luminance transform block of the TU to generate a luminance coefficient block for the TU. The coefficient block can be a two-dimensional array of transform coefficients. The transform coefficients can be scalars. The video encoder 20 can apply one or more transforms to the Cb transform block of the TU to generate a Cb coefficient block for the TU. The video encoder 20 can apply one or more transforms to the Cr transform block of the TU to generate a Cr coefficient block for the TU.

[0086] After generating coefficient blocks (e.g., luminance coefficient blocks, Cb coefficient blocks, or Cr coefficient blocks), video encoder 20 may quantize the coefficient blocks. Quantization typically refers to the process of quantizing transform coefficients to potentially reduce the amount of data used to represent the transform coefficients, thereby providing further compression. After quantizing the coefficient blocks, video encoder 20 may entropy encode the semantic elements indicating the quantized transform coefficients. For example, video encoder 20 may perform context-adaptive binary arithmetic coding (CABAC) on the semantic elements indicating the quantized transform coefficients. Finally, video encoder 20 may output a bitstream comprising a bit sequence that forms a representation of coded frames and associated data; the bitstream may be stored in storage device 32 or transmitted to target device 14.

[0087] After receiving the bitstream generated by the video encoder 20, the video decoder 30 can parse the bitstream to obtain semantic elements. The video decoder 30 can reconstruct frames of video data based at least in part on the semantic elements obtained from the bitstream. The process of reconstructing the video data is generally the inverse of the encoding process performed by the video encoder 20. For example, the video decoder 30 can perform an inverse transform on the coefficient block associated with the TU of the current CU to reconstruct the residual block associated with the TU of the current CU. The video decoder 30 also reconstructs the coded blocks of the current CU by adding samples of the prediction block of the PU for the current CU to the corresponding samples of the transform block of the TU of the current CU. After reconstructing the coded blocks for each CU of the frame, the video decoder 30 can reconstruct the frame.

[0088] In some embodiments, the focus of this disclosure is to reduce the complexity of multi-span component prediction (PMC) tools applied in the AVS3 standard. This document briefly describes the cross-component prediction technique in the AVS3 standard, namely two-step cross-component prediction (TSCPM). The PMC design in the AVS3 standard is used as an example to explain the main aspects of the PMC tool.

[0089] Although the existing PMC design in the AVS3 standard is used as the basic PMC approach in the following description, the methods and systems described herein can also be applied to other cross-component prediction designs or other codec tools with the same or similar design principles by those skilled in the art of video coding and decoding.

[0090] TSCPM is also known as the Cross-Component Prediction Model (CCLM) in the VVC standard. It follows the same design principles but has some subtle differences in certain design details.

[0091] The main TSCPM process includes the following steps. First, a linear model is obtained from neighboring reconstructed samples. Second, the linear model is applied to the original reconstructed luma block to obtain the internal prediction block. Third, the internal prediction block is downsampled to generate the final chromaticity prediction block.

[0092] Figure 5 This is a block diagram depicting the basic process of a chromaticity prediction block generation process according to some embodiments of this disclosure. The left square 502 is connected via R... L(x，y) This represents the original reconstructed luminance sample located at (x, y) within the same luminance block. A temporary chromaticity prediction block 504 is generated by simply applying a linear model with parameters (α, β) to each luminance sample, and the samples are generated by P′. c (x, y) = α × R L (x, y) + β represents this. Subsequently, the temporary chromaticity prediction block 504 is further downsampled to generate the final chromaticity prediction block 506, and the samples are generated by P. c(x, y) represents this.

[0093] The linear model derivation process and the downsampling process are described in the following subsections.

[0094] Figure 6 This is a block diagram illustrating an example of deriving four neighboring sample points according to some embodiments of the present disclosure. When deriving a linear model, four sample points 602, 604, 606, and 608 can be selected, and parameters are calculated using the average of the two larger values ​​and the average of the two smaller values. First, the ratio r of width to height is calculated as shown in equation (1) below. Then, two sample points are selected based on the availability of the top row and the left column.

[0095] First, calculate the width-to-height ratio r as shown in Equation 1. Then, select two sample points based on the availability of the top row and the left column.

[0096]

[0097] The derivation of posA and posL is shown in equation (2) below (position index starts from 0).

[0098] posA = width - r

[0099] posL = height - 1 (2)

[0100] P′ c (x, y) = α × R L (x, y) + β (3)

[0101] Similar to the general intra-frame prediction process, the cropping operation is applied to P′. c (x, y) ensures that it is within [0, 1 << (bit depth - 1)].

[0102] A six-tap filter (i.e., [121; 121]) is introduced for the downsampling process of the temporary chromaticity prediction block, as shown in Equation 4.

[0103] P c =(2×P′) c (2x, 2y) + 2×P′ c (2x, 2y+1)+P′ c (2x-1, 2y)+Pc′2x+1, 2y+Pc′2x-1, 2y+1+Pc′2x+1, 2y-1+4>>3 (4)

[0104] In addition, for the chroma samples located in the leftmost column, the [1,1] downsampling filter is applied instead.

[0105] The semantic design is described below. According to the current TSCPM design, a flag is used to signal whether the chroma intra-frame prediction mode is TSCPM. This flag is encoded and decoded immediately after the DM mode. The detailed binary bit strings for each chroma mode are listed in Table 1 below, which shows the encoded and decoded binary bit signaling in the TSCPM case of the chroma intra-frame mode.

[0106] Table 1: Encoding and decoding binary bit signaling in TSCPM under chroma intra-frame mode.

[0107] index model binary string 0 DM 1 5 TSCPM 01 1 DC 001 2 HOR 0001 3 VER 00001 4 BI 00000

[0108] The AVS3 standard employs the Multi-Span Component Prediction (PMC) method, where the prediction of the Cr component is derived through a linear combination of Y-reconstructed samples and Cb-reconstructed samples. First, the internal block IPred is derived based on a linear model applied to the corresponding luma block, and the final prediction of the Cr block is obtained based on the difference between the downsampled temporary block and the reconstructed Cb block. More specifically, the final prediction of the Cr block is defined as follows.

[0109] IPred = A·Rec Y +B, (5-1)

[0110] FPred Cr =IPred′-Rec Cb (5-2)

[0111] Figure 7 This is a block diagram illustrating an exemplary PMC process according to some embodiments of the present disclosure. In equations (5-1) and (5-2) above, Rec Y IPred represents the reconstruction of the Y component, and IPred is an inner block with the same dimensions as the luminance-coded block. IPred′ represents the downsampled IPred, which uses the same set of downsampling filters as in TSCPM.

[0112] To keep the complexity as low as possible and preserve the logic of TSCPM, the linear parameters (A, B) are set to (α0+α1, β0+β1), where (α0, β0) and (α1, β1) are two sets of linear model parameters derived for Cb and Cr, respectively.

[0113] Since there are three modes in TSCPM based on how the linear model is derived, three additional PMC modes are introduced. These three PMC modes are considered as additional TSCPM modes. Furthermore, if the corresponding Cr block is encoded or decoded using a PMC mode, the inferred coded_block_flag(cbf) of the Cb block is 1. When the current Cr block is encoded or decoded using a PMC mode, the QP of the Cr block is incremented by 1. Table 2 below shows the binary bit strings of the chroma intra-prediction modes in AVS3 and the designed PMC modes. PMC modes have an IntraChromaPredMode value of 8, 9, or 10.

[0114] Table 2: Illustration of the binary string of the chroma intra-prediction mode and the designed PMC mode in AVS3.

[0115]

[0116]

[0117] In some embodiments, while the PMC mode effectively enhances the efficiency of intra-frame and inter-frame prediction, several aspects of its existing design can still be further improved in terms of encoder and decoder hardware complexity. Specifically, as in equation (5-1) and Figure 7 As described, the PMC requires an intermediate buffer (IPred) to store intermediate predictions derived from RecY, which are then fed into the IPred block for further downsampling. However, existing PMC designs do not include a pruning operation at the output of the IPred block, resulting in an unnecessary additional bit range allocation for the buffer memory. For example, in a 10-bit application, the worst-case scenario is that α0 or α1 is (1023 × 65536 + 8) >> 4 = 4190208 (22 bits) and β0 or β1 is 0. This results in a maximum IPred value of (8380416 × 1023 >> 16) + 0 = 130816 (17 bits). However, since the IPred value physically represents the sum of two 10-bit prediction samples, it essentially only requires 11 bits. Based on this analysis, this increase in the dynamic range of intermediate prediction samples not only increases the on-chip memory size but also increases the bit width of the addition used. Both of these factors are very expensive for practical hardware codec implementations.

[0118] In some embodiments, methods and systems for simplifying PMC are implemented to reduce the complexity of the PMC model. Specifically, the main aspects of the implemented methods are summarized below.

[0119] In some embodiments, a clipping operation is added to the output of IPred to limit the dynamic bit range to specific values. For example, in a first approach, the output of IPred is clipped to the range [0, (1 << (bit depth + 1)) - 1], which gives an optimal trade-off between buffer size and performance. In another example, in a second approach, the output of IPred is clipped to the range [0, (1 << bit depth) - 1] if the focus is on reusing the original TSCPM clipping logic. In yet another example, in a third approach, the output of IPred is clipped to the range [0, (1 << 15) - 1], which guarantees that the intermediate dynamic range of the chroma prediction samples does not exceed a 16-bit integer, which is the common bit width for storing intermediate parameters used in the actual hardware codec implementation. The clipping operation is defined below as the function Clip3, where x is the lower boundary of the clipping range, y is the upper boundary of the clipping range, and Z is the input value of the clipping operation.

[0120]

[0121] As an example, the following shows the corresponding AVS specification changes after applying the first method.

[0122] To derive the chromaticity prediction values:

[0123] If IntraChromaPredMode equals 5, 6, or 7, or if IntraChromaPredMode equals 8, 9, or 10, and the current component is Cb, then the chromaticity prediction value is derived as follows:

[0124] ·predChroma[x][y]=Clip1(((α×I[x][y])>>iShift)+β)(x=0~2M-1,y=0~2N-1)

[0125] ·predMatrix[0][y]=(predChroma[0][2y]+predChroma[0][2y+1]+1)>>1, (y=0~N-1)

[0126] ·predMatrix[x][y]=(predChroma[2x-1][2y]+2×predChroma[2x][2y]+predChroma[2x+1][2y]+predC hroma[2x-1][2y+1]+2×predChroma[2x][2y+1]+predChroma[2x+1][2y+1]+4)>>3, (x=1~M-1, y=0~N-1)

[0127] otherwise,

[0128] predChroma[x][y]=Clip3(0, (1<<(BitDepth+1))-1, (((α_Cb+α_Cr)(1[x][y])>>1Shift)+β_Cb+β_Cr)(x=0~2M-1, y=0~2N-1)

[0129] predMatrixTemp[0][y]=(predChroma[0][2y]+predChroma[0][2y+1]+1)>>1, (y=0~N-1)

[0130] predMatrixTemp[x][y]=(predChroma[2x-1][2y]+2×predChroma[2x][2y]+predChroma[2x+1][2y]+pre dChroma[2x-1][2y+1]+2×predChroma[2x][2y+1]+predChroma[2x+1][2y+1]+4)>>3, (x=1~M-1, y=0~N-1)

[0131] predMatrix[x][y]=predMatrixTemp[x][y]–Cb[x][y], (x=0~M-1, y=0~N-1)

[0132] In some embodiments, methods and systems for implementing PMC enhancements improve the encoding / decoding efficiency of PMC by exploring the sign and scaling relationships between the luminance prediction Cb residual and the luminance prediction Cr residual. In the following description, a PMC design from the AVS3 standard is used as an example to illustrate the potential encoding / decoding efficiency when exploring more Cb and Cr residual relationships. Specific implementation methods are then provided.

[0133] Although the PMC design in the AVS3 standard is used as the basic PMC approach in the following description, the implementation methods described herein can also be applied to other cross-component prediction designs or other codec tools with the same or similar design principles by those skilled in the art of video coding and decoding.

[0134] TSCPM is also known as the Cross-Component Prediction Model (CCLM) in the VVC standard. It follows the same design principles but has some subtle differences in certain design details.

[0135] The predicted residuals of Cb and Cr are either positively or negatively correlated. However, by decomposing the formula for PMC Cr sample prediction as shown below, only the negative correlation is included. (a0, b0) and (a1, b1) represent the TSCPM parameters of Cb and Cr, respectively. Rec, Pred, and Res represent the reconstructed sample, predicted sample, and residual sample, respectively. Existing PMC removes redundancy between these two residuals only when the Cb residual predicted by TSCPM is negatively correlated with the Cr residual predicted by TSCPM.

[0136] Pred_Cr=(a0+a1)*Rec_Y+(b0+b1)–Rec_Cb (6-1)

[0137] =(a0+a1)*Rec_Y+(b0+b1)-(Pred_Cb+Res_Cb) (6-2)

[0138] =(a0+a1)*Rec_Y+(b0+b1)-(a0*Rec_Y+b0+Res_Cb) (6-3)

[0139] =a1*Rec_Y+b1–Res_Cb (6-4)

[0140] In some embodiments, to further improve the efficiency of PMC encoding and decoding, the positive correlation between the prediction residuals of Cb and Cr can be considered in the PMC design, as shown below.

[0141] Pred_Cr=-(a0-a1)*Rec_Y-(b0-b1)+Rec_Cb (7-1)

[0142] =(-a0+a1)*Rec_Y+(-b0+b1)+(Pred_Cb+Res_Cb) (7-2)

[0143] =(-a0+a1)*Rec_Y+(-b0+b1)+(a0*Rec_Y+b0+Res_Cb) (7-3)

[0144] = a1*Rec_Y+b1+Res_Cb (7-4)

[0145] In some embodiments, a flag can be signaled at the sequence, frame, strip, CTU, or block level to indicate which form of equation (6-1) or equation (7-1) applies to PMC Cr sample prediction. The flag can be context-encoded into a PMC mode flag. Table 3 provides examples of adding three additional PMC modes 11, 12, and 13 and modifying the bit strings.

[0146] Table 3: Illustration of the bit string of the chroma intra-prediction mode in AVS3 and the bit string of the modified PMC mode with alternative forms.

[0147]

[0148]

[0149]

[0150] In some embodiments, when combined with the above-described PMC simplification, the lower boundary of the pruning becomes -1 << bit depth. For example, in the first method, the output of IPred is pruned to the range [-(1 << bit depth), (1 << (bit depth + 1)) - 1]. The corresponding AVS specification is modified as follows:

[0151] predChroma[x][y]=Clip3(-(1<<bit depth),(1<<(bit depth+1))-1,(((α_cb+α_cr)×I[x][y])>>iShift)+β_cb+β_cr)(x=0~2M-1,y=0~2N-1)

[0152] In some embodiments, the correlation between the predicted residuals of Cb and Cr can be extended to both the sign and the scaling factor, denoted as w, as shown below.

[0153] Pred_Cr=(w*a0+a1)*Rec_Y+(w*b0+b1)-w*Rec_Cb (8-1)

[0154] =(w*a0+a1)*Rec_Y+(w*b0+b1)-w*(Pred_Cb+Res_Cb) (8-2)

[0155] =(w*a0+a1)*Rec_Y+(w*b0+b1)-w*(a0*Rec_Y+b0+Res_Cb) (8-3)

[0156] =a1*Rec_Y+B1-w*Res_Cb (8-4)

[0157] In some embodiments, to signal w, a signaling format flag is first used, followed by an index indicating which absolute value |w| is applied in a finite table. For example, 1 / 4, 1 / 2, 1, 2, 4… etc., where |w| only includes powers of 2. Note that |w| = 0 is excluded because in this case, PMC degenerates into TSCPM. Signaling format flags and indexes can be used at the sequence, frame, stripe, CTU, or block level.

[0158] In some embodiments, when combined with PMC simplification, for example, if |w| only includes 1 / 4, 1 / 2, 1, 2, and 4, then the lower bound of the pruning becomes -1 << (bit depth + log2(max|w|)). In the first method, the output of IPred is pruned to the range [-1 (<< (bit depth + 2)), (1 << (bit depth + 1)) - 1]. And the corresponding AVS specification is modified as follows:

[0159] predChroma[x][y]=Clip3(-(1<<(BitDepth+2)), (1<<(BitDepth+1))-1, (((α_cb+α_cr)×I[x][y])>>iShift)+β_cb+β_cr)(x=0~2M-1, y=0~2N-1)

[0160] In some embodiments, examples of w transmitted by signal at the block level are shown in Table 3 below.

[0161] Table 3: Examples of w transmitted via semaphore at the block level

[0162]

[0163]

[0164] Figure 8 This is a flowchart illustrating an exemplary process 800 of PMC simplification by using a cropping operation at the output of IPred, according to some embodiments of this disclosure. The PMC enhancement process by utilizing both the positive and negative relationships between the prediction residuals of video components Cb and Cr is further described below.

[0165] The video decoder 30 receives from the bitstream of video data a semantic element (810) indicating that the intra-frame chroma prediction mode is one of the multi-span component prediction (PMC) modes for the coding unit.

[0166] The video decoder 30 derives intermediate predictions of the chrominance components of the coding unit based on a linear model applied to the reconstruction of the luminance components of the coding unit (820).

[0167] The video decoder 30 applies a cropping operation to the intermediate prediction output of the chroma component to limit the value of the intermediate prediction output to a predetermined range (830).

[0168] In some embodiments, the intermediate prediction block (820) of the chroma component of the coding unit is derived according to the following formula:

[0169] IPred = A·Rec Y +B

[0170] Where IPred represents the intermediate prediction of the chromaticity component with the same dimension as the luminance component, Rec_Y represents the reconstruction of the luminance component, and A and B are the linear parameters of the linear model.

[0171] In some embodiments, the predetermined range is 0 to 2. BitDepth+1 -1, where BitDepth represents the bit depth of the chroma component.

[0172] In some embodiments, the predetermined range is 0 to 2. BitDepth -1, where BitDepth represents the bit depth of the chroma component.

[0173] In some implementations, the predetermined range is 0 to 2. 15 -1.

[0174] In some embodiments, the intermediate prediction block (820) of the chroma component of the coding unit is derived according to the following formula:

[0175] IPred = S × (A·Rec) Y +B)

[0176] Where IPred represents the intermediate prediction of the chromaticity component with the same dimension as the luminance component, and Rec Y This represents the reconstruction of the luminance component, where A and B are the linear parameters of the linear model, and S indicates a positive or negative sign.

[0177] In some embodiments, the intermediate prediction block (820) of the chrominance components of the derived coding unit further includes: downsampling the output of IPred after the pruning operation; and obtaining the final prediction FPred of the Cr chrominance components according to the following formula. Cr :

[0178] FPred Cr =IPred′-S×w×Rec Cb

[0179] Where IPred' represents the downsampled output of IPred after the cropping operation, and Rec Cb is the reconstructed Cb chromaticity component within the encoding unit, and w is a positive weighting parameter. In some embodiments, the predetermined range of the cropping operation is -2. BitDepth +1 Up to 2 BitDepth+1 -1, where BitDepth represents the bit depth of the chroma component.

[0180] In some embodiments, the intermediate prediction block (820) for the chroma components of the coded unit further includes: receiving an intrachroma PMC mode form flag from the bitstream, the flag indicating a positive correlation between the prediction residuals of chroma components Cb and Cr by assigning a negative sign to S or a negative correlation between the prediction residuals of chroma components Cb and Cr by assigning a positive sign to S; and receiving intrachroma index semantics indicating the value of the positive weight parameter w from the bitstream. In some embodiments, the intrachroma PMC mode form flag is intra_chroma_pmc_mode_form. In some embodiments, the intrachroma index semantics is intra_chroma_pmc_mode_index.

[0181] In some embodiments, from including 2 n The table representing the values ​​of the positive weight parameter w is selected, where n is an integer.

[0182] In some embodiments, the value of the positive weight parameter w is selected from values ​​including 1 / 4, 1 / 2, 1, 2, and 4.

[0183] In some embodiments, intra-frame chroma PMC mode format flags and intra-frame chroma index semantics are signaled in one or more of sequences, frames, stripes, coding tree units, and block levels.

[0184] Other embodiments include various subsets of the above embodiments that are combined or otherwise rearranged in various other embodiments.

[0185] In one or more examples, the described functionality may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functionality may be stored on or transmitted via a computer-readable medium as one or more instructions or code, and executed by a hardware-based processing unit. A computer-readable medium may include a computer-readable storage medium, which corresponds to a tangible medium such as a data storage medium, or a communication medium that includes any medium facilitating the transfer of a computer program from one place to another (e.g., according to a communication protocol). In this way, a computer-readable medium may generally correspond to (1) a non-transitory tangible computer-readable storage medium, or (2) a communication medium, such as a signal or carrier wave. A data storage medium may be any available medium accessible by one or more computers or one or more processors to retrieve instructions, code, and / or data structures used to implement the implementations described in this application. Computer program products may include computer-readable media.

[0186] The terminology used in the description of the embodiments herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of the claims. As used in the description of the embodiments and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the term “and / or,” as used herein, refers to and covers any and all possible combinations of one or more of the associated listed items. It will be further understood that the terms “comprising” and / or “including…”, when used in this specification, specify the presence of the stated features, elements, and / or components, but do not exclude the presence or addition of one or more other features, elements, components, and / or groups thereof.

[0187] It will also be understood that although the terms first, second, etc., may be used herein to describe various elements, these elements should not be limited by these terms. These terms are used only to distinguish one element from another. For example, without departing from the scope of the embodiments, a first electrode may be referred to as a second electrode, and similarly, a second electrode may be referred to as a first electrode. Both the first electrode and the second electrode are electrodes, but they are not the same electrode.

[0188] Throughout this specification, references to the singular or plural forms of "an example," "example," "exemplary example," etc., indicate that one or more specific features, structures, or characteristics described in connection with the example are included in at least one example of this disclosure. Therefore, phrases such as "in an example," "in the example," "in the exemplary example," etc., appearing in various places throughout this specification, do not necessarily refer to the same example. Furthermore, specific features, structures, or characteristics in one or more examples can be combined in any suitable manner.

[0189] The description in this application has been presented for purposes of illustration and description and is not intended to be exhaustive or limited to the invention in the forms disclosed. Many modifications, variations, and alternative embodiments will be apparent to those skilled in the art from the teachings presented in the foregoing description and the associated drawings. The embodiments were chosen and described in order to best explain the principles of the invention, its practical application, and to enable others skilled in the art to understand the various embodiments of the invention, and preferably utilize the basic principles and various embodiments with modifications suitable for the intended particular use. Therefore, it will be understood that the scope of the claims is not limited to the specific examples of the disclosed embodiments, and that modifications and other embodiments are intended to be included within the scope of the appended claims.

Claims

1. A method of encoding video data, comprising: determining whether an intra chroma prediction mode for a coding unit is one of a multi-cross component prediction (PMC) mode; in a case where it is determined that the intra chroma prediction mode is one of the PMC mode, deriving an intermediate prediction of a chroma component of the coding unit according to a linear model applied to a reconstruction of a luma component of the coding unit; applying a clipping operation to an output of the intermediate prediction of the chroma component to limit values of the output of the intermediate prediction within a predetermined range; down-sampling an output of IPred after the clipping operation; and signaling a semantic element, wherein the semantic element indicates whether the intra chroma prediction mode is one of the PMC mode. The final prediction of the Cr chrominance component is obtained according to the formula : where IPred denotes the intermediate prediction of the chroma component having the same dimension as the luma component, IPred' denotes a down-sampled output of IPred after the clipping operation, is the reconstructed Cb chroma component within the coding unit, and w is a positive weight parameter; deriving the intermediate prediction of the chroma component of the coding unit according to:

2. The method of claim 1, wherein, 5. The method of claim 1, further comprising: where IPred denotes the intermediate prediction of the chroma component having the same dimension as the luma component, denotes the reconstruction of the luma component, and A and B are linear parameters of the linear model.

3. The method of claim 2, wherein the predetermined range is 0 to 2 BitDepth+1 - 1, where BitDepth denotes the bit depth of the chroma component.

4. The method of claim 2, wherein, said predetermined range is 0 to 2 15 -1. determining a value of the positive weight parameter w based on the intra chroma prediction mode.

7. The method of claim 5, wherein the value of the positive weight parameter w is selected from values comprising 1 / 4, 1 / 2, 1, 2, and 4.

6. The method of claim 5, wherein the value of the positive weight parameter w is selected from the values represented by 2 n where n is an integer.

8. The method of claim 1, wherein the semantic element is signaled in one or more of a sequence level, a frame level, a slice level, a coding tree unit level, and a coding unit level.

9. An electronic device, comprising: one or more processing units; memory coupled to the one or more processing units; and a plurality of programs stored in the memory that, when executed by the one or more processing units, cause the electronic device to perform the method of any of claims 1-8.

10. A non-transitory computer-readable storage medium storing a plurality of programs for execution by an electronic device having one or more processing units, wherein the plurality of programs, when executed by the one or more processing units, cause the electronic device to perform the method of any of claims 1-8.

11. A computer program product comprising a non-transitory computer-readable storage medium storing a plurality of programs for execution by a computing device having one or more processors, wherein the plurality of programs, when executed by the one or more processors, cause the computing device to perform the method of any of claims 1-8.

12. A method of transmitting a bitstream, comprising: performing the method of any of claims 1-8 to generate a bitstream; transmitting the bitstream to a decoding device. ​ ​

Images