Residual coding and decoding for color components

Abstract

The present disclosure provides a method for video decoding, comprising: obtaining a joint residual block for generating a residual block of a first color component and a residual block of a second color component associated with a current block to be reconstructed; a weight candidate list including a plurality of weight candidates is determined, and for each of the plurality of weight candidates, a template cost of a template of the current block is calculated, the template cost indicates a difference degree between the following two values: a product of a residual value of a second color component of each sample point in the template and the weight candidate, and a residual value of a first color component of each sample point in the template; selecting a weight candidate for generating a residual block of the first color component based on the calculated template cost; and generating a residual block of the second color component based on the obtained joint residual block, and generating a residual block of the first color component based on the joint residual block and the selected weight candidate.

Description

[0001] Cross-references to related applications

[0002] This application is based on and claims priority to U.S. Provisional Application 63 / 699,768, filed September 26, 2024, and PCT Application PCT / CN2024 / 125090, filed October 15, 2024, the entire contents of which are incorporated herein by reference. Technical Field

[0003] This disclosure relates to video encoding / decoding and compression. More specifically, this application relates to improved joint encoding / decoding of residuals from different color components. Background Technology

[0004] Various electronic devices support digital video. These devices transmit and receive, or otherwise transfer, digital video data via communication networks, and / or store digital video data on storage devices. Because communication networks have limited bandwidth capacity and storage devices have limited storage resources, video data can be compressed using one or more video codec standards before transmission or storage to generate coded video data using a lower bit rate, while avoiding or minimizing video quality degradation. Summary of the Invention

[0005] Embodiments of this disclosure provide details of proposed improvements for joint encoding and decoding of residuals of different color components.

[0006] According to one aspect of this disclosure, a method for video decoding is provided, comprising: obtaining a joint residual block for generating a residual block of a first color component and a residual block of a second color component associated with a current block to be reconstructed; determining a weight candidate list including a plurality of weight candidates, and for each of the plurality of weight candidates, calculating a template cost of a template of the current block, wherein the template cost indicates the degree of difference between: the product of the residual value of the second color component of each sample in the template and the weight candidate, and the residual value of the first color component of each sample in the template; selecting a weight candidate for generating the residual block of the first color component based on the calculated template cost; and generating the residual block of the second color component based on the obtained joint residual block, and generating the residual block of the first color component based on the joint residual block and the selected weight candidate.

[0007] According to an aspect of the present disclosure, a method for video decoding is provided, including: determining a first syntax element, the first syntax element indicating whether residual blocks of a first color component and a second color component associated with a current block to be reconstructed both do not exist; and in response to determining that the first syntax element indicates that the residual blocks of the first color component and the second color component do not both exist, determining a second syntax element, the second syntax element indicating whether to use a joint residual block to generate the residual blocks of the first color component and the second color component.

[0008] According to an aspect of the present disclosure, a method for video encoding is provided, including: determining and sending a joint residual block in a bitstream, the joint residual block being used to generate residual blocks of a first color component and a second color component associated with a current block to be reconstructed; determining a weight candidate list including a plurality of weight candidates, and for each weight candidate in the plurality of weight candidates, calculating a template cost of a template of the current block, wherein the template cost indicates a degree of difference between a product of residual values of the second color component of each sample in the template and the weight candidate and residual values of the first color component of each sample in the template; based on the calculated template costs, selecting a weight candidate used to generate the residual block of the first color component; and generating the residual block of the second color component based on the obtained joint residual block, and generating the residual block of the first color component based on the joint residual block and the selected weight candidate.

[0009] According to an aspect of the present disclosure, a method for video encoding is provided, including: determining a first syntax element, the first syntax element indicating whether residual blocks of a first color component and a second color component associated with a current block to be reconstructed both do not exist; and in response to determining that the first syntax element indicates that the residual blocks of the first color component and the second color component do not both exist, determining a second syntax element, the second syntax element indicating whether to use a joint residual block to generate the residual blocks of the first color component and the second color component.

[0010] According to an aspect of the disclosure, there is provided a computer-readable storage medium storing a bitstream formed of instructions that, when executed by a computing device having one or more processors, cause the one or more processors to perform the following steps in an encoding method: determining and sending, in the bitstream, a joint residual block used to generate a residual block of a first color component and a residual block of a second color component associated with a current block to be reconstructed; determining a weight candidate list comprising a plurality of weight candidates, and for each weight candidate of the plurality of weight candidates, computing a template cost of a template of the current block, wherein the template cost indicates a degree of difference between a product of a residual value of the second color component of each sample in the template and the weight candidate and a residual value of the first color component of each sample in the template; selecting, based on the computed template costs, a weight candidate used to generate the residual block of the first color component; and generating the residual block of the second color component based on the obtained joint residual block and the residual block of the first color component based on the joint residual block and the selected weight candidate, wherein the bitstream is to be decoded by a decoding method according to the disclosure.

[0011] According to an aspect of the disclosure, there is provided a computer-readable storage medium storing a bitstream formed of instructions that, when executed by a computing device having one or more processors, cause the one or more processors to perform the following steps in an encoding method: determining a first syntax element indicating whether a residual block of a first color component and a residual block of a second color component associated with a current block to be reconstructed are both absent; and in response to determining that the first syntax element indicates that the residual block of the first color component and the residual block of the second color component are not both absent, determining a second syntax element indicating whether the residual block of the first color component and the residual block of the second color component are generated using a joint residual block, wherein the bitstream is to be decoded by a decoding method according to the disclosure.

[0012] According to an aspect of the disclosure, there is provided a computer-readable storage medium storing a bitstream formed of instructions that, when executed by a computing device having one or more processors, cause the one or more processors to perform the following steps in an encoding method: determining and sending, in the bitstream, a joint residual block used to generate a residual block of a first color component and a residual block of a second color component associated with a current block to be reconstructed; determining a weight candidate list comprising a plurality of weight candidates, and for each weight candidate of the plurality of weight candidates, computing a template cost of a template of the current block, wherein the template cost indicates a degree of difference between a product of a residual value of the second color component of each sample in the template and the weight candidate and a residual value of the first color component of each sample in the template; selecting, based on the computed template costs, a weight candidate used to generate the residual block of the first color component; and generating the residual block of the second color component based on the obtained joint residual block and the residual block of the first color component based on the joint residual block and the selected weight candidate, wherein the bitstream is to be decoded by a decoding method according to the disclosure.

[0013] According to one aspect of this disclosure, a method for storing a bitstream is provided, comprising: performing the following steps in an encoding method to generate a bitstream: determining and transmitting a joint residual block in the bitstream, the joint residual block being used to generate a residual block for a first color component and a residual block for a second color component associated with a current block to be reconstructed; determining a list of weight candidates including a plurality of weight candidates, and for each of the plurality of weight candidates, calculating a template cost of a template of the current block, wherein the template cost indicates the degree of difference between: the product of the residual value of the second color component at each sample in the template and the weight candidate, and the residual value of the first color component at each sample in the template; selecting a weight candidate for generating a residual block for the first color component based on the calculated template cost; generating a residual block for the second color component based on the obtained joint residual block, and generating a residual block for the first color component based on the joint residual block and the selected weight candidate; and storing the bitstream, wherein the bitstream will be decoded by a decoding method according to this disclosure.

[0014] According to one aspect of this disclosure, a method for storing a bitstream is provided, comprising: performing the steps of an encoding method to generate a bitstream: determining a first syntax element, the first syntax element indicating whether a residual block of a first color component and a residual block of a second color component associated with a current block to be reconstructed both exist; and in response to determining that the first syntax element indicates that the residual blocks of the first color component and the residual blocks of the second color component do not both exist, determining a second syntax element, the second syntax element indicating whether to use joint residual blocks to generate the residual blocks of the first color component and the residual blocks of the second color component; and storing the bitstream, wherein the bitstream will be decoded by a decoding method according to this disclosure.

[0015] According to one aspect of this disclosure, a method for storing a bit stream is provided, comprising: performing an encoding method according to this disclosure to generate a bit stream; and storing the bit stream.

[0016] According to one aspect of this disclosure, a method for transmitting a bitstream is provided, comprising: performing the following steps in an encoding method to generate a bitstream: determining and transmitting a joint residual block in the bitstream, the joint residual block being used to generate a residual block for a first color component and a residual block for a second color component associated with a current block to be reconstructed; determining a weight candidate list comprising a plurality of weight candidates, and for each of the plurality of weight candidates, calculating a template cost of a template of the current block, wherein the template cost indicates the degree of difference between: the product of the residual value of the second color component for each sample in the template and the weight candidate, and the residual value of the first color component for each sample in the template; selecting a weight candidate for generating a residual block for the first color component based on the calculated template cost; generating a residual block for the second color component based on the obtained joint residual block, and generating a residual block for the first color component based on the joint residual block and the selected weight candidate; and transmitting the bitstream, wherein the bitstream will be decoded by a decoding method according to this disclosure.

[0017] According to one aspect of this disclosure, a method for transmitting a bitstream is provided, comprising: performing the following steps in an encoding method to generate a bitstream: determining a first syntax element, the first syntax element indicating whether a residual block of a first color component and a residual block of a second color component associated with a current block to be reconstructed both exist; and in response to determining that the first syntax element indicates that the residual blocks of the first color component and the residual blocks of the second color component do not both exist, determining a second syntax element, the second syntax element indicating whether to use joint residual blocks to generate the residual blocks of the first color component and the residual blocks of the second color component; and transmitting the bitstream, wherein the bitstream will be decoded by a decoding method according to this disclosure.

[0018] According to one aspect of this disclosure, a method for transmitting a bit stream is provided, comprising: performing an encoding method according to this disclosure to generate a bit stream; and transmitting the bit stream.

[0019] According to one aspect of this disclosure, an electronic device is provided, comprising: a non-transitory computer-readable storage medium; and a processor configured to perform the following steps in an encoding method to generate a bitstream and store and / or transmit the bitstream: determining and transmitting a joint residual block in the bitstream, the joint residual block being used to generate a residual block of a first color component and a residual block of a second color component associated with a current block to be reconstructed; determining a list of weight candidates including a plurality of weight candidates, and for each of the plurality of weight candidates, calculating a template cost of a template of the current block, wherein the template cost indicates the degree of difference between: the product of the residual value of the second color component of each sample in the template and the weight candidate, and the residual value of the first color component of each sample in the template; selecting a weight candidate for generating a residual block of the first color component based on the calculated template cost; and generating a residual block of the second color component based on the obtained joint residual block, and generating a residual block of the first color component based on the joint residual block and the selected weight candidate, wherein the bitstream will be decoded by a decoding method according to this disclosure.

[0020] According to one aspect of this disclosure, an electronic device is provided, comprising: a non-transitory computer-readable storage medium; and a processor configured to perform the following steps in an encoding method to generate a bitstream and store and / or transmit the bitstream: determining a first syntax element indicating whether a residual block of a first color component and a residual block of a second color component associated with a current block to be reconstructed are both absent; and in response to determining that the first syntax element indicates that the residual blocks of the first color component and the residual blocks of the second color component are not both absent, determining a second syntax element indicating whether to use joint residual blocks to generate the residual blocks of the first color component and the residual blocks of the second color component, wherein the bitstream will be decoded by a decoding method according to this disclosure.

[0021] According to one aspect of this disclosure, an electronic device is provided, comprising: a non-transitory computer-readable storage medium; and a processor configured to perform an encoding method according to this disclosure to generate a bit stream, and to store and / or transmit the bit stream.

[0022] It will be understood that the above general description and the following detailed description are merely examples and do not limit this disclosure. Attached Figure Description

[0023] The accompanying drawings, which are incorporated in and form part of this specification, illustrate examples according to this disclosure and, together with this description, serve to explain the principles of this disclosure.

[0024] FIG. 1This is a block diagram illustrating an exemplary system for encoding and decoding video blocks according to some embodiments of the present disclosure.

[0025] FIG. 2 This is a block diagram illustrating an exemplary video encoder according to some embodiments of the present disclosure.

[0026] FIG. 3 This is a block diagram illustrating an exemplary video decoder according to some embodiments of the present disclosure.

[0027] FIG. 4A to FIG. 4E This is a block diagram illustrating how a frame is recursively divided into multiple video blocks of different sizes and shapes according to some embodiments of this disclosure.

[0028] FIG. 5 A workflow for a method for video decoding according to one or more aspects of this disclosure is shown.

[0029] FIG. 6 A workflow for a method for video encoding according to one or more aspects of this disclosure is shown.

[0030] FIG. 7 A workflow for a method for video decoding according to one or more aspects of this disclosure is shown.

[0031] FIG. 8 A workflow for a method for video encoding according to one or more aspects of this disclosure is shown.

[0032] FIG. 9 This is a diagram illustrating a computing environment coupled to a user interface according to some embodiments of the present disclosure. Detailed Implementation

[0033] Referring now to the detailed description, examples of which are illustrated in the accompanying drawings. Numerous non-limiting details are set forth in the following detailed description to aid in understanding the subject matter presented herein. However, various alternatives may be used without departing from the scope of the claims, and the subject matter may be practiced without these specific details. For example, the subject matter presented herein can be implemented on many types of electronic devices with digital video capabilities.

[0034] It should be noted that the terms "first," "second," etc., used in the specification, claims, and drawings of this disclosure are used to distinguish objects and not to describe any specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in sequences other than those shown in the drawings or described in this disclosure.

[0035] FIG. 1 This is a block diagram illustrating an exemplary system 10 for encoding and decoding video blocks in parallel, according to some embodiments of the present disclosure. FIG. 1 As shown, system 10 includes a source device 12 that generates and encodes video data for later decoding by a target device 14. The source device 12 and target device 14 can include any electronic device from a wide variety of electronic devices, including cloud servers, server computers, desktop or laptop computers, tablet computers, smartphones, set-top boxes, digital televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, etc. In some embodiments, the source device 12 and target device 14 are equipped with wireless communication capabilities.

[0036] like FIG. 1 As shown, source device 12 includes a video source 18, a video encoder 20, and an output interface 22. Video source 18 may include sources or combinations of such sources, such as: a video capture device (e.g., a camera), a video archive containing previously captured video, a video feed interface for receiving video from a video content provider, and / or a computer graphics system for generating computer graphics data as source video.

[0037] Video can be encoded by video encoder 20, whether captured, pre-captured, or computer-generated. Encoded video data may include a series of images, each of which may include one or more sample arrays, for example, for monochrome, only luminance (Y); luminance and two chrominance in the YCbCr or YCgCo domain; or green, blue, and red in the GBR (also known as RGB) domain. For ease of reference and terminology in this application, in some embodiments, the variables and terms associated with each set having three sample arrays may be referred to as luminance and chrominance, where the two chrominance arrays may be referred to as Cb and Cr, regardless of the actual color representation used. Video data may be in chrominance format 4:0:0, chrominance format 4:2:0, chrominance format 4:2:2, or chrominance format 4:4:4, but this application is not limited thereto. The bit depth (BitDepth) of the sample array can be an integer ranging from 8 to 16, for example, a BitDepth value of 8, 9, 10, 11, 12, 13, 14, 15, or 16. It should be noted that the value of BitDepth is not limited to this, but can be any other value proposed in the future.

[0038] Encoded video data can be directly transmitted to the target device 14 via link 16 through the output interface 22 of the source device 12. Output interface 22 may include a modem and / or a transmitter. Link 16 may include any type of wireless communication medium or device and / or any type of wired communication medium or device capable of transmitting encoded video data from the source device 12 to the target device 14. The encoded video data may also (or alternatively) be stored on storage device 32 for later access by the target device 14 or by other devices via, for example, input interface 28, for decoding and / or playback. Storage device 32 may include any data storage medium of various distributed or locally accessed data storage media, such as hard disk drives, Blu-ray discs, digital universal discs (DVDs), compact disc read-only memory (CD-ROMs), flash memory, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded video data.

[0039] Target device 14 includes an input interface 28, a video decoder 30, and a display device 34. Input interface 28 may include a receiver and / or a modem and receives encoded video data via link 16. Alternatively, target device 14 may access stored video data from storage device 32 via streaming, downloading, or a combination of both. Encoded video data may include various syntax elements generated by video encoder 20 for use by video decoder 30 when decoding the video data. Display device 34 may be an integrated display device or an external display device configured to communicate with target device 14 and may display the decoded video data to a user.

[0040] The video encoder 20 and video decoder 30 can operate (e.g., encode and decode video data) according to proprietary or industry standards (e.g., Universal Video Codec (VVC), Joint Explore Test Model (JEM), High Efficiency Video Codec (HEVC / H.265), Advanced Video Codec (AVC / H.264), Moving Picture Experts Group (MPEG) codec) or extensions of such standards. It should be understood that this application is not limited to any specific video coding / decoding standard and may be applicable to other current and future video coding / decoding standards.

[0041] The video encoder 20 and video decoder 30 can be implemented as any circuit of a variety of suitable encoder and / or decoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic devices, software, hardware, firmware, or any combination thereof. When partially implemented in software, the electronic device may store instructions for the software in a suitable non-transitory computer-readable medium and use one or more processors to execute the instructions in the hardware to perform the video encoding / decoding operations disclosed in this disclosure. Each of the video encoder 20 and video decoder 30 may be included in one or more encoders or decoders, either of which may be integrated as part of a combined encoder / decoder (CODEC) in the respective device.

[0042] In some implementations, components of source device 12 and / or target device 14 (e.g., FIG. 1 , FIG. 2 and / or FIG. 3 At least some of the components shown can operate in a cloud computing service network such as Software as a Service (SaaS), Platform as a Service (PaaS), or Infrastructure as a Service (IaaS), where the cloud computing service network can provide software, platform, and / or infrastructure.

[0043] In some embodiments, one or more components of the source device 12 and / or target device 14 not included in the cloud computing service network may be located in one or more client devices, and these client devices may communicate with server computers in the cloud computing service network via wireless communication networks or wired communication networks. In one embodiment, at least a portion of the operations described herein may be implemented as a cloud-based service provided by one or more server computers, wherein the one or more server computers are implemented by at least a portion of the components of the source device 12 and / or target device 14 in the cloud computing service network; and one or more other operations described herein may be implemented by one or more client devices. In some embodiments, the cloud computing service network may be a private cloud, a public cloud, or a hybrid cloud. Terms such as “cloud,” “cloud computing,” and “cloud-based” used herein may be used interchangeably without departing from the scope of this disclosure. It should be understood that this disclosure is not limited to implementation in the aforementioned cloud computing service network. Instead, this disclosure may also be implemented in any other type of computing environment currently known or to be developed in the future.

[0044] FIG. 2 This is a block diagram illustrating an exemplary video encoder 20 according to some embodiments described in this application.

[0045] like FIG. 2 As shown, the video encoder 20 includes a video data memory 40, a prediction processing unit 41, a decoded picture buffer (DPB) 64, an adder 50, a transform processing unit 52, a quantization unit 54, and an entropy coding unit 56. The prediction processing unit 41 further includes a motion estimation unit 42, a motion compensation unit 44, a segmentation unit 45, an intra-frame prediction processing unit 46, and an intra-frame block copy (IBC) unit 48. In some embodiments, the video encoder 20 also includes an inverse quantization unit 58, an inverse transform processing unit 60, and an adder 62 for video block reconstruction. A loop filter 63, such as a deblocking filter, can be located between the adder 62 and the DPB 64 to filter block boundaries to remove block artifacts from the reconstructed video. In addition to the deblocking filter, another loop filter (e.g., a sample adaptive offset (SAO) filter, a cross-component sample adaptive offset (CCSAO) filter, and / or an adaptive loop filter (ALF)) can be used to filter the output of the adder 62. It should be noted that this application is not limited to the embodiments described herein regarding CCSAO technology. It can also be applied to situations where one or more samples of any one of the luminance component and two chrominance components (as mentioned above, for ease of reference and terminology in this application, in the YCbCr domain, the luminance component and the two chrominance components can represent Y, Cb, and Cr; in the YCgCo domain, the luminance component and the two chrominance components can represent Y, Cg, and Co; or in the RGB domain, the luminance component and the two chrominance components can represent G, B, and R) are used to select an offset for the sample points of any other component among the luminance component and the two chrominance components, in order to modify the sample points of that other component based on the selected offset. Alternatively, a SAO technology is also provided, which is substantially the same as CCSAO technology, except that in this SAO technology, an offset is selected for the sample points of any one of the luminance component and the two chrominance components based on one or more samples of that component, in order to modify the sample points of that component based on the selected offset. Furthermore, it should be noted that the first component mentioned herein can be either the luminance component or either of the two chrominance components, the second component mentioned herein can be either the luminance component or either of the two chrominance components, and the third component mentioned herein can be the remaining component of the luminance component and the two chrominance components. In some examples, the loop filter can be omitted, and the decoded video block can be directly provided to the DPB 64 by the adder 62. The video encoder 20 can take the form of a fixed or programmable hardware unit, or it can be distributed among one or more of the fixed or programmable hardware units described.

[0046] The video data storage device 40 can store video data encoded by the components of the video encoder 20. For example, it can store data from... FIG. 1The video source 18 shown receives video data from the video data memory 40. The DPB 64 is a buffer that stores reference video data (e.g., reference frames or pictures) used by the video encoder 20 when encoding the video data. The video data memory 40 and DPB 64 can be formed from any of a variety of memory devices. In various examples, the video data memory 40 may be on-chip along with other components of the video encoder 20, or off-chip relative to those components. It should be noted that in the field of video encoding and decoding, the term "frame" can be used as a synonym for the terms "image" or "picture".

[0047] like FIG. 2 As shown, after receiving video data, segmentation unit 45 segments the video data into video blocks. This segmentation may also include segmenting the video frame into strips, tiles (e.g., a collection of video blocks) or other larger coding units (CUs) according to a predefined splitting structure associated with the video data (e.g., a quadtree (QT) structure). A video frame is, or can be considered, a two-dimensional array or matrix of sample points with sample values. Sample points in the array may also be referred to as pixels or image elements (pel). The number of samples in the horizontal and vertical directions (or axes) of the array or image defines the size and / or resolution of the video frame. For example, a video frame can be divided into multiple video blocks using QT segmentation. A video block is again, or can be considered, a two-dimensional array or matrix of sample points with sample values, but its dimension is smaller than that of the video frame. The number of samples in the horizontal and vertical directions (or axes) of the video block defines the size of the video block. By iteratively using, for example, QT segmentation, binary tree (BT) segmentation, or ternary tree (TT) segmentation, or any combination thereof, a video block can be further segmented into one or more block partitions or sub-blocks (which can again form blocks). It should be noted that the term "block" or "video block" as used herein can refer to a portion of a frame or image, particularly a rectangular (square or non-square) portion. Referring, for example, to HEVC and VVC, a block or video block can be or corresponds to a coding tree unit (CTU), CU, prediction unit (PU), or transform unit (TU) and / or can be or corresponds to a corresponding block (e.g., coding tree block (CTB), coding block (CB), prediction block (PB), or transform block (TB)) and / or sub-block.

[0048] Prediction processing unit 41 can select one of several feasible predictive coding modes for the current video block based on error results (e.g., coding rate and distortion level), such as one of several intra-frame or inter-frame predictive coding modes. Prediction processing unit 41 can provide the resulting intra-frame or inter-frame predictive coded block to adder 50 to generate a residual block, and to adder 62 to reconstruct the coded block for subsequent use as part of a reference frame. Prediction processing unit 41 also provides at least one of the syntax elements (e.g., motion vectors, intra-frame or inter-frame mode indicators, segmentation information, and other such syntax information) to entropy coding unit 56.

[0049] To select a suitable intra-predictive coding mode for the current video block, the intra-predictive processing unit 46 may perform intra-predictive coding of the current video block relative to one or more neighboring blocks in the same frame as the current block to be encoded to provide spatial prediction. The motion estimation unit 42 and the motion compensation unit 44 perform inter-predictive coding of the current video block relative to one or more prediction blocks in one or more reference frames to provide temporal prediction. The video encoder 20 may perform multiple coding passes, for example, to select a suitable coding mode for each block of video data.

[0050] In some implementations, motion estimation unit 42 generates a motion vector for the current block based on a predetermined pattern within the video frame sequence during motion estimation. The motion vector can indicate the displacement of a video block within the current frame relative to a predicted block within a reference frame associated with the current block being encoded. The predetermined pattern can designate video frames in the sequence as P-frames or B-frames. In some implementations, the motion vector prediction value (MVP) of the current block is subtracted from the actual motion vector of the current block to produce the motion vector difference (MVD) of the current block, where the MVP can be determined based on motion information from spatially neighboring blocks and / or temporally co-located blocks. The MVP and MVD information can then be encoded into the video bitstream instead of the actual motion vector of the current block. IBC unit 48 can determine the vectors (e.g., block vectors) for IBC encoding / decoding in a similar manner to how motion estimation unit 42 determines the motion vectors for inter-frame prediction, or the block vectors can be determined using motion estimation unit 42. It should be noted that the IBC pattern can be considered an intra-frame prediction pattern, or a prediction pattern other than intra-frame and inter-frame prediction patterns.

[0051] Regarding pixel differences, the predicted block for a video block can be, or can correspond to, a block or reference block of a reference frame considered to closely match the video block to be encoded. Pixel differences can be determined by the sum of absolute differences (SAD), sum of squared differences (SSD), or other difference metrics. In some implementations, the video encoder 20 can compute values ​​for sub-integer pixel positions of the reference frame stored in the DPB 64. For example, the video encoder 20 can interpolate values ​​for quarter-pixel positions, eighth-pixel positions, or other fractional pixel positions of the reference frame. Therefore, the motion estimation unit 42 can perform motion search relative to full-pixel positions and fractional pixel positions and output a motion vector with fractional-pixel accuracy.

[0052] The motion estimation unit 42 determines the motion vector information of a video block in an inter-frame predictive coding frame by comparing the position of the video block with the position of the predicted block of a reference frame selected from either a first reference frame list (list 0) or a second reference frame list (list 1), where each reference frame list identifies one or more reference frames stored in the DPB 64. The motion estimation unit 42 sends the determined motion vector information to the motion compensation unit 44, and then to the entropy coding unit 56.

[0053] Motion compensation performed by motion compensation unit 44 may involve acquiring or generating prediction blocks based on motion vector information determined by motion estimation unit 42. Upon receiving motion vector information for the current video block, motion compensation unit 44 may locate the prediction block pointed to by the motion vector in a reference frame list within a reference frame list, retrieve the prediction block from DPB 64, and forward the prediction block to adder 50. Motion compensation unit 44 may also generate syntax elements associated with video blocks of a video frame for use by video decoder 30 when decoding video blocks of the video frame. Syntax elements may include, for example, syntax elements defining motion vectors for identifying prediction blocks, any flags indicating prediction modes, or any other syntax information described herein. It should be noted that motion estimation unit 42 and motion compensation unit 44 may be highly integrated, but are described separately for conceptual purposes.

[0054] In some implementations, IBC unit 48 may generate vectors and acquire prediction blocks in a manner similar to that described above in conjunction with motion estimation unit 42 and motion compensation unit 44, but these prediction blocks are in the same frame as the current block being encoded, and these vectors are referred to as block vectors rather than motion vectors.

[0055] In other examples, IBC unit 48 may use motion estimation unit 42 and motion compensation unit 44, in whole or in part, to perform such functions for IBC prediction according to the embodiments described herein. In any case, for intra-block copying, in terms of pixel difference, the predicted block may be a block considered to closely match the block to be encoded, the pixel difference may be determined by SAD, SSD, or other difference metrics, and identifying the predicted block may include calculating values ​​for sub-integer pixel positions.

[0056] As an alternative to the inter-frame prediction performed by the motion estimation unit 42 and the motion compensation unit 44 as described above, or the intra-block copy prediction performed by the IBC unit 48, the intra-frame prediction processing unit 46 can perform intra-frame prediction on the current video block. Specifically, the intra-frame prediction processing unit 46 can determine the intra-frame prediction mode for encoding the current block. The intra-frame prediction processing unit 46 can provide information indicating the selected intra-frame prediction mode for the block to the entropy coding unit 56. The entropy coding unit 56 can encode the information indicating the selected intra-frame prediction mode into the bitstream.

[0057] After prediction processing unit 41 determines the prediction block for the current video block, adder 50 forms a residual block by subtracting the pixel values ​​of the prediction block from the pixel values ​​of the current video block (forming a pixel difference). The pixel difference may include a luminance component difference or a chrominance component difference, or both. The residual video data in the residual block may be included in one or more TUs and provided to transform processing unit 52. Transform processing unit 52 transforms the residual video data into residual transform coefficients using one or more transforms (e.g., Discrete Cosine Transform (DCT) or a conceptually similar transform).

[0058] The transform processing unit 52 can send the resulting transform coefficients to the quantization unit 54. The quantization unit 54 quantizes the transform coefficients to further reduce the bit rate. The quantization process can also reduce the bit depth associated with some or all of the coefficients. The degree of quantization can be modified by adjusting the quantization parameters. In some examples, the quantization unit 54 can subsequently perform a scan on the matrix including the quantized transform coefficients. Alternatively, the entropy coding unit 56 can perform the scan.

[0059] After quantization, the entropy coding unit 56 entropy codes the quantized transform coefficients into a video bitstream using, for example, context-adaptive variable-length coding (CAVLC), context-adaptive binary arithmetic coding (CABAC), syntax-based context-adaptive binary arithmetic coding (SBAC), probabilistic interval segmented entropy (PIPE) coding, or another entropy coding method or technique. The encoded bitstream can then be sent to, for example,... FIG. 1 The video decoder 30 shown, or archived in, for example FIG. 1The data is stored in storage device 32 for later transmission to or retrieval by video decoder 30. Entropy coding unit 56 can also entropy code the motion vectors and other syntax elements used for the current video frame.

[0060] The inverse quantization unit 58 and the inverse transform processing unit 60 apply inverse quantization and inverse transform, respectively, to reconstruct the residual block in the pixel domain for generating a reference block for predicting other video blocks.

[0061] VVC supports Joint Codec for Chromatic Residuals (JCCR) tools, where chroma residuals are jointly encoded and decoded. When JCCR mode is activated for the current chroma TU, only a single joint chroma residual block is signaled, and the residual block for Cb (resCb) and the residual block for Cr (resCr) are derived. More details are captured from JVET-V2002 below.

[0062] Joint coding of chroma residuals (JCCR)

[0063] VVC supports Joint Coding / Decoding (JCCR) tools for chroma residuals, where chroma residuals are jointly encoded and decoded. The use (activation) of a JCCR mode is indicated by the TU-level flag `tu_joint_cbcr_residual_flag`, and the selected mode is implicitly indicated by the chroma CBF (coded block flag). The flag `tu_joint_cbcr_residual_flag` is present if any one or both chroma CBFs of the TU are equal to 1. In the PPS (Picture Parameter Set) and slice headers, chroma QP offset values ​​are signaled for JCCR modes to distinguish them from the usual chroma QP offset values ​​signaled for regular chroma residual encoding / decoding modes. These chroma QP offset values ​​are used to derive the chroma QP values ​​for some blocks encoded using the JCCR mode. The JCCR mode has three sub-modes. When the corresponding JCCR submode (submode 2 in Table 1) is active in the TU, the chroma QP offset is added to the applied luminance-derived chroma QP during quantization and decoding in that TU. For other JCCR submodes (submodes 1 and 3 in Table 1), the chroma QP is derived in the same manner as regular Cb or Cr blocks. Table 1 depicts the reconstruction process of the chroma residual blocks (resCb and resCr) from the transmitted transform blocks. When a JCCR mode is active, the individual joint chroma residuals (resJointC[x][y] in Table 1) are signaled, and information such as tu_cbf_cb, tu_cbf_cr, and CSign (which are the sign values ​​specified in the stripe header) is taken into account to derive the residual blocks (resCb) for Cb and (resCr) for Cr.

[0064] On the encoder side, the joint chromaticity components are derived as explained below. Depending on the mode (listed in the table below), resJointC{1,2} is generated by the encoder as follows:

[0065] If mode equals 2 (with a single joint residual block resJointC, reconstructed chroma residual block resCb = resJointC, and reconstructed chroma residual block resCr = CSign * resJointC), then the joint residual is determined according to the following formula:

[0066] resJointC[x][y]=(ResCb[x][y]+CSign*ResCr[x][y]) / 2

[0067] The value CSign is a sign value (+1 or -1) specified in the strip header. resJointC[x][y] is the joint residual of the transmitted x and y positions. ResCb[x][y] and ResCr[x][y] are the actual residuals of the x and y positions of Cb and Cr formed on the encoder side by, for example, adder 50.

[0068] Otherwise, if the mode is equal to 1 (with a single joint residual block resJointC, reconstructed chroma residual block resCb = resJointC, and reconstructed chroma residual block resCr = (CSign * resJointC) / 2), then the joint residual is determined according to the following formula:

[0069] resJointC[x][y]=(4*ResCb[x][y]+2*CSign*ResCr[x][y]) / 5

[0070] Otherwise (mode equals 3, i.e., with a single joint residual block resJointC, the reconstructed chroma residual block resCr = resJointC, and the reconstructed chroma residual block resCb = (CSign * resJointC) / 2), the joint residual is determined according to the following formula:

[0071] resJointC[x][y]=(4*ResCr[x][y]+2*CSign*ResCb[x][y]) / 5

[0072]

[0073] Table 1 - Reconstruction of chromaticity residuals

[0074] The three joint chroma codec submodes described in Table 1 are supported only in the I-strip. In the P and B-strips, only mode 2 is supported. Therefore, in the P and B-strips, the syntax element tu_joint_cbcr_residual_flag exists only when both chroma CBF values ​​are 1.

[0075] The JCCR mode can be combined with the Chroma Transform Skip (TS) mode. To accelerate encoder decisions, the JCCR transform selection depends on whether the independent encoding and decoding of the Cb and Cr components chooses DCT-2 or TS as the optimal transform, and whether there are non-zero coefficients in the independent chroma encoding and decoding. Specifically, if one chroma component chooses DCT-2 (or TS) and the other component is all zero, or if both chroma components choose DCT-2 (or TS), then only DCT-2 (or TS) will be considered in JCCR encoding. Otherwise, if one component chooses DCT-2 and the other component chooses TS, then both DCT-2 and TS will be considered in JCCR encoding.

[0076] Adder 62 adds the reconstructed residual block to the predicted block to produce a reference block to be stored in DPB 64.

[0077] FIG. 3 This is a block diagram illustrating an exemplary video decoder 30 according to some embodiments of this application. The video decoder 30 includes a video data memory 79, an entropy decoding unit 80, a prediction processing unit 81, an inverse quantization unit 86, an inverse transform processing unit 88, an adder 90, and a DPB 92. The prediction processing unit 81 includes a motion compensation unit 82, an intra-frame prediction unit 84, and an IBC unit 85. The video decoder 30 can perform operations in conjunction with the above. FIG. 2 The decoding process described for the video encoder 20 is essentially the inverse of the encoding process. For example, the motion compensation unit 82 can generate prediction data based on the motion vectors received from the entropy decoding unit 80, while the intra-frame prediction unit 84 can generate prediction data based on the intra-frame prediction mode indicator received from the entropy decoding unit 80.

[0078] In some examples, the units of video decoder 30 may be assigned tasks to perform embodiments of the present disclosure. Furthermore, in some examples, embodiments of the present disclosure may be distributed across one or more units of video decoder 30.

[0079] Video data memory 79 can store video data, such as encoded video bitstreams, that will be decoded by other components of video decoder 30. The video data stored in video data memory 79 can be obtained, for example, from storage device 32, from a local video source (e.g., a camera), via wired or wireless network communication of video data, or by accessing a physical data storage medium (e.g., a flash drive or hard disk). Video data memory 79 may include an encoded picture buffer (CPB) that stores encoded video data from the encoded video bitstream. The DPB 92 of video decoder 30 stores reference video data for use by video decoder 30 when decoding video data. Video data memory 79 and DPB 92 can be formed of any memory device from a variety of memory devices, such as dynamic random access memory (DRAM) (including synchronous DRAM (SDRAM)), magnetoresistive RAM (MRAM), resistive RAM (RRAM), or other types of memory devices. In some examples, video data memory 79 may be on-chip along with other components of video decoder 30, or off-chip relative to those components.

[0080] During the decoding process, the video decoder 30 receives an encoded video bitstream representing video blocks of encoded video frames and associated syntax elements. The video decoder 30 may receive syntax elements at the video frame level and / or the video block level. The entropy decoding unit 80 performs entropy decoding on the bitstream to generate quantization coefficients, motion vector information or intra-prediction mode indicators, and other syntax elements. The entropy decoding unit 80 then forwards the motion vectors or intra-prediction mode indicators, and other syntax elements to the prediction processing unit 81.

[0081] When a video frame is encoded as an intra-predictive coded (I) frame or for intra-coded prediction blocks in other types of frames, the intra-predictive unit 84 can generate prediction data for the video block of the current video frame based on the intra-predictive mode transmitted by signaling and reference data from the previous decoded block of the current frame.

[0082] When a video frame is encoded as an inter-frame predictive coded (i.e., B or P) frame, the motion compensation unit 82 generates one or more prediction blocks for the current video frame based on motion vector information and other syntax elements received from the entropy decoding unit 80. Each of the prediction blocks can be generated from a reference frame within a reference frame list. The video decoder 30 can construct the reference frame list, i.e., list 0 and list 1, based on the reference frames stored in the DPB 92 using a default construction technique.

[0083] In some examples, when encoding a video block according to the IBC mode described herein, IBC unit 85 generates a predicted block for the current video block based on block vector information and other syntax elements received from entropy decoding unit 80. The predicted block can be located within a reconstructed region of the same image as the current video block, defined by video encoder 20.

[0084] Motion compensation unit 82 and / or IBC unit 85 determine prediction information for video blocks in the current video frame by parsing vector information and other syntax elements, and then use the prediction information to generate prediction blocks for the current video blocks. For example, motion compensation unit 82 uses some syntax elements from the received syntax elements to determine the prediction mode for encoding video blocks in the video frame, the inter-frame prediction frame type (e.g., B or P), construction information for one or more reference frames in the reference frame list for the frame, motion vectors for each inter-frame prediction encoded video block in the frame, the inter-frame prediction state for each inter-frame prediction encoded video block in the frame, and other information for decoding video blocks in the current video frame.

[0085] Similarly, IBC unit 85 can use some of the syntax elements in the received syntax elements, such as flags, to determine which video blocks in the current video frame are predicted using the IBC mode, which video blocks in the frame are in the reconstruction region and should be stored in the DPB 92, the block vector for each IBC predicted video block in the frame, the IBC prediction state for each IBC predicted video block in the frame, and other information for decoding the video blocks in the current video frame.

[0086] The motion compensation unit 82 can also perform interpolation using interpolation filters, such as those used by the video encoder 20 during encoding of video blocks, to calculate interpolated values ​​for sub-integer pixels of the reference block. In this case, the motion compensation unit 82 can determine the interpolation filters used by the video encoder 20 based on the received syntax elements and use these interpolation filters to generate the prediction block.

[0087] Similar to the process of selecting a prediction block in a reference frame during inter-frame prediction of a video block, both the video encoder 20 and the video decoder 30 need to employ a set of rules to construct a motion vector candidate list (also known as a "merging list") for the current block using those potential candidate motion vectors associated with spatially neighboring blocks and / or temporally co-located blocks. Then, a member is selected from the motion vector candidate list as the motion vector prediction value for the current block. By doing so, it is not necessary to send the motion vector candidate list itself from the video encoder 20 to the video decoder 30, and the index of the selected motion vector prediction value within the motion vector candidate list is sufficient for both the video encoder 20 and the video decoder 30 to encode and decode the current block using the same motion vector prediction value from the motion vector candidate list.

[0088] The dequantization unit 86 dequantizes the quantized transform coefficients, which are provided in the bitstream and entropy decoded by the entropy decoding unit 80, using the same quantization parameters calculated by the video encoder 20 for each video block in the video frame. The inverse transform processing unit 88 applies an inverse transform (e.g., inverse DCT, inverse integer transform, or a conceptually similar inverse transform process) to the transform coefficients to reconstruct the residual block in the pixel domain.

[0089] Adder 90 reconstructs the decoded video block for the current video block by adding the residual block from inverse transform processing unit 88 to the corresponding prediction block. A loop filter 91 (e.g., a deblocking filter, SAO filter, CCSAO filter, and / or ALF) may be located between adder 90 and DPB 92 for further processing of the decoded video block. In some examples, loop filter 91 may be omitted, and the decoded video block may be directly provided to DPB 92 by adder 90. The decoded video block in a given frame is then stored in DPB 92, which stores reference frames for subsequent motion compensation of the next video block. DPB 92, or a separate memory device from DPB 92, may also store the decoded video for later presentation on a display device (e.g., ...). FIG. 1 On the display device 34).

[0090] In a typical video encoding and decoding process, a video sequence usually consists of an ordered set of frames or images. Each frame may include three sample arrays, denoted as SL, SCb, and SCr. SL is a two-dimensional array of luma samples. SCb is a two-dimensional array of chroma samples (Cb). SCr is a two-dimensional array of chroma samples (Cr). In other instances, a frame may be monochrome and therefore consist of only a two-dimensional array of luma samples.

[0091] like FIG. 4AAs shown, the video encoder 20 (or more specifically, the segmentation unit 45) generates an encoded representation of a frame by first segmenting the frame into a set of CTUs. A video frame may include an integer number of CTUs ordered consecutively from left to right and top to bottom in raster scan order. Each CTU is the largest logical coding unit, and the width and height of the CTU are signaled by the video encoder 20 in a sequence parameter set such that all CTUs in the video sequence have the same size, such as 128×128, 64×64, 32×32, and 16×16. However, it should be noted that this application is not limited to a specific size. FIG. 4B As shown, each CTU may include a CTB for luma samples, two corresponding CTBs for chroma samples, and syntax elements for encoding the samples of the CTBs. The syntax elements describe the properties of different types of units of the encoded pixel block and how the video sequence can be reconstructed at the video decoder 30, including inter-frame prediction or intra-frame prediction, intra-frame prediction mode, motion vectors, and / or other parameters. In a monochrome image or an image with three separate color planes, the CTU may include a single CTB and syntax elements for encoding the samples of that CTB. The CTB may be an N×N sample block.

[0092] To achieve better performance, the video encoder 20 can recursively perform tree segmentation on the CTU's CTB, such as binary tree segmentation, ternary tree segmentation, quadtree segmentation, or a combination thereof, and divide the CTU into smaller CUs. For example... FIG. 4C As described, the 64×64 CTU 400 is first divided into four smaller CUs, each with a block size of 32×32. Of these four smaller CUs, CU 410 and CU 420 are each divided into four CUs with a block size of 16×16. The two 16×16 CUs, CU430 and CU440, are further divided into four CUs with a block size of 8×8. FIG. 4D Depicting as shown FIG. 4C The final result of the CTU 400 partitioning process described in the figure is a quadtree data structure, where each leaf node of the quadtree corresponds to a CU of a corresponding size ranging from 32×32 to 8×8. Similar to... FIG. 4B The CTU depicted in the image may include, for example, a CB for each chromaticity sample and two corresponding CBs for the luminance sample, as well as syntax elements for encoding the samples of the CB. In a monochrome image or an image with three separate color planes, a CU may include a single CB and a syntax structure for encoding the samples of the CB. It should be noted that... FIG. 4C and FIG. 4DThe quadtree partitioning depicted is for illustrative purposes only, and a CTU can be split into multiple CUs based on quadtree partitioning / ternary tree partitioning / binary tree partitioning to adapt to varying local characteristics. In multi-type tree structures, a CTU is partitioned according to a quadtree structure, and each quadtree leaf CU can be further partitioned according to a binary and / or ternary tree structure. FIG. 4E As shown, a CB with width W and height H has five possible segmentation types: quadrilateral segmentation, horizontal binary segmentation, vertical binary segmentation, horizontal ternary segmentation, and vertical ternary segmentation.

[0093] In some implementations, the video encoder 20 may further segment the CB of the CU into one or more M×N PBs. A PB is a rectangular (square or non-square) block of samples to which the same prediction (inter-frame, intra-frame, etc.) is applied. The PU of the CU may include a PB of luma samples, two corresponding PBs of chroma samples, and syntax elements for predicting the PBs. In a monochrome image or an image with three separate color planes, a PU may include a single PB and a syntax structure for predicting the PBs. The video encoder 20 may generate predicted luma blocks, predicted Cb blocks, and predicted Cr blocks for each PU of the CU, representing the luma PB, Cb PB, and Cr PB.

[0094] In addition, such as FIG. 4C As shown, the video encoder 20 can use quadtree partitioning to decompose the luminance residual block, Cb residual block, and Cr residual block of the CU into one or more luminance transform blocks, Cb transform blocks, and Cr transform blocks, respectively. A transform block is a rectangular (square or non-square) sample block to which the same transform is applied. The TU of the CU can include a TB for luminance samples, two corresponding TBs for chrominance samples, and syntax elements for transforming the TBs. Therefore, each TU of the CU can be associated with a luminance TB, a Cb TB, and a Cr TB. In some examples, the luminance TB associated with a TU can be a sub-block of the CU's luminance residual block. The Cb TB can be a sub-block of the CU's Cb residual block. The Cr TB can be a sub-block of the CU's Cr residual block. In a monochrome image or an image with three separate color planes, the TU can include a single TB and syntax structures for transforming the samples of that TB.

[0095] In the current JCCR design, the transmitted joint residual block is associated with two residual blocks for the chrominance components. As shown in Table 1, the weights or relationships between the two chrominance components are indicated by the chrominance cbf. However, the residual relationship between Cb and Cr can be more complex than just three weights w. Table 2 below shows three patterns consistent with Table 1, where CSign is exemplarily -1. ResCb and ResCr are the actual residual blocks for Cb and Cr formed on the encoder side by, for example, adder 50. resCb and resCr are the reconstructed chrominance residual blocks. resJointC is the joint residual block notified by the signal.

[0096]

[0097] Table 2

[0098] In this disclosure, multiple weights are proposed for joint encoding and decoding of at least two color components. On the encoder side, one or more residual blocks are transmitted, wherein each residual block is associated with at least two residual blocks for different color components. On the decoder side, the relationship or weight between the two residual blocks is explicitly notified by the encoder via signaling, or implicitly derived from available prediction samples, reconstructed samples, or residual samples.

[0099] In some examples, the encoder may explicitly signal the weights w: sending a residual block for both the Cb and Cr components, and signaling a syntax element to indicate the corresponding weight w = resCr / resCb for each chroma TU. On the decoder side, the received joint residual block resJointC is directly applied to the residual block resCb for reconstructing the Cb component, and the joint residual block resJointC multiplied by the indicated weight w is applied to the residual block resCr for reconstructing the Cr component, as shown in Table 3.

[0100]

[0101] Table 3

[0102] The index of the weight table can be predefined. Table 4 shows weights such as -8, -4, -8 / 3, -2, -8 / 5, -4 / 3, -8 / 7, -1, -7 / 8, -3 / 4, -5 / 8, -1 / 2, -3 / 8, -1 / 4, and -1 / 8. The symbol (-) can be identified and signaled in the PH (Picture Header) as in VVC, or it can be included in the weight table and signaled by chroma TU.

[0103]

[0104] Table 4

[0105] According to embodiments of this disclosure, on the encoder side, in order to determine the optimal joint residual block (written as ResJ in the following formula) that minimizes the MSE (Mean Squared Error) of both Cb and Cr, the following problem can be solved:

[0106] ResCr=w*ResCb

[0107] Find ResJ, so that

[0108] min[(ResCb-ResJ) 2 +(ResCr-w*ResJ) 2 ]

[0109] -2*ResCb+2*ResJ-2w*ResCr+2w 2 *ResJ=0

[0110] ResJ+w 2 *ResJ=ResCb+w*ResCr

[0111] ResJ=(ResCb+w*ResCr) / (1+w 2 )

[0112] Table 5 shows the optimal ResJ given the available target raw ResCb and ResCr on the encoder side. In some examples, the weight w = resCr / resCb = 8 / 3, where the optimal...

[0113]

[0114]

[0115] Table 5

[0116] In some examples, the above method can be extended. Instead of always applying weights to Cr, weights can be applied to Cb or Cr to derive residuals for other components whose absolute weight values ​​are always less than 1. It is similar to VVC JCCR in Table 1.

[0117] In one or more examples, weight indices 1–7 can indicate the weights (absolute weight values ​​1 / 8–7 / 8) to be applied to Cb, instead of always applying weight indices 1–15 (absolute weight values ​​8–1 / 8) to Cr. The encoder or decoder can apply the transmitted joint residual to one component, where the other component has an absolute weight value less than 1. This method can improve residual accuracy because it signals a finer joint residual.

[0118] In some examples, the weight index can be signaled when both cbf Cb and cbf Cr are 1, where the JCCR flag is 1. In this case, cbf Cb and cbf Cr are 01 or 10, and the encoder does not need to signal the JCCR flag (returning to the original meaning of cbf, the Cb and Cr blocks have no residuals).

[0119] Since signaling weight-related syntax elements requires more bits, it can potentially reduce encoding / decoding efficiency. Therefore, implicit derivation methods can be applied to determine weights or relationships, where the encoder does not need to use signaling bits to select weights.

[0120] In some examples, neighboring Cb and Cr residual samples (e.g., an L-shape with M rows at the top and N columns at the left, where M and N are positive integers; or only the M rows at the top, or only the N columns at the left, where M and N are positive integers) are used to derive weights. To implicitly derive or select weights or patterns from a candidate list (e.g., {-1 / 2, -1, -2}, or other candidate lists containing multiple weight candidates), template costs can be calculated by applying each weight to the residuals of neighboring Cb and Cr samples (i.e., template samples). Template costs indicate the degree of difference between the product of resCb and w of the samples in the template and resCr. Sum of Absolute Differences (SAD) / Sum of Squared Differences (SSE) / Sum of Absolute Transform Differences (SATD) can be used to calculate template costs. For example, for each w, template costs in the form of SAD, SSE, or SATD can be calculated based on the difference resCb*w-resCr. In the SAD example, for each weight candidate, the decoder or encoder can generate a SAD (sum of |resCb*w - resCr|) template cost for each Cb,Cr neighboring sample residual. In this example, the decoder or encoder can choose the weight with the minimum SAD as the final weight for reconstructing the current residual block. In this example, the SAD is calculated using |resCb*w - resCr|, and the weights are applied to the residual block using resCr = resCb*w.

[0121]

[0122]

[0123] Table 6

[0124] Because neighboring residual samples may not always be relevant to the current block samples, the above example may not derive the expected weights in some cases or under certain conditions. For example, if the neighboring block used as the template has Cb and Cr residuals, the weights can be directly derived using the above residuals. However, if the neighboring block used as the template does not have Cb or Cr residuals (cbf Cb and / or cbf Cr = 0), weights cannot be derived for the current block. Therefore, instead of directly using the real neighboring sample residuals, pseudo-neighboring residuals can be generated to derive the weights.

[0125] In some examples, in addition to generating predicted samples in the current chroma block, the intra / inter-frame prediction mode for the current chroma block is applied to neighboring template samples. This produces pseudo-nearest neighbor predicted samples. The decoder or encoder then subtracts the neighboring predicted samples from the neighboring reconstructed samples. This produces pseudo-nearest neighbor residual samples. Pseudo-nearest neighbor residual samples can be used to implicitly derive weights.

[0126] In some examples, the intra / inter-frame prediction mode used for the current chroma block can be a cross-component linear model (CCLM), a cross-component convolutional model (CCCM), or an inter-frame CCCM. In this case, the linear or convolutional model used for the current chroma block can also be applied to neighboring reconstructed samples to generate pseudo-neighboring predicted samples. The decoder or encoder then subtracts the neighboring predicted samples from the neighboring reconstructed samples and implicitly derives the weights using the pseudo-residual samples.

[0127] In some examples, the intra-prediction mode used for the current chroma block can be an intra / inter-frame mode with block vectors (BV) or motion vectors (MV), such as the intra-block copy (IBC) mode. In such cases, in addition to the reference block samples, the decoder or encoder also acquires template samples adjacent to the reference block. In one or more examples, the decoder or encoder may extract not only the reference block samples but also the L-shaped template samples adjacent to the reference block. To derive the weights, the decoder or encoder subtracts the extracted L-shaped template samples (pseudo-predicted samples) from the neighboring reconstructed samples and uses the resulting pseudo-residual samples to derive the weights.

[0128] In some examples, the intra-prediction mode used for the current block can be a regular intra-prediction mode (e.g., planar, DC, orientation). In such cases, external stencil samples can be used to generate internal stencil prediction samples. In one or more examples, the current block can be the current chroma block. In one or more examples, the decoder or encoder can use a second L-shaped stencil sample to intra-predict (e.g., planar, DC, or orientation) the first (nearest) L-shaped stencil sample. The intra-prediction mode can be the same as the intra-prediction mode for the current block. The decoder or encoder subtracts the first (nearest) L-shaped stencil sample (pseudo-predicted sample) from the neighboring reconstructed samples to generate pseudo-residual samples and uses the generated pseudo-residual samples to derive weights.

[0129] As mentioned above, when template samples are available, the implicit derivation method does not require signaling bits to select weights. Therefore, the current JCCR syntax can be improved when the implicit derivation method is applied. As shown in the table below, the current JCCR (“anchor” indicates the existing method) requires signaling 2-3 bits for 2 Cb / Cr block cbfs and 1 JCCR flag.

[0130]

[0131]

[0132] Table 7

[0133] In some examples, the modified improved syntax is shown in the right column of Table 7. The joint cbf flag is signaled to indicate whether both the Cb and Cr residuals are 0 (i.e., neither the Cb nor Cr residuals exist). If it is determined that neither the Cb nor Cr residuals exist, the current block can be reconstructed directly using the predicted blocks of Cb and Cr. If not (i.e., at least one of the Cb and Cr residuals exists), the JCCR flag is signaled to indicate whether implicit JCCR is on or off. If off (e.g., "10"), the decoder must further decode the cbf for Cb (cbf Cb) and determine whether to decode cbf Cr based on the value of cbf Cb. If cbf Cb is 1 (e.g., "101"), then cbf Cr needs to be decoded to determine if it is 1 (if it is 1, it means there are Cb and Cr residuals, and the current block needs to be reconstructed based on both; if it is 0, it means there are no Cr residuals, so the current block needs to be reconstructed based on the Cb residual). Otherwise, if cbf Cb is 0 (e.g., "100"), cbf Cr is inferred to be 1, and the current block needs to be reconstructed based on the Cr residual. When JCCR is off, the modified syntax requires 1 to 3 bits for cbf Cb, cbf Cr, and one JCCR flag.

[0134] In some examples of the modified syntax, when JCCR is enabled (e.g., "11") and a neighboring template is available, no additional bits need to be sent. Otherwise, if the template is unavailable, the decoder must further decode cbfCb and / or cbfCr, as in the case described above where JCCR is disabled. Note that when JCCR is enabled, the modified syntax requires 2–3 bits for cbfCb, cbfCr, and 1 JCCR flag. In video frames, templates for most video blocks are available and fall within the 2-bit range (e.g., "11x"), therefore the average number of bits required is less than that of the anchor (2–3 bits), which improves encoding / decoding efficiency.

[0135] For implicitly derived methods, the decoder must check or determine the availability of neighboring templates during the parsing phase. This may include determining whether the current block is at a picture or CTU boundary, a specific prediction mode, the block size of the current block, etc.

[0136] In some examples, if the current block is at an image or CTU boundary, the neighboring template is marked as unavailable. A single-sided boundary can still be marked as available because the weights can be derived from the single-sided template.

[0137] In some examples, when the prediction mode of the current block is not one of Cross-Component Linear Model (CCLM), Cross-Component Convolutional Model (CCCM), Inter-Frame CCCM, Intra-Frame Block Copy (IBC), or Inter-Frame Mode, the neighboring template is marked as unavailable.

[0138] In some examples, neighboring templates are marked as unavailable when the current block size is within a predefined range. For instance, neighboring templates are marked as unavailable when the current block size is greater than or less than a threshold.

[0139] FIG. 5 The workflow of a method 500 for video decoding according to one or more aspects of this disclosure is shown.

[0140] At step 510, method 500 includes obtaining a joint residual block for generating a residual block of a first color component and a residual block of a second color component associated with the current block to be reconstructed.

[0141] At step 520, method 500 includes determining a weight candidate list comprising a plurality of weight candidates, and for each of the plurality of weight candidates, calculating a template cost for the template of the current block, wherein the template cost indicates the degree of difference between: the product of the residual value of the second color component of each sample point in the template and the weight candidate, and the residual value of the first color component of each sample point in the template.

[0142] At step 530, method 500 includes selecting weight candidates for the residual block used to generate the first color component based on the calculated template cost.

[0143] At step 540, method 500 includes generating a residual block for the second color component based on the obtained joint residual block, and generating a residual block for the first color component based on the joint residual block and the selected weight candidate.

[0144] In one example, the template includes sample points in the M rows above the current block, where M is a positive integer; or the template includes sample points in the N columns to the left of the current block, where N is a positive integer; or the template includes sample points in the M rows above and the N columns to the left of the current block, where M and N are positive integers.

[0145] In one example, calculating the template cost of the current block's template for each of the plurality of weight candidates includes: using one of the sum of absolute differences (SAD), sum of squared differences (SSE), and sum of absolute transform differences (SATD), and calculating the template cost of the current block's template based on the difference between: the product of the residual value of the second color component of each sample point in the template and the weight candidate, and the residual value of the first color component of the sample point.

[0146] In one example, the first color component is the Cr component, and the second color component is the Cb component.

[0147] In one example, calculating the template cost of the current block's template for each of the plurality of weight candidates includes: in response to determining that the residual value of the second color component of each sample point in the template and / or the residual value of the first color component of each sample point in the template does not exist, generating a predicted value of the second color component of each sample point in the template and a predicted value of the first color component of each sample point in the template based on the sample prediction mode of the current block; determining the residual value of the second color component of each sample point in the template based on the reconstructed value of the second color component of each sample point in the template and the generated predicted value of the second color component of each sample point in the template; determining the residual value of the first color component of each sample point in the template based on the reconstructed value of the first color component of each sample point in the template and the generated predicted value of the first color component of each sample point in the template; and calculating the template cost based on the determined residual value of the second color component and the determined residual value of the first color component.

[0148] In one example, the sample prediction modes include: cross-component linear model (CCLM), cross-component convolutional model (CCCM), inter-frame CCCM, intra-frame block copy (IBC), inter-frame mode, planar intra-frame mode, DC intra-frame mode, and directional intra-frame mode.

[0149] FIG. 6 A workflow for a method 600 for video encoding according to one or more aspects of this disclosure is shown.

[0150] At step 610, method 600 includes determining and transmitting a joint residual block in a bitstream, the joint residual block being used to generate a residual block of a first color component and a residual block of a second color component associated with the current block to be reconstructed.

[0151] At step 620, method 600 includes determining a weight candidate list comprising a plurality of weight candidates, and for each of the plurality of weight candidates, calculating a template cost for the template of the current block, wherein the template cost indicates the degree of difference between: the product of the residual value of the second color component of each sample point in the template and the weight candidate, and the residual value of the first color component of each sample point in the template.

[0152] At step 630, method 600 includes selecting weight candidates for the residual block used to generate the first color component based on the calculated template cost.

[0153] At step 640, method 600 includes generating a residual block for the second color component based on the obtained joint residual block, and generating a residual block for the first color component based on the joint residual block and the selected weight candidate.

[0154] In one example, the template includes sample points in the M rows above the current block, where M is a positive integer; or the template includes sample points in the N columns to the left of the current block, where N is a positive integer; or the template includes sample points in the M rows above and the N columns to the left of the current block, where M and N are positive integers.

[0155] In one example, calculating the template cost of the current block's template for each of the plurality of weight candidates includes: using one of the sum of absolute differences (SAD), sum of squared differences (SSE), and sum of absolute transform differences (SATD), and calculating the template cost of the current block's template based on the difference between: the product of the residual value of the second color component of each sample point in the template and the weight candidate, and the residual value of the first color component of the sample point.

[0156] In one example, the first color component is the Cr component, and the second color component is the Cb component.

[0157] In one example, calculating the template cost of the current block's template for each of the plurality of weight candidates includes: in response to determining that the residual value of the second color component of each sample point in the template and / or the residual value of the first color component of each sample point in the template does not exist, generating a predicted value of the second color component of each sample point in the template and a predicted value of the first color component of each sample point in the template based on the sample prediction mode of the current block; determining the residual value of the second color component of each sample point in the template based on the reconstructed value of the second color component of each sample point in the template and the generated predicted value of the second color component of each sample point in the template; determining the residual value of the first color component of each sample point in the template based on the reconstructed value of the first color component of each sample point in the template and the generated predicted value of the first color component of each sample point in the template; and calculating the template cost based on the determined residual value of the second color component and the determined residual value of the first color component.

[0158] In one example, the sample prediction modes include: cross-component linear model (CCLM), cross-component convolutional model (CCCM), inter-frame CCCM, intra-frame block copy (IBC), inter-frame mode, planar intra-frame mode, DC intra-frame mode, and directional intra-frame mode.

[0159] FIG. 7 The workflow of a method 700 for video decoding according to one or more aspects of this disclosure is shown.

[0160] At step 710, method 700 includes determining a first syntax element indicating whether neither the residual block of the first color component nor the residual block of the second color component associated with the current block to be reconstructed exists.

[0161] At step 720, method 700 includes determining a second syntax element in response to determining that the first syntax element indicating that the residual block of the first color component and the residual block of the second color component are not both absent, the second syntax element indicating whether a joint residual block is used to generate the residual block of the first color component and the residual block of the second color component.

[0162] In one example, method 700 further includes performing the steps of method 500 to generate the residual blocks of the first color component and the second color component in response to determining that the second syntax element indicates that the joint residual block is used to generate the residual blocks of the first color component and the second color component, and in response to determining that the template of the current block is available.

[0163] In one example, method 700 further includes reconstructing the current block based on the prediction block of the first color component and the prediction block of the second color component in response to determining that neither the residual block of the first color component nor the residual block of the second color component exists, as indicated by the first syntax element.

[0164] In one example, method 700 further includes determining a third syntax element in response to determining that the second syntax element indicates that the joint residual block is not used to generate the residual block of the first color component and the residual block of the second color component, wherein the third syntax element indicates whether the residual block of the second color component exists.

[0165] In one example, method 700 further includes reconstructing the current block based on the residual block of the first color component in response to determining that the third syntax element indicates that the residual block of the second color component does not exist.

[0166] In one example, method 700 further includes determining a fourth syntax element in response to determining that the third syntax element indicates the existence of a residual block for the second color component, wherein the fourth syntax element indicates whether a residual block for the first color component exists; reconstructing the current block based on the residual block for the first color component and the residual block for the second color component in response to determining that the fourth syntax element indicates the existence of a residual block based on the first color component; and reconstructing the current block based on the residual block for the second color component in response to determining that the fourth syntax element indicates that a residual block based on the first color component does not exist.

[0167] In one example, method 700 further includes determining a third syntax element in response to determining that the second syntax element indicates the use of the joint residual block to generate a residual block for the first color component and a residual block for the second color component, and in response to determining that the template for the current block is unavailable, wherein the third syntax element indicates whether a residual block for the second color component exists.

[0168] In one example, method 700 further includes reconstructing the current block based on the residual block of the first color component in response to determining that the third syntax element indicates that the residual block of the second color component does not exist.

[0169] In one example, method 700 further includes determining a fourth syntax element in response to determining that the third syntax element indicates the existence of a residual block for the second color component, wherein the fourth syntax element indicates whether a residual block for the first color component exists; reconstructing the current block based on the residual block for the first color component and the residual block for the second color component in response to determining that the fourth syntax element indicates the existence of a residual block based on the first color component; and reconstructing the current block based on the residual block for the second color component in response to determining that the fourth syntax element indicates that a residual block based on the first color component does not exist.

[0170] In one example, the template for the current block is determined to be unavailable based on at least one of the following: the current block is at the boundary of an image or CTU; the sample prediction mode of the current block is not one of Cross-Component Linear Model (CCLM), Cross-Component Convolutional Model (CCCM), Inter-Frame CCCM, Intra-Frame Block Copy (IBC), or Inter-Frame Mode; and the size of the current block is within a predefined range.

[0171] FIG. 8 The workflow of a method 800 for video encoding according to one or more aspects of this disclosure is shown.

[0172] At step 810, method 800 includes determining a first syntax element indicating whether neither the residual block of the first color component nor the residual block of the second color component associated with the current block to be reconstructed exists.

[0173] At step 820, method 800 includes determining a second syntax element in response to determining that the first syntax element indicating that the residual block of the first color component and the residual block of the second color component are not both absent, the second syntax element indicating whether a joint residual block is used to generate the residual block of the first color component and the residual block of the second color component.

[0174] In one example, method 800 further includes performing the steps of method 600 to generate the residual blocks of the first color component and the second color component in response to determining that the second syntax element indicates that the joint residual block is used to generate the residual blocks of the first color component and the second color component, and in response to determining that the template of the current block is available.

[0175] In one example, method 800 further includes reconstructing the current block based on the prediction block of the first color component and the prediction block of the second color component in response to determining that neither the residual block of the first color component nor the residual block of the second color component exists, as indicated by the first syntax element.

[0176] In one example, method 800 further includes determining a third syntax element in response to determining that the second syntax element indicates that the joint residual block is not used to generate the residual block of the first color component and the residual block of the second color component, wherein the third syntax element indicates whether the residual block of the second color component exists.

[0177] In one example, method 800 further includes reconstructing the current block based on the residual block of the first color component in response to determining that the third syntax element indicates that the residual block of the second color component does not exist.

[0178] In one example, method 800 further includes determining a fourth syntax element in response to determining that the third syntax element indicates the existence of a residual block for the second color component, wherein the fourth syntax element indicates whether a residual block for the first color component exists; reconstructing the current block based on the residual block for the first color component and the residual block for the second color component in response to determining that the fourth syntax element indicates the existence of a residual block based on the first color component; and reconstructing the current block based on the residual block for the second color component in response to determining that the fourth syntax element indicates that a residual block based on the first color component does not exist.

[0179] In one example, method 800 further includes determining a third syntax element in response to determining that the second syntax element indicates the use of the joint residual block to generate a residual block for the first color component and a residual block for the second color component, and in response to determining that the template for the current block is unavailable, wherein the third syntax element indicates whether a residual block for the second color component exists.

[0180] In one example, method 800 further includes reconstructing the current block based on the residual block of the first color component in response to determining that the third syntax element indicates that the residual block of the second color component does not exist.

[0181] In one example, method 800 further includes determining a fourth syntax element in response to determining that the third syntax element indicates the existence of a residual block for the second color component, wherein the fourth syntax element indicates whether a residual block for the first color component exists; reconstructing the current block based on the residual block for the first color component and the residual block for the second color component in response to determining that the fourth syntax element indicates the existence of a residual block based on the first color component; and reconstructing the current block based on the residual block for the second color component in response to determining that the fourth syntax element indicates that a residual block based on the first color component does not exist.

[0182] In one example, the template for the current block is determined to be unavailable based on at least one of the following: the current block is at the boundary of an image or CTU; the sample prediction mode of the current block is not one of Cross-Component Linear Model (CCLM), Cross-Component Convolutional Model (CCCM), Inter-Frame CCCM, Intra-Frame Block Copy (IBC), or Inter-Frame Mode; and the size of the current block is within a predefined range.

[0183] FIG. 9 A computing environment 910 coupled to a user interface 950 is shown. The computing environment 910 may be part of a data processing server. The computing environment 910 includes a processor 920, a memory 930, and an input / output (I / O) interface 940.

[0184] Processor 920 typically controls the overall operation of computing environment 910, such as operations associated with display, data acquisition, data communication, and image processing. Processor 920 may include one or more processors for executing instructions to perform all or some of the steps in the methods described above. Furthermore, processor 920 may include one or more modules that facilitate interaction between processor 920 and other components. The processor may be a central processing unit (CPU), microprocessor, microcontroller, graphics processing unit (GPU), etc.

[0185] Memory 930 is configured to store various types of data to support the operation of computing environment 910. Memory 930 may include predefined software 932. Examples of such data include instructions for any application or method operating on computing environment 910, video datasets, image data, etc. Memory 930 can be implemented using any type of volatile or non-volatile memory device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0186] I / O interface 940 provides an interface between processor 920 and peripheral interface modules (such as keyboard, click wheel, buttons, etc.). I / O interface 940 can be coupled with encoders and decoders.

[0187] In embodiments, a non-transitory computer-readable storage medium or computer program product is also provided, including, for example, a plurality of programs in memory 930 and / or a storage bitstream, said plurality of programs which can be executed by processor 920 to perform the above-described encoding or decoding methods, said bitstream being generated by the above-described encoding method and / or to be decoded by the above-described decoding method. For example, the computer program product may include a non-transitory computer-readable storage medium. In one example, said plurality of programs can be executed by processor 920 to (e.g., from...) FIG. 2 The video encoder 20 in the process receives a bitstream or data stream including encoded video information (e.g., video blocks representing encoded video frames, and / or one or more associated syntax elements, etc.), and can also be executed by the processor 920 to perform the above-described decoding method based on the received bitstream or data stream. In another example, the plurality of programs can be executed by the processor 920 to perform the above-described encoding method to encode video information (e.g., video blocks representing video frames, and / or one or more associated syntax elements, etc.) into a bitstream or data stream, and can also be executed by the processor 920 to (e.g., to...) FIG. 3 The video decoder 30) in the middle transmits or stores the bit stream or data stream. Alternatively, the bit stream or data stream may be stored in a non-transitory computer-readable storage medium or a computer program product.

[0188] In an embodiment, a bitstream (e.g., including encoded video information) generated by the above encoding method and / or decoded by the above decoding method is provided.

[0189] In an embodiment, a computing device is also provided, comprising: one or more processors (e.g., processor 920); and a non-transitory computer-readable storage medium or memory 930 therein storing a plurality of programs executable by the one or more processors, wherein the one or more processors are configured to perform the methods described above when executing the plurality of programs. In one example, the one or more processors are configured to perform the encoding method described above to generate a bitstream when executing the plurality of programs, and the computing device may further include a transmitter configured to transmit the bitstream. In an alternative example, the one or more processors are configured to perform the encoding method described above to generate a bitstream when executing the plurality of programs, and to transmit or store the bitstream. In one example, the bitstream will be decoded using the decoding method described above.

[0190] In an embodiment, the computing environment 910 may be implemented by one or more ASICs, DSPs, digital signal processing devices (DSPDs), programmable logic devices (PLDs), FPGAs, GPUs, controllers, microcontrollers, microprocessors, or other electronic components for performing the methods described above.

[0191] In one embodiment, a method for storing a bitstream is also provided, comprising: storing the bitstream on a non-transitory computer-readable storage medium, wherein the bitstream is generated by the above-described encoding method and / or decoded by the above-described decoding method. In another embodiment, a method for storing a bitstream or a method for encoding video data is also provided, comprising: performing the above-described encoding method to generate a bitstream, and storing the bitstream on a non-transitory computer-readable storage medium. In one example, the bitstream will be decoded by the above-described decoding method.

[0192] In one embodiment, a method for transmitting a bitstream is also provided, the bitstream being generated by the above-described encoding method and / or decoded by the above-described decoding method. In another embodiment, a method for transmitting a bitstream or for encoding video data is also provided, comprising: performing the above-described encoding method to generate a bitstream, and transmitting the bitstream to a decoder. In one example, the bitstream will be decoded by the above-described decoding method. In another embodiment, a method for receiving a bitstream is also provided, the bitstream being generated by the above-described encoding method and / or decoded by the above-described decoding method.

[0193] The description in this disclosure has been presented for illustrative purposes and is not intended to be exhaustive or limited to this disclosure. Many modifications, variations, and alternative embodiments will be apparent to those skilled in the art from the teachings presented in the foregoing description and the associated drawings.

[0194] Unless otherwise specifically stated, the order of steps in the method according to this disclosure is intended to be illustrative only, and the steps of the method according to this disclosure are not limited to the specific order described above, but may be changed according to actual circumstances. Furthermore, at least one step in the method according to this disclosure may be adjusted, combined, or omitted as needed.

[0195] The examples chosen and described are intended to explain the principles of this disclosure and to enable others skilled in the art to understand the various embodiments of this disclosure, and preferably to utilize the basic principles and various embodiments with various modifications suitable for the intended particular purpose. Therefore, it will be understood that the scope of this disclosure is not limited to the specific examples of the disclosed embodiments, and that modifications and other embodiments are intended to be included within the scope of this disclosure.

Claims

1. A method for video decoding, comprising: Obtain a joint residual block, which is used to generate a residual block for the first color component and a residual block for the second color component associated with the current block to be reconstructed; A weight candidate list comprising multiple weight candidates is determined, and for each of the multiple weight candidates, a template cost of the template of the current block is calculated, wherein the template cost indicates the degree of difference between the following two: the product of the residual value of the second color component of each sample point in the template and the weight candidate, and the residual value of the first color component of each sample point in the template; Based on the calculated template cost, candidate weights are selected for the residual blocks used to generate the first color component; and The residual block of the second color component is generated based on the obtained joint residual block, and the residual block of the first color component is generated based on the joint residual block and the selected weight candidate.

2. The method according to claim 1, wherein, The template includes sample points in the M rows above the current block, where M is a positive integer; or The template includes sample points from the N columns to the left of the current block, where N is a positive integer; or The template includes sample points in the M rows above the current block and the N columns to the left, where M and N are integers greater than zero.

3. The method according to claim 1, wherein, The step of calculating the template cost of the current block for each of the plurality of weight candidates includes: The template cost of the current block's template is calculated using one of the following: the sum of absolute differences (SAD), the sum of squared differences (SSE), and the sum of absolute transformation differences (SATD), based on the difference between: the product of the residual value of the second color component of each sample point in the template and the weight candidate, and the residual value of the first color component of the sample point.

4. The method according to claim 1, wherein, The first color component is the Cr component, and the second color component is the Cb component.

5. The method according to claim 1, wherein, The step of calculating the template cost of the current block for each of the plurality of weight candidates includes: In response to determining that the residual value of the second color component of each sample point in the template and / or the residual value of the first color component of each sample point in the template does not exist, a predicted value of the second color component of each sample point in the template and a predicted value of the first color component of each sample point in the template are generated based on the sample point prediction mode of the current block. Based on the reconstructed value of the second color component of each sample point in the template and the predicted value of the second color component of each sample point in the generated template, the residual value of the second color component of each sample point in the template is determined. Based on the reconstructed value of the first color component of each sample point in the template and the predicted value of the first color component of each sample point in the generated template, the residual value of the first color component of each sample point in the template is determined; and The template cost is calculated based on the residual values ​​of the determined second color component and the determined first color component.

6. The method according to claim 5, wherein, The sample prediction modes include: Cross-component linear model (CCLM), Cross-component convolutional model (CCCM), Inter-frame CCCM, Intra-frame block copy (IBC), Inter-frame mode, Planar intra-frame mode, DC intra-frame mode, and Directional intra-frame mode.

7. A method for video decoding, comprising: Determine a first syntax element, which indicates whether the residual blocks of the first color component and the residual blocks of the second color component associated with the current block to be reconstructed do not exist; as well as In response to determining that the first syntax element indicates that the residual blocks of the first color component and the second color component are not both absent, a second syntax element is determined, the second syntax element indicating whether to use a joint residual block to generate the residual blocks of the first color component and the second color component.

8. The method according to claim 7, further comprising: In response to determining that the second syntax element indicates the use of the joint residual block to generate the residual block of the first color component and the residual block of the second color component, and in response to determining that the template of the current block is available, the method according to any one of claims 1-6 is performed to generate the residual block of the first color component and the residual block of the second color component.

9. The method according to claim 7, further comprising: In response to determining that neither the residual block for the first color component nor the residual block for the second color component, indicated by the first syntax element, exists, the current block is reconstructed based on the prediction block for the first color component and the prediction block for the second color component.

10. The method of claim 7, further comprising: In response to determining that the second syntax element indicates that the joint residual block is not used to generate the residual block of the first color component and the residual block of the second color component, a third syntax element is determined, wherein the third syntax element indicates whether the residual block of the second color component exists.

11. The method of claim 10, further comprising: In response to determining that the third syntax element indicates that the residual block of the second color component does not exist, the current block is reconstructed based on the residual block of the first color component.

12. The method of claim 11, further comprising: In response to determining that the third syntax element indicates the existence of a residual block for the second color component, a fourth syntax element is determined, wherein the fourth syntax element indicates whether a residual block for the first color component exists. In response to determining that the fourth syntax element indicates the existence of a residual block based on the first color component, the current block is reconstructed based on the residual blocks of the first color component and the residual blocks of the second color component; and In response to determining that the fourth syntax element indicates that the residual block based on the first color component does not exist, the current block is reconstructed based on the residual block based on the second color component.

13. The method of claim 7, further comprising: In response to determining that the second syntax element indicates the use of the joint residual block to generate the residual block of the first color component and the residual block of the second color component, and in response to determining that the template of the current block is unavailable, a third syntax element is determined, wherein the third syntax element indicates whether the residual block of the second color component exists.

14. The method of claim 13, further comprising: In response to determining that the third syntax element indicates that the residual block of the second color component does not exist, the current block is reconstructed based on the residual block of the first color component.

15. The method of claim 14, further comprising: In response to determining that the third syntax element indicates the existence of a residual block for the second color component, a fourth syntax element is determined, wherein the fourth syntax element indicates whether a residual block for the first color component exists. In response to determining that the fourth syntax element indicates the existence of a residual block based on the first color component, the current block is reconstructed based on the residual blocks of the first color component and the residual blocks of the second color component; and In response to determining that the fourth syntax element indicates that the residual block based on the first color component does not exist, the current block is reconstructed based on the residual block based on the second color component.

16. The method according to claim 13, wherein, The template for the current block is deemed unavailable based on at least one of the following: The current block is at the boundary of the image or CTU; The sample prediction mode of the current block is not one of the following: Cross-component linear model (CCLM), Cross-component convolutional model (CCCM), Inter-frame CCCM, Intra-frame block copying (IBC), or Inter-frame mode; and The size of the current block is within a predefined range.

17. A method for video encoding, comprising: Determine and transmit a joint residual block in the bitstream, the joint residual block being used to generate a residual block for the first color component and a residual block for the second color component associated with the current block to be reconstructed; A weight candidate list comprising multiple weight candidates is determined, and for each of the multiple weight candidates, a template cost of the template of the current block is calculated, wherein the template cost indicates the degree of difference between the following two: the product of the residual value of the second color component of each sample point in the template and the weight candidate, and the residual value of the first color component of each sample point in the template; Based on the calculated template cost, candidate weights are selected for the residual blocks used to generate the first color component; and The residual block of the second color component is generated based on the obtained joint residual block, and the residual block of the first color component is generated based on the joint residual block and the selected weight candidate.

18. The method according to claim 17, wherein, The template includes sample points in the M rows above the current block, where M is a positive integer; or The template includes sample points from the N columns to the left of the current block, where N is a positive integer; or The template includes sample points in the M rows above the current block and the N columns to the left, where M and N are integers greater than zero.

19. The method according to claim 17, wherein, The step of calculating the template cost of the current block for each of the plurality of weight candidates includes: The template cost of the current block's template is calculated using one of the following: the sum of absolute differences (SAD), the sum of squared differences (SSE), and the sum of absolute transformation differences (SATD), based on the difference between: the product of the residual value of the second color component of each sample point in the template and the weight candidate, and the residual value of the first color component of the sample point.

20. The method of claim 17, wherein, The first color component is the Cr component, and the second color component is the Cb component.

21. The method according to claim 17, wherein, The step of calculating the template cost of the current block for each of the plurality of weight candidates includes: In response to determining that the residual value of the second color component of each sample point in the template and / or the residual value of the first color component of each sample point in the template does not exist, a predicted value of the second color component of each sample point in the template and a predicted value of the first color component of each sample point in the template are generated based on the sample point prediction mode of the current block. Based on the reconstructed value of the second color component of each sample point in the template and the predicted value of the second color component of each sample point in the generated template, the residual value of the second color component of each sample point in the template is determined. Based on the reconstructed value of the first color component of each sample point in the template and the predicted value of the first color component of each sample point in the generated template, the residual value of the first color component of each sample point in the template is determined; and The template cost is calculated based on the residual values ​​of the determined second color component and the determined first color component.

22. The method according to claim 21, wherein, The sample prediction modes include: Cross-component linear model (CCLM), Cross-component convolutional model (CCCM), Inter-frame CCCM, Intra-frame block copy (IBC), Inter-frame mode, Planar intra-frame mode, DC intra-frame mode, and Directional intra-frame mode.

23. A method for video encoding, comprising: Determine a first syntax element, which indicates whether the residual blocks of the first color component and the residual blocks of the second color component associated with the current block to be reconstructed do not exist; as well as In response to determining that the first syntax element indicates that the residual blocks of the first color component and the second color component are not both absent, a second syntax element is determined, the second syntax element indicating whether to use a joint residual block to generate the residual blocks of the first color component and the second color component.

24. The method of claim 23, further comprising: In response to determining that the second syntax element indicates the use of the joint residual block to generate the residual block of the first color component and the residual block of the second color component, and in response to determining that the template of the current block is available, the method according to any one of claims 17-22 is performed to generate the residual block of the first color component and the residual block of the second color component.

25. The method of claim 23, further comprising: In response to determining that neither the residual block for the first color component nor the residual block for the second color component, indicated by the first syntax element, exists, the current block is reconstructed based on the prediction block for the first color component and the prediction block for the second color component.

26. The method of claim 23, further comprising: In response to determining that the second syntax element indicates that the joint residual block is not used to generate the residual block of the first color component and the residual block of the second color component, a third syntax element is determined, wherein the third syntax element indicates whether the residual block of the second color component exists.

27. The method of claim 26, further comprising: In response to determining that the third syntax element indicates that the residual block of the second color component does not exist, the current block is reconstructed based on the residual block of the first color component.

28. The method of claim 27, further comprising: In response to determining that the third syntax element indicates the existence of a residual block for the second color component, a fourth syntax element is determined, wherein the fourth syntax element indicates whether a residual block for the first color component exists. In response to determining that the fourth syntax element indicates the existence of a residual block based on the first color component, the current block is reconstructed based on the residual blocks of the first color component and the residual blocks of the second color component; and In response to determining that the fourth syntax element indicates that the residual block based on the first color component does not exist, the current block is reconstructed based on the residual block based on the second color component.

29. The method of claim 23, further comprising: In response to determining that the second syntax element indicates the use of the joint residual block to generate the residual block of the first color component and the residual block of the second color component, and in response to determining that the template of the current block is unavailable, a third syntax element is determined, wherein the third syntax element indicates whether the residual block of the second color component exists.

30. The method of claim 29, further comprising: In response to determining that the third syntax element indicates that the residual block of the second color component does not exist, the current block is reconstructed based on the residual block of the first color component.

31. The method of claim 30, further comprising: In response to determining that the third syntax element indicates the existence of a residual block for the second color component, a fourth syntax element is determined, wherein the fourth syntax element indicates whether a residual block for the first color component exists. In response to determining that the fourth syntax element indicates the existence of a residual block based on the first color component, the current block is reconstructed based on the residual blocks of the first color component and the residual blocks of the second color component; and In response to determining that the fourth syntax element indicates that the residual block based on the first color component does not exist, the current block is reconstructed based on the residual block based on the second color component.

32. The method according to claim 29, wherein, The template for the current block is deemed unavailable based on at least one of the following: The current block is at the boundary of the image or CTU; The sample prediction mode of the current block is not one of the following: Cross-component linear model (CCLM), Cross-component convolutional model (CCCM), Inter-frame CCCM, Intra-frame block copying (IBC), or Inter-frame mode; and The size of the current block is within a predefined range.

33. A computer-readable storage medium storing a bit stream formed by instructions, which, when executed by a computing device having one or more processors, cause the one or more processors to perform the following steps in an encoding method: Determine and transmit a joint residual block in the bitstream, the joint residual block being used to generate a residual block for the first color component and a residual block for the second color component associated with the current block to be reconstructed; A weight candidate list comprising multiple weight candidates is determined, and for each of the multiple weight candidates, a template cost of the template of the current block is calculated, wherein the template cost indicates the degree of difference between the following two: the product of the residual value of the second color component of each sample point in the template and the weight candidate, and the residual value of the first color component of each sample point in the template; Based on the calculated template cost, candidate weights are selected for the residual blocks used to generate the first color component; and The residual block for the second color component is generated based on the obtained joint residual block, and the residual block for the first color component is generated based on the joint residual block and the selected weight candidates. The bitstream will be decoded by the decoding method according to claim 1.

34. A computer-readable storage medium storing a bit stream of instructions, which, when executed by a computing device having one or more processors, cause the one or more processors to perform the following steps in an encoding method: Determine a first syntax element, which indicates whether neither the residual block of the first color component nor the residual block of the second color component associated with the current block to be reconstructed exists; and In response to determining that the first syntax element indicates that the residual blocks for the first color component and the second color component are not both absent, a second syntax element is determined, indicating whether a joint residual block is used to generate the residual blocks for the first color component and the second color component. in, The bitstream will be decoded by the decoding method according to claim 7.

35. A computer-readable storage medium storing a bit stream formed by instructions, which, when executed by a computing device having one or more processors, cause the one or more processors to perform the encoding method according to claim 17 or 23.

36. A method for storing a bit stream, comprising: Perform the encoding method according to claim 17 or 23 to generate a bitstream; as well as Store the bit stream.

37. A method for transmitting a bit stream, comprising: Perform the encoding method according to claim 17 or 23 to generate a bitstream; as well as Send the bit stream.

38. An electronic device comprising: Non-transitory computer-readable storage medium; as well as A processor configured to perform the following steps in the encoding method to generate a bitstream and store and / or transmit the bitstream: Determine and transmit a joint residual block in the bitstream, the joint residual block being used to generate a residual block for the first color component and a residual block for the second color component associated with the current block to be reconstructed; A weight candidate list comprising multiple weight candidates is determined, and for each of the multiple weight candidates, a template cost of the template of the current block is calculated, wherein the template cost indicates the degree of difference between the following two: the product of the residual value of the second color component of each sample point in the template and the weight candidate, and the residual value of the first color component of each sample point in the template; Based on the calculated template cost, candidate weights are selected for the residual blocks used to generate the first color component; and The residual block for the second color component is generated based on the obtained joint residual block, and the residual block for the first color component is generated based on the joint residual block and the selected weight candidates. The bitstream will be decoded by the decoding method according to claim 1.

39. An electronic device comprising: Non-transitory computer-readable storage medium; as well as A processor configured to perform the following steps in the encoding method to generate a bitstream and store and / or transmit the bitstream: Determine a first syntax element, which indicates whether the residual blocks of the first color component and the residual blocks of the second color component associated with the current block to be reconstructed do not exist; as well as In response to determining that the first syntax element indicates that the residual blocks for the first color component and the second color component are not both absent, a second syntax element is determined, indicating whether a joint residual block is used to generate the residual blocks for the first color component and the second color component. The bitstream will be decoded by the decoding method according to claim 7.

40. An electronic device comprising: Non-transitory computer-readable storage medium; as well as A processor configured to perform the encoding method according to claim 17 or 23 to generate a bit stream, and to store and / or transmit the bit stream.

Images