Chroma motion compensation method and device based on adaptive cross-component filtering

Abstract

Methods, apparatus, and non-transitory computer-readable storage media for video decoding and encoding are provided. In a method for video decoding, a decoder can obtain a plurality of reconstructed luma samples of a current inter-coded block, obtain an adaptive cross-component filter, and obtain filtered motion compensated chroma samples based on the adaptive cross-component filter and the plurality of reconstructed luma samples.

Description

[0001] Cross-references to related applications

[0002] This application is based on and claims priority to U.S. Provisional Application No. 63 / 466,637, filed May 15, 2023, entitled “Method and Apparatus for Chromaticity Motion Compensation Based on Adaptive Cross-Component Filtering,” the entire contents of which are incorporated herein by reference. Technical Field

[0003] This disclosure relates to video coding and compression, and more particularly, but not limited to, methods and apparatus for improving the coding efficiency of inter-frame blocks by applying cross-component filtering to generate prediction samples of the chroma components of the block. Background Technology

[0004] Various video codec technologies can be used to compress video data. Video codecs are performed according to one or more video codec standards. For example, video codec standards include Universal Video Codec (VVC), High Efficiency Video Codec (H.265 / HEVC), High-Advanced Video Codec (H.264 / AVC), and Moving Picture Experts Group (MPEG) codecs. Video codecs typically employ prediction methods that utilize redundancy present in video images or sequences (e.g., inter-frame prediction, intra-frame prediction, etc.). A key goal of video codec technology is to compress video data to a lower bitrate while avoiding or minimizing video quality degradation.

[0005] The first version of the VVC standard was completed in July 2020, offering approximately 50% bitrate savings or equivalent perceived quality compared to its predecessor, HEVC. While the VVC standard provides significant codec improvements over its predecessor, there is evidence that even higher codec efficiency can be achieved using additional codec tools. Recently, the Joint Video Development Group (JVET), in collaboration with ITU-TVECG and ISO / IEC MPEG, began developing advanced technologies to significantly improve codec efficiency on VVC. In April 2021, a software codebase called the Enhanced Compression Model (ECM) was established for future video codec exploration. The ECM reference software is based on the VVC Test Model (VTM) developed by JVET for VVC and further extends and / or improves several existing modules (e.g., intra / inter-frame prediction, transform, loop filters, etc.). In the future, any new codec tools beyond the VVC standard will need to be integrated into the ECM platform and tested using the JVET Common Test Conditions (CTC). Summary of the Invention

[0006] This invention provides examples of techniques related to improving the encoding and decoding efficiency of inter-frame blocks.

[0007] According to a first aspect of this disclosure, a method for video decoding of an inter-frame coded block is provided. In this method, the decoder is capable of acquiring multiple reconstructed luminance samples of the current inter-frame coded block, acquiring an adaptive cross-component filter, and acquiring filtered motion-compensated chrominance samples based on the adaptive cross-component filter and the multiple reconstructed luminance samples.

[0008] According to a second aspect of this disclosure, a method for video coding of inter-frame coding blocks is provided. In this method, the encoder is able to acquire multiple reconstructed luminance samples of the current inter-frame coding block, acquire an adaptive cross-component filter, and acquire filtered motion-compensated chrominance samples based on the adaptive cross-component filter and the multiple reconstructed luminance samples.

[0009] According to a third aspect of this disclosure, a video decoding apparatus is provided. The apparatus is capable of including one or more processors and a memory coupled to the one or more processors and configured to store instructions executable by the one or more processors. Furthermore, the one or more processors, when executing the instructions, are configured to perform the method according to the first aspect.

[0010] According to a fourth aspect of this disclosure, a video encoding apparatus is provided. The apparatus is capable of including one or more processors and a memory coupled to the one or more processors and configured to store instructions executable by the one or more processors. Furthermore, the one or more processors, when executing the instructions, are configured to perform the method according to the second aspect.

[0011] According to a fifth aspect of this disclosure, a non-transitory computer-readable storage medium is provided for storing computer-executable instructions that, when executed by one or more computer processors, cause the one or more computer processors to receive a bit stream and perform the method according to the first aspect based on the bit stream.

[0012] According to a sixth aspect of this disclosure, a non-transitory computer-readable storage medium is provided for storing computer-executable instructions that, when executed by one or more computer processors, cause the one or more computer processors to perform the method according to the second aspect to encode a current block into a bit stream and transmit the bit stream. Attached Figure Description

[0013] Examples of this disclosure will be described in more detail with reference to the specific examples shown in the accompanying drawings. Given that these drawings illustrate only a few examples, they are not intended to be limiting in scope; rather, these examples are described and explained with additional features and details using the drawings.

[0014] Figure 1AThis is a block diagram illustrating a system for encoding and decoding video blocks according to some examples of this disclosure.

[0015] Figure 1B This is a block diagram of an encoder based on some examples of this disclosure.

[0016] Figures 1C to 1F This is a block diagram illustrating, according to some examples of this disclosure, how a frame can be recursively divided into multiple video blocks of different sizes and shapes.

[0017] Figure 1G This is a block diagram illustrating an exemplary video encoder according to some examples of this disclosure.

[0018] Figure 2A This is a block diagram of a decoder based on some examples of this disclosure.

[0019] Figure 2B This is a block diagram illustrating an exemplary video decoder according to some examples of this disclosure.

[0020] Figure 3A This is a diagram illustrating block partitioning in a multi-type tree structure according to some examples of this disclosure.

[0021] Figure 3B This is a diagram illustrating block partitioning in a multi-type tree structure according to some examples of this disclosure.

[0022] Figure 3C This is a diagram illustrating block partitioning in a multi-type tree structure according to some examples of this disclosure.

[0023] Figure 3D This is a diagram illustrating block partitioning in a multi-type tree structure according to some examples of this disclosure.

[0024] Figure 3E This is a diagram illustrating block partitioning in a multi-type tree structure according to some examples of this disclosure.

[0025] Figure 4 One example is shown in some examples according to this disclosure, where dx and dy are the horizontal and vertical values ​​of MV.

[0026] Figure 5 Examples are shown in some examples according to this disclosure, where one MV has a score value and an interpolation filter is applied to produce a corresponding predicted sample at the score sample location.

[0027] Figure 6 shows examples of two diamond filter shapes according to some examples of this disclosure.

[0028] Figure 7Subsampled 1-D Laplace calculations for gradient computation in all directions are shown as examples of some of the methods described in this disclosure.

[0029] Figure 8 The following are examples of filtering operations in CC-ALF implemented by applying a diamond filter to the luminance channel, according to some examples of this disclosure.

[0030] Figure 9 This is a block diagram illustrating a video encoder when the CC-MCP according to this disclosure is applied.

[0031] Figure 10 Is receiving by Figure 9 A block diagram of the decoder of the encoder in this disclosure.

[0032] Figure 11 This is a diagram illustrating the derivation of the CC-MCP filter coefficients on the decoder side according to some examples of this disclosure.

[0033] Figure 12 This is a block diagram of a CC-MCP based on some examples of the present disclosure, which uses high-frequency information from reconstructed luminance samples as guidance to improve the quality of motion-compensated chrominance samples.

[0034] Figure 13 This is a diagram illustrating a computing environment coupled with a user interface, according to some examples of this disclosure.

[0035] Figure 14 This is a flowchart illustrating some examples of video decoding methods according to this disclosure.

[0036] Figure 15 This illustrates some examples corresponding to, for example, this disclosure. Figure 14 The flowchart of the video encoding method shown in the video decoding method is as follows. Detailed Implementation

[0037] Referring now to the detailed description, examples of which are illustrated in the accompanying drawings. Numerous non-limiting details are set forth in the following detailed description to aid in understanding the subject matter presented herein. Those skilled in the art will recognize that various alternatives can be used. For example, those skilled in the art will recognize that the subject matter presented herein can be implemented on many types of electronic devices with digital video capabilities.

[0038] The terminology used in this disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of this disclosure. The singular forms “a,” “the,” and “the” in the disclosure and appended claims are also intended to include the plural forms unless otherwise expressly indicated throughout the disclosure. It should also be understood that the term “and / or” as used in this disclosure refers to and includes one or any or all possible combinations of the listed related items.

[0039] Throughout this specification, references to "an embodiment," "an embodiment," "an example," "some embodiments," "some examples," or similar language mean that a particular feature, structure, or characteristic described is included in at least one embodiment or example. Features, structures, elements, or characteristics described in connection with one or more embodiments also apply to other embodiments, unless otherwise expressly stated.

[0040] Throughout this disclosure, the terms “first,” “second,” “third,” etc., are used only as terms for related elements, such as equipment, components, ingredients, steps, etc., and do not imply any spatial or temporal order unless otherwise expressly stated. For example, “first equipment” and “second equipment” can refer to two separately formed devices, or two parts, components, or operating states of the same device, and can be arbitrarily named.

[0041] The terms "module," "submodule," "circuit," "subcircuit," "circuit system," "subcircuit system," "unit," or "subunit" can include memory (shared, dedicated, or grouped) that stores code or instructions executable by one or more processors. A module can include one or more circuits, with or without stored code or instructions. A module or circuit can include one or more components that are directly or indirectly connected. These components may or may not be physically connected to each other or adjacent to each other.

[0042] As used herein, the terms "if" or "when," depending on the context, can be understood to mean "according to" or "in response to." These terms, if appearing in a claim, may not imply that the associated limitation or feature is conditional or optional. For example, a method can include the steps of: i) performing a function or action X' when or if condition X exists, and ii) performing a function or action Y' when or if condition Y exists. The method can be implemented using the ability to perform function or action X' and the ability to perform function or action Y'. Therefore, functions X' and Y' can be performed at different times in multiple executions of the method.

[0043] Units or modules can be implemented entirely by software, entirely by hardware, or a combination of hardware and software. For example, in a pure software implementation, a unit or module can include functionally related code blocks or software components that are directly or indirectly linked together to perform a specific function.

[0044] Figure 1A This is a block diagram illustrating an exemplary system 10 for encoding and decoding video blocks in parallel, according to some embodiments of the present disclosure. Figure 1A As shown, system 10 includes a source device 12 that generates and encodes video data that will later be decoded by a target device 14. The source device 12 and target device 14 can include any electronic device from a wide variety of electronic devices, including cloud servers, server computers, desktop or laptop computers, tablet computers, smartphones, set-top boxes, digital televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, etc. In some embodiments, the source device 12 and target device 14 are equipped with wireless communication capabilities.

[0045] In some implementations, target device 14 may receive encoded video data to be decoded via link 16. Link 16 may include any type of communication medium or device capable of moving encoded video data from source device 12 to target device 14. In one example, link 16 may include a communication medium enabling source device 12 to transmit encoded video data directly to target device 14 in real time. The encoded video data may be modulated according to a communication standard (e.g., a wireless communication protocol) and transmitted to target device 14. The communication medium may include any wireless or wired communication medium, such as radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium may form part of a packet-based network (e.g., a local area network, a wide area network, or a global network such as the Internet). The communication medium may include a router, switch, base station, or any other device that facilitates communication from source device 12 to target device 14.

[0046] In some other implementations, encoded video data can be sent from output interface 22 to storage device 32. The target device 14 can then access the encoded video data in storage device 32 via input interface 28. Storage device 32 can include any data storage medium of various distributed or locally accessed data storage media, such as hard disk drives, Blu-ray discs, digital universal discs (DVDs), compact disc read-only memory (CD-ROMs), flash memory, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded video data. In another example, storage device 32 can correspond to a file server or another intermediate storage device that can hold the encoded video data generated by source device 12. Target device 14 can access the stored video data from storage device 32 via streaming or downloading. The file server can be any type of computer capable of storing and sending encoded video data to target device 14. Exemplary file servers include web servers (e.g., for websites), file transfer protocol (FTP) servers, network attached storage (NAS) devices, or local disk drives. Target device 14 can access the encoded video data via any standard data connection suitable for accessing encoded video data stored on the file server. Standard data connections include wireless channels (e.g., Wi-Fi connections), wired connections (e.g., digital subscriber line (DSL), cable modems, etc.), or a combination of both. Transfer of encoded video data from storage device 32 can be streaming, downloading, or a combination of both.

[0047] like Figure 1A As shown, source device 12 includes a video source 18, a video encoder 20, and an output interface 22. Video source 18 may include sources or combinations of such sources, such as: a video capture device (e.g., a camera), a video archive containing previously captured video, a video feed interface for receiving video from a video content provider, and / or a computer graphics system for generating computer graphics data as source video. As an example, if video source 18 is a camera in a security monitoring system, source device 12 and target device 14 may form a camera phone or video phone. However, the embodiments described in this application are generally applicable to video encoding and decoding and can be applied to wireless and / or wired applications.

[0048] The captured, pre-captured, or computer-generated video can be encoded by the video encoder 20. The encoded video data can be sent directly to the target device 14 via the output interface 22 of the source device 12. Alternatively, the encoded video data can be stored on the storage device 32 for later access by the target device 14 or other devices for decoding and / or playback. The output interface 22 may further include a modem and / or transmitter.

[0049] Target device 14 includes an input interface 28, a video decoder 30, and a display device 34. Input interface 28 may include a receiver and / or a modem, and receives encoded video data via link 16. The encoded video data transmitted via link 16 or provided on storage device 32 may include various syntax elements generated by video encoder 20 for use by video decoder 30 when decoding the video data. Such syntax elements may be included within encoded video data transmitted on a communication medium, stored on a storage medium, or stored on a file server.

[0050] In some embodiments, the target device 14 may include a display device 34, which may be an integrated display device or an external display device configured to communicate with the target device 14. The display device 34 displays decoded video data to a user and may include any of a variety of display devices, such as a liquid crystal display (LCD), a plasma display, an organic light-emitting diode (OLED) display, or another type of display device.

[0051] Video encoder 20 and video decoder 30 can operate according to proprietary or industry standards (e.g., VVC, HEVC, MPEG-4 Part 10, AVC) or extensions of such standards. It should be understood that this application is not limited to a specific video encoding / decoding standard and can be applied to other video encoding / decoding standards. It is generally understood that the video encoder 20 of source device 12 can be configured to encode video data according to any of these current or future standards. Similarly, it is also generally understood that the video decoder 30 of target device 14 can be configured to decode video data according to any of these current or future standards.

[0052] The video encoder 20 and video decoder 30 can be implemented as any circuit of a variety of suitable encoder and / or decoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic devices, software, hardware, firmware, or any combination thereof. When partially implemented in software, the electronic device may store instructions for software in a suitable non-transitory computer-readable medium and use one or more processors to execute the instructions in hardware to perform the video encoding / decoding operations disclosed in this disclosure. Each of the video encoder 20 and video decoder 30 may be included in one or more encoders or decoders, and either encoder or decoder may be integrated as part of a combined encoder / decoder (CODEC) in the respective device.

[0053] In some implementations, components of source device 12 (e.g., video source 18, video encoder 20, or the following references) Figure 1G The components included in the video encoder 20, and at least a portion of the components in the output interface 22, and / or the components of the target device 14 (e.g., the input interface 28, the video decoder 30, or the following references) Figure 2BAt least a portion of the components included in the video decoder 30 and the display device 34 can operate in a cloud computing service network such as Software as a Service (SaaS), Platform as a Service (PaaS), or Infrastructure as a Service (IaaS), where the cloud computing service network can provide software, platform, and / or infrastructure. In some embodiments, one or more components in the source device 12 and / or target device 14 that are not included in the cloud computing service network can be located in one or more client devices, and these client devices can communicate with the server computer in the cloud computing service network via a wireless communication network (e.g., a cellular communication network, a short-range wireless communication network, or a Global Navigation Satellite System (GNSS) communication network) or a wired communication network (e.g., a local area network (LAN) communication network or a power line communication (PLC) network). In one embodiment, at least a portion of the operations described herein can be implemented as a cloud-based service provided by one or more server computers, wherein the one or more server computers are implemented by at least a portion of the components of the source device 12 and / or at least a portion of the components of the target device 14 in the cloud computing service network; and one or more other operations described herein can be implemented by one or more client devices. In some embodiments, the cloud computing service network can be a private cloud, a public cloud, or a hybrid cloud. Terms such as “cloud,” “cloud computing,” and “cloud-based” used herein are used interchangeably without departing from the scope of this disclosure. It should be understood that this disclosure is not limited to implementation in the aforementioned cloud computing service network. Instead, this disclosure can also be implemented in any other type of computing environment currently known or developed in the future.

[0054] Like HEVC, VVC is also built on a block-based hybrid video coding framework. Figure 1B This is a block diagram illustrating a block-based video encoder according to some embodiments of the present disclosure. In encoder 100, the input video signal is processed block by block, referred to as a coding unit (CU). Encoder 100 can be as follows: Figure 1AThe video encoder 20 is shown. In VTM 1.0, the maximum CU size is 128x128 pixels. However, unlike HEVC, which is based solely on quadtree-based block partitioning, in VVC, a coding tree unit (CTU) is partitioned into CUs to accommodate the local characteristics of variations based on quadtrees / binary / ternary trees. Furthermore, the concept of multi-partition unit types in HEVC is removed; that is, the separation of CUs, prediction units (PUs), and transform units (TUs) no longer exists in VVC. Instead, each CU is always used as the basic unit for prediction and transform without further partitioning. In the multi-type tree structure, a CTU is first partitioned by a quadtree structure. Then, each quadtree leaf node can be further partitioned using binary and ternary tree structures.

[0055] Figures 3A to 3E This is a schematic diagram illustrating a multi-type tree segmentation pattern according to some embodiments of the present disclosure. Figures 3A-3E Five segmentation types are shown, including quadruple segmentation ( Figure 3A Vertical binary segmentation ( Figure 3B ), horizontal binary segmentation ( Figure 3C Vertical ternary segmentation ( Figure 3D ) and horizontal ternary segmentation ( Figure 3E ).

[0056] For each given video block, spatial prediction and / or temporal prediction can be performed. Spatial prediction (or "intra-frame prediction") uses pixels from samples (called reference samples) of neighboring blocks already encoded in the same video image / strip to predict the current video block. Spatial prediction reduces the spatial redundancy inherent in the video signal. Temporal prediction (also known as "inter-frame prediction" or "motion-compensated prediction") uses reconstructed pixels from the encoded video image to predict the current video block. Temporal prediction reduces the temporal redundancy inherent in the video signal. The temporal prediction signal for a given CU is typically transmitted by signaling one or more motion vectors (MVs) indicating the amount and direction of motion between the current CU and its temporal reference. Additionally, if multiple reference images are supported, a reference image index is sent to identify which reference image in the reference image memory the temporal prediction signal originates from.

[0057] Following spatial and / or temporal prediction, the intra / inter-frame mode determination circuit 121 in encoder 100 selects the optimal prediction mode, for example, based on a rate-distortion optimization method. The block prediction value 120 is then subtracted from the current video block; and the resulting prediction residual is decorrelated using transform circuit 102 and quantization circuit 104. The resulting quantization residual coefficients are inversely quantized by inverse quantization circuit 116 and inversely transformed by inverse transform circuit 118 to form the reconstruction residual, which is then added back to the prediction block to form the reconstructed signal of the CU. Furthermore, before placing the reconstructed CU into the reference image memory of image buffer 117 and using it to encode future video blocks, loop filtering 115, such as a deblocking filter, sample adaptive offset (SAO), and / or adaptive loop filter (ALF), can be applied to the reconstructed CU. To form the output video bitstream 114, the encoding mode (inter-frame or intra-frame), prediction mode information, motion information, and quantization residual coefficients are sent to entropy coding unit 106 for further compression and packing to form the bitstream.

[0058] For example, deblocking filters are available in current versions of AVC, HEVC, and VVC. In HEVC, an additional loop filter called SAO is defined to further improve encoding and decoding efficiency. In the current version of the VVC standard, another loop filter called ALF is under active research and is very likely to be incorporated into the final standard.

[0059] These loop filter operations are optional. Performing these operations helps improve encoding / decoding efficiency and visual quality. They can also be turned off as a decision made by encoder 100 to save computational complexity.

[0060] It should be noted that intra-frame prediction is typically based on unfiltered reconstructed pixels, while inter-frame prediction is based on filtered reconstructed pixels if the encoder 100 has these filter options enabled.

[0061] Figure 2A This is a block diagram illustrating a block-based video decoder 200, which can be used in conjunction with many video coding standards. This decoder 200 is similar to... Figure 1B The reconstruction-related part in the encoder 100. The block-based video decoder 200 can be as follows: Figure 1AThe video decoder 30 is shown. In decoder 200, the input video bitstream 201 is first decoded via entropy decoding 202 to derive quantization coefficient levels and prediction-related information. Then, the quantized coefficient levels are processed via inverse quantization 204 and inverse transform 206 to obtain the reconstructed prediction residual. The block prediction mechanism implemented in the intra / inter-frame mode selector 212 is configured to perform intra-frame prediction 208 or motion compensation 210 based on the decoded prediction information. A set of unfiltered reconstructed pixels is obtained by adding the reconstructed prediction residual from inverse transform 206 and the prediction output generated by the block prediction mechanism using adder 214.

[0062] The reconstructed blocks can be further passed through loop filter 209 before being stored in image buffer 213, which serves as a reference image memory. The reconstructed video in image buffer 213 can be sent to drive a display device, as well as to predict future video blocks. With loop filter 209 enabled, filtering operations are performed on these reconstructed pixels to derive the final reconstructed video output 222.

[0063] Figure 1G This is a block diagram illustrating another exemplary video encoder 20 according to some embodiments described in this application. The video encoder 20 can perform intra-frame predictive coding and inter-frame predictive coding on video blocks within a video frame. Intra-frame predictive coding relies on spatial prediction to reduce or remove spatial redundancy in the video data within a given video frame or image. Inter-frame predictive coding relies on temporal prediction to reduce or remove temporal redundancy in the video data within neighboring video frames or images of a video sequence. It should be noted that in the field of video encoding and decoding, the term "frame" can be used as a synonym for the terms "image" or "picture".

[0064] like Figure 1GAs shown, the video encoder 20 includes a video data memory 40, a prediction processing unit 41, a decoded image buffer (DPB) 64, an adder 50, a transform processing unit 52, a quantization unit 54, and an entropy coding unit 56. The prediction processing unit 41 further includes a motion estimation unit 42, a motion compensation unit 44, a segmentation unit 45, an intra-frame prediction processing unit 46, and an intra-frame block copying (BC) unit 48. In some embodiments, the video encoder 20 also includes an inverse quantization unit 58, an inverse transform processing unit 60, and an adder 62 for video block reconstruction. A loop filter 63, such as a deblocking filter, can be located between the adder 62 and the DPB 64 to filter block boundaries to remove block artifacts from the reconstructed video. In addition to the deblocking filter, another loop filter, such as a sample adaptive offset (SAO) filter, a cross-component sample adaptive offset (CCSAO) filter, and / or an adaptive loop filter (ALF), can be used to filter the output of the adder 62. It should be noted that, regarding the CCSAO technology, this application is not limited to the embodiments described herein, but can also be applied to situations where an offset is selected for any other component among the luminance component, Cb chrominance component, and Cr chrominance component based on any one of the luminance component, Cb chrominance component, and Cr chrominance component to modify that other component based on the selected offset. Furthermore, it should be noted that the first component mentioned herein can be any one of the luminance component, Cb chrominance component, and Cr chrominance component; the second component mentioned herein can be any other one of the luminance component, Cb chrominance component, and Cr chrominance component; and the third component mentioned herein can be the remaining components among the luminance component, Cb chrominance component, and Cr chrominance component. In some examples, the loop filter can be omitted, and the decoded video block can be directly provided to the DPB 64 by the adder 62. The video encoder 20 can take the form of a fixed or programmable hardware unit, or can be distributed among one or more of the described fixed or programmable hardware units.

[0065] The video data storage device 40 can store video data encoded by the components of the video encoder 20. For example, it can store data from... Figure 1A The video source 18 shown acquires video data from the video data memory 40. The DPB 64 is a buffer that stores reference video data (e.g., reference frames or images) used by the video encoder 20 when encoding the video data (e.g., in intra-frame or inter-frame predictive coding modes). The video data memory 40 and DPB 64 can be formed from any of a variety of memory devices. In various examples, the video data memory 40 may be on-chip along with other components of the video encoder 20, or off-chip relative to those components.

[0066] like Figure 1GAs shown, after receiving video data, the segmentation unit 45 within the prediction processing unit 41 segments the video data into video blocks. This segmentation may also include segmenting the video frame into strips, tiles (e.g., a collection of video blocks) or other larger coding units (CUs) according to a predefined splitting structure associated with the video data (e.g., a quadtree (QT) structure). A video frame is, or can be considered, a two-dimensional array or matrix of samples with sample values. Samples in the array may also be referred to as pixels or image elements (pel). The number of samples in the horizontal and vertical directions (or axes) of the array or image defines the size and / or resolution of the video frame. For example, a video frame can be divided into multiple video blocks using QT segmentation. A video block is also, or can be considered, a two-dimensional array or matrix of samples with sample values, but its scale is smaller than that of the video frame. The number of samples in the horizontal and vertical directions (or axes) of the video block defines the size of the video block. By iteratively using, for example, QT segmentation, binary tree (BT) segmentation, or ternary tree (TT) segmentation, or any combination thereof, a video block can be further segmented into one or more block partitions or sub-blocks (which can then re-form blocks). It should be noted that the term "block" or "video block" as used herein can refer to a portion of a frame or image, particularly a rectangular (square or non-square) portion. Referring, for example, to HEVC and VVC, a block or video block can be or corresponds to a coding tree unit (CTU), CU, prediction unit (PU), or transform unit (TU) and / or can be or corresponds to a corresponding block (e.g., coding tree block (CTB), coding block (CB), prediction block (PB), or transform block (TB)) and / or sub-block.

[0067] The prediction processing unit 41 can select one of several feasible predictive coding modes for the current video block based on error results (e.g., coding rate and distortion level), such as one or more inter-frame predictive coding modes among multiple intra-frame predictive coding modes. The prediction processing unit 41 can provide the resulting intra-frame or inter-frame predictive coding block to adder 50 to generate a residual block, and to adder 62 to reconstruct the coding block for subsequent use as part of a reference frame. The prediction processing unit 41 also provides syntax elements (e.g., motion vectors, intra-frame mode indicators, segmentation information, and other such syntax information) to entropy coding unit 56.

[0068] To select a suitable intra-predictive coding mode for the current video block, the intra-predictive processing unit 46 within the prediction processing unit 41 can perform intra-predictive coding of the current video block in relation to one or more neighboring blocks in the same frame as the current block to be encoded to provide spatial prediction. The motion estimation unit 42 and motion compensation unit 44 within the prediction processing unit 41 perform inter-predictive coding of the current video block in relation to one or more prediction blocks in one or more reference frames to provide temporal prediction. The video encoder 20 can perform multiple coding passes, for example, to select a suitable coding mode for each block of video data.

[0069] In some implementations, motion estimation unit 42 determines an inter-frame prediction mode for the current video frame by generating motion vectors based on a predetermined pattern within the video frame sequence. The motion vectors indicate the displacement of a video block within the current video frame relative to a prediction block within a reference video frame. Motion estimation performed by motion estimation unit 42 is the process of generating motion vectors that estimate the motion of video blocks. For example, the motion vectors may indicate the displacement of a video block within the current video frame or image relative to a prediction block within a reference frame associated with the current block being encoded in the current frame. The predetermined pattern may designate video frames in the sequence as P-frames or B-frames. Intra-frame BC unit 48 may determine vectors (e.g., block vectors) for intra-frame BC coding in a similar manner to how motion estimation unit 42 determines motion vectors for inter-frame prediction, or the block vectors may be determined using motion estimation unit 42.

[0070] Regarding pixel differences, the predicted block for a video block can be, or can correspond to, a block or reference block of a reference frame considered to closely match the video block to be encoded. Pixel differences can be determined by the sum of absolute differences (SAD), sum of squared differences (SSD), or other difference metrics. In some implementations, the video encoder 20 can compute values ​​for sub-integer pixel positions of the reference frame stored in the DPB 64. For example, the video encoder 20 can interpolate values ​​for quarter-pixel positions, eighth-pixel positions, or other fractional pixel positions of the reference frame. Therefore, the motion estimation unit 42 can perform motion search relative to full-pixel positions and fractional pixel positions and output a motion vector with fractional-pixel accuracy.

[0071] The motion estimation unit 42 calculates the motion vector for a video block in an inter-frame predictive coding frame by comparing the position of the video block with the position of the predicted block in a reference frame selected from either a first reference frame list (list 0) or a second reference frame list (list 1), where each reference frame list identifies one or more reference frames stored in the DPB 64. The motion estimation unit 42 sends the calculated motion vector to the motion compensation unit 44, and then to the entropy coding unit 56.

[0072] Motion compensation performed by motion compensation unit 44 may involve acquiring or generating prediction blocks based on motion vectors determined by motion estimation unit 42. Upon receiving motion vectors for the current video block, motion compensation unit 44 may locate the prediction block pointed to by the motion vector in a reference frame list within a reference frame list, retrieve the prediction block from DPB 64, and forward the prediction block to adder 50. Adder 50 then forms a residual video block of pixel differences by subtracting the pixel values ​​of the prediction block provided by motion compensation unit 44 from the pixel values ​​of the currently encoded video block. The pixel differences forming the residual video block may include luminance component differences or chrominance component differences, or both. Motion compensation unit 44 may also generate syntax elements associated with video blocks of a video frame for use by video decoder 30 when decoding video blocks of a video frame. Syntax elements may include, for example, syntax elements defining motion vectors for identifying prediction blocks, any flags indicating prediction modes, or any other syntax information described herein. It should be noted that motion estimation unit 42 and motion compensation unit 44 may be highly integrated, but are described separately for conceptual purposes.

[0073] In some implementations, the intra-BC unit 48 can generate vectors and acquire prediction blocks in a manner similar to that described above in conjunction with the motion estimation unit 42 and the motion compensation unit 44; however, these prediction blocks are in the same frame as the current block being encoded, and these vectors are referred to as block vectors rather than motion vectors. Specifically, the intra-BC unit 48 can determine the intra-prediction mode to be used for encoding the current block. In some examples, the intra-BC unit 48 can, for example, use various intra-prediction modes to encode the current block during individual encoding passes and test their performance through rate-distortion analysis. Next, the intra-BC unit 48 can select a suitable intra-prediction mode from the various tested intra-prediction modes to use and generate an intra-mode indicator accordingly. For example, the intra-BC unit 48 can use rate-distortion analysis to calculate rate-distortion values ​​for the various tested intra-prediction modes and select the intra-prediction mode with the best rate-distortion characteristics from the tested modes as the suitable intra-prediction mode to use. Rate-distortion analysis generally determines the amount of distortion (or error) between the coded block and the original uncoded block that was encoded to produce the coded block, as well as the bit rate (i.e., the number of bits) used to produce the coded block. Intra-frame BC unit 48 can calculate the ratio based on the distortion and rate for various coded blocks to determine which intra-frame prediction mode exhibits the optimal rate-distortion value for the block.

[0074] In other examples, the intra-frame BC unit 48 may use, in whole or in part, the motion estimation unit 42 and the motion compensation unit 44 to perform such functions for intra-frame BC prediction according to the embodiments described herein. In any case, for intra-frame block copying, in terms of pixel differences, the predicted block may be a block considered to closely match the block to be encoded, the pixel differences may be determined by SAD, SSD, or other difference metrics, and identifying the predicted block may include calculating values ​​for sub-integer pixel positions.

[0075] Regardless of whether the predicted block comes from the same frame predicted intra-frame or from different frames predicted inter-frame, the video encoder 20 can form a residual video block by subtracting the pixel values ​​of the predicted block from the pixel values ​​of the current video block being encoded. The pixel difference forming the residual video block can include both luma component difference and chroma component difference.

[0076] As an alternative to the inter-frame prediction performed by the motion estimation unit 42 and the motion compensation unit 44 as described above, or the intra-block copy prediction performed by the intra-BC unit 48, the intra-prediction processing unit 46 can perform intra-frame prediction on the current video block. Specifically, the intra-prediction processing unit 46 can determine an intra-prediction mode for encoding the current block. To this end, the intra-prediction processing unit 46 can use various intra-prediction modes to encode the current block, for example, during individual encoding passes, and the intra-prediction processing unit 46 (or, in some examples, the mode selection unit) can select a suitable intra-prediction mode from the tested intra-prediction modes for use. The intra-prediction processing unit 46 can provide information indicating the intra-prediction mode selected for the block to the entropy coding unit 56. The entropy coding unit 56 can encode the information indicating the selected intra-prediction mode into the bitstream.

[0077] After prediction processing unit 41 determines the prediction block for the current video block via inter-frame prediction or intra-frame prediction, adder 50 forms a residual video block by subtracting the prediction block from the current video block. The residual video data in the residual block may be included in one or more TUs and provided to transform processing unit 52. Transform processing unit 52 uses a transform (e.g., discrete cosine transform (DCT) or a conceptually similar transform) to transform the residual video data into residual transform coefficients.

[0078] The transform processing unit 52 can send the resulting transform coefficients to the quantization unit 54. The quantization unit 54 quantizes the transform coefficients to further reduce the bit rate. The quantization process can also reduce the bit depth associated with some or all of the coefficients. The degree of quantization can be modified by adjusting the quantization parameters. In some examples, the quantization unit 54 can subsequently perform a scan on the matrix including the quantized transform coefficients. Alternatively, the entropy coding unit 56 can perform the scan.

[0079] After quantization, the entropy coding unit 56 entropy codes the quantized transform coefficients into a video bitstream using, for example, context-adaptive variable-length coding (CAVLC), context-adaptive binary arithmetic coding (CABAC), syntax-based context-adaptive binary arithmetic coding (SBAC), probabilistic interval segmented entropy (PIPE) coding, or another entropy coding method or technique. The encoded bitstream can then be sent to, for example,... Figure 1A The video decoder 30 shown, or archived in, for example Figure 1A The data is stored in storage device 32 for later transmission to or retrieval by video decoder 30. Entropy coding unit 56 can also entropy code the motion vectors and other syntax elements used for the current video frame being encoded.

[0080] The inverse quantization unit 58 and the inverse transform processing unit 60 apply inverse quantization and inverse transform, respectively, to reconstruct residual video blocks in the pixel domain for generating reference blocks to predict other video blocks. As noted above, the motion compensation unit 44 can generate motion-compensated prediction blocks from one or more reference blocks of frames stored in the DPB 64. The motion compensation unit 44 can also apply one or more interpolation filters to the prediction blocks to compute sub-integer pixel values ​​for use during motion estimation.

[0081] Adder 62 adds the reconstructed residual block to the motion-compensated prediction block generated by motion compensation unit 44 to generate a reference block for storage in DPB 64. The reference block can then be used as a prediction block by intra-frame BC unit 48, motion estimation unit 42, and motion compensation unit 44 for inter-frame prediction of another video block in subsequent video frames.

[0082] Figure 2B This is a block diagram illustrating another exemplary video decoder 30 according to some embodiments of this application. The video decoder 30 includes a video data memory 79, an entropy decoding unit 80, a prediction processing unit 81, an inverse quantization unit 86, an inverse transform processing unit 88, an adder 90, and a DPB 92. The prediction processing unit 81 further includes a motion compensation unit 82, an intra-frame prediction unit 84, and an intra-frame BC unit 85. The video decoder 30 can perform operations in conjunction with the above. Figure 1G The decoding process described for the video encoder 20 is essentially the inverse of the encoding process. For example, the motion compensation unit 82 can generate prediction data based on the motion vectors received from the entropy decoding unit 80, while the intra-frame prediction unit 84 can generate prediction data based on the intra-frame prediction mode indicator received from the entropy decoding unit 80.

[0083] In some examples, units of the video decoder 30 may be assigned tasks to perform embodiments of this application. Furthermore, in some examples, embodiments of this disclosure may be distributed across one or more units of the video decoder 30. For example, the intra-frame BC unit 85 may perform embodiments of this application individually or in combination with other units of the video decoder 30 (e.g., motion compensation unit 82, intra-frame prediction unit 84, and entropy decoding unit 80). In some examples, the video decoder 30 may not include the intra-frame BC unit 85, and the functionality of the intra-frame BC unit 85 may be performed by other components of the prediction processing unit 81 (e.g., motion compensation unit 82).

[0084] Video data memory 79 can store video data, such as encoded video bitstreams, that will be decoded by other components of video decoder 30. The video data stored in video data memory 79 can be retrieved, for example, from storage device 32, from a local video source (e.g., a camera), via wired or wireless network communication of video data, or by accessing a physical data storage medium (e.g., a flash drive or hard disk). Video data memory 79 may include an encoded image buffer (CPB) that stores encoded video data from the encoded video bitstream. The DPB 92 of video decoder 30 stores reference video data for use by video decoder 30 when decoding video data (e.g., in intra-frame or inter-frame predictive coding modes). Video data memory 79 and DPB 92 can be formed of any memory device from a variety of memory devices, such as dynamic random access memory (DRAM) (including synchronous DRAM (SDRAM)), magnetoresistive RAM (MRAM), resistive RAM (RRAM), or other types of memory devices. For illustrative purposes, video data memory 79 and DPB 92 are shown in... Figure 2B The video data memory 79 and DPB 92 are depicted as two distinct components of the video decoder 30. However, it will be apparent to those skilled in the art that the video data memory 79 and DPB 92 may be provided by the same memory device or separate memory devices. In some examples, the video data memory 79 may be on-chip along with other components of the video decoder 30, or off-chip relative to those components.

[0085] During the decoding process, the video decoder 30 receives an encoded video bitstream representing video blocks of encoded video frames and associated syntax elements. The video decoder 30 may receive syntax elements at the video frame level and / or the video block level. The entropy decoding unit 80 of the video decoder 30 performs entropy decoding on the bitstream to generate quantization coefficients, motion vectors or intra-prediction mode indicators, and other syntax elements. The entropy decoding unit 80 then forwards the motion vectors or intra-prediction mode indicators, and other syntax elements to the prediction processing unit 81.

[0086] When a video frame is encoded as an intra-predictive coded (I) frame or used as an intra-coded prediction block in other types of frames, the intra-prediction unit 84 of the prediction processing unit 81 can generate prediction data for the video block of the current video frame based on the intra-prediction mode transmitted by the signal and reference data from the previous decoded block of the current frame.

[0087] When a video frame is encoded as an inter-frame predictive coded (i.e., B or P) frame, the motion compensation unit 82 of the prediction processing unit 81 generates one or more prediction blocks for the current video frame based on motion vectors and other syntax elements received from the entropy decoding unit 80. Each of the prediction blocks can be generated from a reference frame within a reference frame list. The video decoder 30 can construct the reference frame list, i.e., list 0 and list 1, based on the reference frames stored in the DPB 92 using a default construction technique.

[0088] In some examples, when a video block is encoded according to the intra-BC mode described herein, the intra-BC unit 85 of the prediction processing unit 81 generates a prediction block for the current video block based on the block vector and other syntax elements received from the entropy decoding unit 80. The prediction block can be located within the reconstructed region of the same image as the current video block, as defined by the video encoder 20.

[0089] Motion compensation unit 82 and / or intra-frame prediction (BC) unit 85 determine prediction information for video blocks in the current video frame by parsing motion vectors and other syntax elements, and then use this prediction information to generate prediction blocks for the current video block being decoded. For example, motion compensation unit 82 uses some of the received syntax elements to determine the prediction mode (e.g., intra-frame prediction or inter-frame prediction) for encoding video blocks in the video frame, the inter-frame prediction frame type (e.g., B or P), construction information for one or more reference frames in the reference frame list for the frame, motion vectors for each inter-frame prediction encoded video block in the frame, the inter-frame prediction state for each inter-frame prediction encoded video block in the frame, and other information for decoding video blocks in the current video frame.

[0090] Similarly, the intra-BC unit 85 can use some of the received syntax elements, such as flags, to determine which video blocks in the frame are predicted using the intra-BC mode, which video blocks in the frame are in the reconstruction region and should be stored in the DPB 92, the block vector for each intra-BC predicted video block in the frame, the intra-BC prediction state for each intra-BC predicted video block in the frame, and other information for decoding video blocks in the current video frame.

[0091] The motion compensation unit 82 can also perform interpolation using interpolation filters, such as those used by the video encoder 20 during encoding of video blocks, to calculate interpolated values ​​for sub-integer pixels of the reference block. In this case, the motion compensation unit 82 can determine the interpolation filters used by the video encoder 20 based on the received syntax elements and use these interpolation filters to generate the prediction block.

[0092] The dequantization unit 86 dequantizes the quantized transform coefficients provided in the bitstream and entropy-decoded by the entropy decoding unit 80 using the same quantization parameters calculated by the video encoder 20 for each video block in the video frame to determine the degree of quantization. The inverse transform processing unit 88 applies an inverse transform (e.g., inverse DCT, inverse integer transform, or a conceptually similar inverse transform process) to the transform coefficients in order to reconstruct the residual block in the pixel domain.

[0093] After the motion compensation unit 82 or the intra-frame BC unit 85 generates a prediction block for the current video block based on vectors and other syntax elements, the adder 90 reconstructs the decoded video block for the current video block by adding the residual block from the inverse transform processing unit 88 to the corresponding prediction block generated by the motion compensation unit 82 and the intra-frame BC unit 85. A loop filter 91 (e.g., a deblocking filter, SAO filter, CCSAO filter, and / or ALF) may be located between the adder 90 and the DPB 92 for further processing of the decoded video block. In some examples, the loop filter 91 may be omitted, and the decoded video block may be directly provided to the DPB 92 by the adder 90. The decoded video block in a given frame is then stored in the DPB 92, which stores reference frames for subsequent motion compensation of the next video block. The DPB 92 or a separate memory device may also store the decoded video for later presentation on a display device (e.g., ...). Figure 1A On the display device 34).

[0094] In current VVC and AVS3 standards, motion information for the current coded block is copied from spatially or temporally adjacent blocks specified by the merge candidate index, or obtained through explicit signaling of motion estimation. The purpose of this disclosure is to improve the accuracy of motion vectors in affine merging patterns by refining the method of deriving affine merge candidates. For ease of description, existing affine merging pattern designs in the VVC standard are used as examples to illustrate the proposed ideas. Note that while existing affine pattern designs in the VVC standard are used as examples throughout this disclosure, the proposed techniques can be applied to different designs of affine motion prediction patterns or other codec tools with the same or similar design principles, and will be readily apparent to those skilled in the art of modern video coding and decoding.

[0095] In a typical video coding process, a video sequence usually consists of an ordered set of frames or images. Each frame may include three sample arrays, denoted as SL, SCb, and SCr. SL is a two-dimensional array of luminance samples. SCb is a two-dimensional array of Cb chrominance samples. SCr is a two-dimensional array of Cr chrominance samples. In other instances, a frame may be monochrome and therefore consist of only a two-dimensional array of luminance samples.

[0096] like Figure 1C As shown, the video encoder 20 (or more specifically, the segmentation unit in the predictive processing unit of the video encoder 20) generates a coded representation of a frame by first segmenting the frame into a set of CTUs. A video frame may include an integer number of CTUs ordered consecutively from left to right and from top to bottom in raster scan order. Each CTU is the largest logical coding unit, and the width and height of the CTU are signaled by the video encoder 20 in a sequence parameter set such that all CTUs in the video sequence have the same size, one of 128×128, 64×64, 32×32, and 16×16. However, it should be noted that this application is not limited to a specific size. Figure 1D As shown, each CTU may include a CTB for the luminance sample, two corresponding coding tree blocks for the chrominance sample, and syntax elements for encoding the samples of the coding tree blocks. The syntax elements describe the properties of different types of units in the coded pixel block and how the video sequence can be reconstructed at the video decoder 30, including inter-frame prediction or intra-frame prediction, intra-frame prediction mode, motion vectors, and other parameters. In a monochrome image or an image with three separate color planes, the CTU may include a single coding tree block and syntax elements for encoding the samples of that coding tree block. The coding tree block may be an N×N sample block.

[0097] To achieve better performance, the video encoder 20 can recursively perform tree splitting on the coding tree blocks of the CTU, such as binary tree splitting, ternary tree splitting, quadtree splitting, or combinations thereof, and divide the CTU into smaller CUs. Figure 1E As described, the 64×64 CTU 400 is first divided into four smaller CUs, each with a block size of 32×32. Of these four smaller CUs, CU 410 and CU 420 are each divided into four CUs with a block size of 16×16. The two 16×16 CUs, 430 and CU 440, are further divided into four CUs with a block size of 8×8. Figure 1F Depicting as shown Figure 1E The final result of the CTU 400 partitioning process described in the figure is a quadtree data structure, where each leaf node of the quadtree corresponds to a CU of a corresponding size ranging from 32×32 to 8×8. Similar to... Figure 1DThe CTU depicted in the image may include, for each CU, two corresponding coded blocks of luminance and chrominance samples of the same size frame, as well as syntax elements for encoding the samples of the coded blocks. In monochrome images or images with three separate color planes, a CU may include a single coded block and a syntax structure for encoding the samples of the coded block. It should be noted that... Figures 1E to 1F The quadtree partitioning depicted is for illustrative purposes only, and a CTU can be split into multiple CUs based on quadtree / ternary / binary partitioning to adapt to varying local characteristics. In multi-type tree structures, a CTU is partitioned according to a quadtree structure, and each quadtree leaf CU can be further partitioned according to binary and ternary tree structures. Figures 3A-3E As shown, a coded block with width W and height H has five possible segmentation types: quad segmentation, horizontal binary segmentation, vertical binary segmentation, horizontal triple segmentation, and vertical triple segmentation.

[0098] In some implementations, the video encoder 20 may further segment the coded blocks of the CU into one or more (M×N) PBs. A PB is a rectangular (square or non-square) sample block to which the same prediction (inter-frame or intra-frame) is applied. A PU of the CU may include a PB for luma samples, two corresponding PBs for chroma samples, and syntax elements for predicting the PBs. In a monochrome image or an image with three separate color planes, a PU may include a single PB and a syntax structure for predicting the PBs. The video encoder 20 may generate predicted luma blocks, predicted Cb blocks, and predicted Cr blocks for each PU of the CU, representing the luma PB, Cb PB, and Cr PB.

[0099] Video encoder 20 can generate prediction blocks for a PU using intra-frame prediction or inter-frame prediction. If video encoder 20 uses intra-frame prediction to generate prediction blocks for a PU, then video encoder 20 can generate prediction blocks for a PU based on decoded samples of the frame associated with the PU. If video encoder 20 uses inter-frame prediction to generate prediction blocks for a PU, then video encoder 20 can generate prediction blocks for a PU based on decoded samples of one or more frames other than the frame associated with the PU.

[0100] After the video encoder 20 generates predicted luminance blocks, predicted Cb blocks, and predicted Cr blocks for one or more PUs of the CU, the video encoder 20 can generate luminance residual blocks for the CU by subtracting the predicted luminance blocks of the CU from the original luminance coding blocks of the CU, such that each sample in the luminance residual block of the CU indicates the difference between a luminance sample in one of the predicted luminance blocks of the CU and a corresponding sample in the original luminance coding block of the CU. Similarly, the video encoder 20 can generate Cb residual blocks and Cr residual blocks for the CU, respectively, such that each sample in the Cb residual block of the CU indicates the difference between a Cb sample in one of the predicted Cb blocks of the CU and a corresponding sample in the original Cb coding block of the CU, and each sample in the Cr residual block of the CU indicates the difference between a Cr sample in one of the predicted Cr blocks of the CU and a corresponding sample in the original Cr coding block of the CU.

[0101] In addition, such as Figure 1E As shown, the video encoder 20 can use quadtree partitioning to decompose the luminance residual block, Cb residual block, and Cr residual block of the CU into one or more luminance transform blocks, Cb transform blocks, and Cr transform blocks, respectively. A transform block is a rectangular (square or non-square) sample block to which the same transform is applied. A TU of the CU can include a transform block of luminance samples, two corresponding transform blocks of chrominance samples, and syntax elements for transforming the transform block samples. Therefore, each TU of the CU can be associated with a luminance transform block, a Cb transform block, and a Cr transform block. In some examples, the luminance transform block associated with a TU can be a sub-block of the CU's luminance residual block. A Cb transform block can be a sub-block of the CU's Cb residual block. A Cr transform block can be a sub-block of the CU's Cr residual block. In a monochrome image or an image with three separate color planes, a TU can include a single transform block and syntax structures for transforming the samples of that transform block.

[0102] The video encoder 20 can apply one or more transforms to the luminance transform block of the TU to generate a luminance coefficient block for the TU. The coefficient block can be a two-dimensional array of transform coefficients. The transform coefficients can be scalars. The video encoder 20 can apply one or more transforms to the Cb transform block of the TU to generate a Cb coefficient block for the TU. The video encoder 20 can apply one or more transforms to the Cr transform block of the TU to generate a Cr coefficient block for the TU.

[0103] After generating coefficient blocks (e.g., luminance coefficient blocks, Cb coefficient blocks, or Cr coefficient blocks), video encoder 20 can quantize the coefficient blocks. Quantization typically refers to the process of quantizing transform coefficients to potentially reduce the amount of data used to represent the transform coefficients, thereby providing further compression. After quantizing the coefficient blocks, video encoder 20 can entropy encode the syntax elements indicating the quantized transform coefficients. For example, video encoder 20 can perform CABAC on the syntax elements indicating the quantized transform coefficients. Finally, video encoder 20 can output a bitstream comprising a bit sequence that forms a representation of coded frames and associated data; the bitstream is stored in storage device 32 or transmitted to target device 14.

[0104] After receiving the bitstream generated by the video encoder 20, the video decoder 30 can parse the bitstream to obtain syntax elements. The video decoder 30 can reconstruct frames of video data, at least in part, based on the syntax elements obtained from the bitstream. The video data reconstruction process is generally the inverse of the encoding process performed by the video encoder 20. For example, the video decoder 30 can perform an inverse transform on the coefficient block associated with the TU of the current CU to reconstruct the residual block associated with the TU of the current CU. The video decoder 30 also reconstructs the coded blocks of the current CU by adding samples of the predicted blocks of the PU for the current CU to corresponding samples of the transformed blocks of the TU of the current CU. After reconstructing the coded blocks for each CU of the frame, the video decoder 30 can reconstruct the frame.

[0105] As mentioned above, video coding primarily uses two modes (i.e., intra-frame prediction and inter-frame prediction) to achieve video compression. It should be noted that IBC can be considered either intra-frame prediction or a third mode. Between the two modes, inter-frame prediction contributes more to coding efficiency than intra-frame prediction because it uses motion vectors to predict the current video block based on a reference video block.

[0106] However, with continuously improving video data capture technologies and finer video tile sizes used to preserve details in video data, the amount of data required to represent the motion vectors for the current frame has also increased significantly. One way to overcome this challenge benefits from the fact that a set of neighboring CUs in the spatial and temporal domains not only have similar video data for prediction purposes, but the motion vectors among these neighboring CUs are also similar. Therefore, the motion information of spatially neighboring CUs and / or temporally co-located CUs can be used as an approximation of the motion information (e.g., motion vectors) of the current CU (also known as the "Motion Vector Prediction" (MVP) of the current CU) by exploring their spatial and temporal correlations.

[0107] Instead of the above combination Figure 1BThe method described involves encoding the actual motion vector of the current CU, determined by the motion estimation unit, into the video bitstream, and subtracting the predicted motion vector value of the current CU from the actual motion vector of the current CU to produce the motion vector difference (MVD) for the current CU. By doing so, it is not necessary to encode the motion vector determined by the motion estimation unit for each CU of the frame into the video bitstream, and the amount of data used to represent motion information in the video bitstream can be significantly reduced.

[0108] Similar to the process of selecting a prediction block in a reference frame during inter-frame prediction of a coded block, both the video encoder 20 and the video decoder 30 need to employ a set of rules to construct a motion vector candidate list (also known as a "merging list") for the current CU using those potential candidate motion vectors associated with spatially neighboring CUs and / or temporally co-located CUs. Then, a member is selected from the motion vector candidate list as the motion vector prediction value for the current CU. By doing so, it is not necessary to send the motion vector candidate list itself from the video encoder 20 to the video decoder 30; the index of the selected motion vector prediction value within the motion vector candidate list is sufficient for both the video encoder 20 and the video decoder 30 to encode and decode the current CU using the same motion vector prediction value from the motion vector candidate list.

[0109] This disclosure aims to further improve the chroma encoding and decoding efficiency of motion compensation modules applied in ECM. The following sections briefly review some relevant encoding and decoding tools used in the transform and entropy coding processes of ECM. Then, some shortcomings in existing motion compensation designs are discussed. Finally, solutions to improve existing designs are provided.

[0110] Motion Compensation Prediction (MCP)

[0111] Motion-compensated prediction (MCP), also known simply as motion compensation, is one of the most widely used video codec techniques in modern video codec standards. In MCP, a video frame is divided into multiple blocks (called prediction units (PUs)). Each PU is predicted based on blocks of the same size from a temporal reference image, significantly reducing the overhead required to signal that block. In all existing video codec standards, each inter-frame PU is associated with a set of motion parameters, including one or two motion indicators (MVs) and a reference image index. An internal PU in a P-strip has only one list of reference images, while a PU in a B-strip can use up to two lists of reference images. In MCP, the corresponding inter-frame prediction samples are generated from the corresponding regions of the reference image identified by the MV and the reference image index. The MV specifies the horizontal and vertical displacement between the current block and its reference block in the reference image. Figure 4An example is shown where dx and dy are the horizontal and vertical values ​​of MV. In practice, a value of MV can be fractional in precision. Figure 5 As shown, when an MV has a fractional value, an interpolation filter is applied to generate the corresponding predicted sample at the fractional sample location. In VVC, MVs are supported in units of 1 / 16 of the distance between two adjacent luminance samples of the luminance MC and 1 / 32 of the distance between two adjacent chrominance samples of the chrominance MC.

[0112] Adaptive Loop Filter

[0113] In VVC and ECM, the Adaptive Loop Filter (ALF) selects one of 25 filters for each 4×4 block based on the direction and activity of the local gradient.

[0114] Filter shapes: Two diamond filter shapes were used (e.g. Figures 6A to 6B (As shown). A 7×7 rhombus is applied to the luminance component, and a 5×5 rhombus is applied to the chrominance component.

[0115] Block classification: For the luminance component, each The block is divided into one of 25 categories. The classification index C is based on its directionality. Quantitative values ​​of activities Export, as shown below:

[0116] In order to calculate and First, the gradients in the horizontal, vertical, and two diagonal directions are calculated using the 1-D Laplacian operator:

[0117] Where index and It refers to The coordinates of the top-left sample within the block, and Indicator coordinates Reconstructed samples at the location. To reduce the complexity of block classification, such as... Figure 7 As shown, the gradient calculation in all directions is performed using subsampled 1-D Laplace calculation.

[0118] Then, The maximum and minimum values ​​of the gradients in the horizontal and vertical directions are set as follows:

[0119] The maximum and minimum values ​​of the gradients in the two diagonal directions are set as follows:

[0120] To derive directionality The values ​​are compared with each other and compared with two thresholds. and Comparison: Step 1 If If both are true, then Set as .

[0121] Step 2 If If yes, continue from step 3; otherwise, continue from step 4.

[0122] Step 3 If ;otherwise Set as .

[0123] Step 4 If ,but Set as ;otherwise Set as .

[0124] Activity Value The calculation is as follows: It is further quantized to 0 to 4, and the quantized value is represented as For the chromaticity components in an image, no classification method should be applied.

[0125] Geometric transformation of filter coefficients and limiting values

[0126] Before filtering each 4×4 luminance block, geometric transformations (such as rotation or diagonal and vertical flipping) are applied to the filter coefficients. and the corresponding filter limiting value The specific value depends on the gradient value calculated for that block. This is equivalent to applying these transformations to samples in the filter's support region. The idea is to make different blocks to which ALF is applied more similar by arranging the orientation of the different blocks.

[0127] Three geometric transformations are provided: diagonal flip, vertical flip, and rotation.

[0128] in It is the size of the filter. These are coefficient coordinates, therefore the position In the top left corner, the location In the bottom right corner. Based on the gradient values ​​calculated for this block, the transform is applied to the filter coefficients. f ( k ,l) and limit value Table 1 below summarizes the relationship between the transformation and the four gradients in the four directions.

[0129]

[0130] Table 1

[0131] Filtering process

[0132] When ALF is enabled for CTB, each sample within the CU The filtered values ​​produce the sample values ​​shown below. .

[0133]

[0134] in These represent the filter coefficients used for decoding. It is a limiting function. This represents the limiting parameter for decoding. Variables k and L are between... and The interval is between, where L represents the filter length. It is a clipping function that clips the input value of x to the range [-y, y]. The clipping operation introduces non-linearity, making ALF more efficient by reducing the influence of neighboring sample values ​​that differ too much from the current sample value.

[0135] Cross-component adaptive loop filter

[0136] The Cross-Component Adaptive Loop Filter (CC-ALF) refines each of the two chroma components using luminance samples by applying an adaptive linear filter to the luminance channel and then using the output of this filter operation for chroma refinement. Figure 8 As shown, filtering in CC-ALF is achieved by applying a diamond filter to the luminance channel. Each chroma channel uses one filter, and the operation is represented as follows:

[0137] in, It's the chroma position that's being refined. Based on The matching brightness position, It is the filter support region in the luminance component. .

[0138] Up to eight CC-ALF filters can be designed and transmitted per image. The resulting filters are then indicated for each of the two chroma channels based on the CTU. Furthermore, existing CC-ALF designs include the following features: The design uses a 3x4 rhombus shape and has 8 taps.

[0139] The seven filter coefficients are transmitted at the access point.

[0140] Each transmission coefficient has a 6-bit dynamic range and is limited to a power of 2.

[0141] The eighth filter coefficients are derived at the decoder such that the sum of the filter coefficients equals 0.

[0142] APS can be referenced in the strip header.

[0143] For each chromaticity component, the selection of the CC-ALF filter is controlled at the CTU level.

[0144] Among all existing video codec standards, MCP plays a crucial role in ensuring the efficiency of inter-frame coding and decoding. Using MCP, the video signal to be encoded is predicted from temporally adjacent signals, and only the prediction error, MV, and reference image index are transmitted. Meanwhile, ALF and CC-ALF can effectively improve the quality of the reconstructed video, thereby improving the performance of inter-frame coding by providing a high-quality reference image. However, the quality of the temporal reference image may be insufficient to provide effective inter-frame prediction, especially for the chroma component, for the following reasons: Video signals can be encoded using coarse quantization, i.e., high quantization parameter (QP) values. When coarse quantization is applied, the reconstructed image may contain severe coding artifacts, such as block artifacts, ringing artifacts, etc. This can cause some high-frequency information present in the original image to be lost and / or distorted in the reconstructed image, for example, in the form of distorted edges and blurred textures. Assuming that the reconstructed signal of the current image will be used as a reference for temporal prediction, this lost and / or distorted high-frequency information reduces the effectiveness of MCP and thus reduces the inter-frame coding efficiency of subsequent images.

[0145] Because the human visual system is more sensitive to changes in brightness than to changes in color, video coding systems typically allocate more bits to the luma component than to the chroma component, for example, by adjusting the QP delta value between the luma and chroma components. Furthermore, the chroma component generally has a smaller dynamic range and is therefore smoother than the luma component. Consequently, more of the chroma component's transform coefficients become zero after quantization. Therefore, the problem of lost or distorted high-frequency information is more pronounced in the reconstructed chroma signal. This can severely impact the prediction efficiency of the chroma component, as more bits need to be generated to encode the chroma residual signal. While CC-ALF may be able to recover lost high-frequency information in the reconstructed image, this high-frequency information may be attenuated during the motion compensation stage when it is used as a reference image for inter-frame prediction.

[0146] This disclosure proposes a method to improve the efficiency of motion-compensated prediction for chromaticity components, and thus improve the quality of temporal prediction. Specifically, an adaptive cross-component filtering method, termed Cross-Component Motion Compensation Prediction (CC-MCP), is proposed to be applied in the motion compensation stage. This method uses high-frequency information from the motion-compensated luminance samples as guidance to improve the quality of the motion-compensated chromaticity samples. In this way, the energy of the chromaticity residual is minimized, thereby reducing the overhead of transmitting the chromaticity signal.

[0147] Figure 9 A block diagram of the video encoder when the proposed CC-MCP is applied is provided. First, similar to a conventional video encoder, the motion estimation and compensation module generates motion-compensated luma and chroma signals by matching the current block with a block in a reference image using the optimal MV. Then, an adaptive cross-component filter, i.e., CC-MCP filtering, is provided, where the motion-compensated chroma signal is filtered using the proposed CC-MCP filter according to the corresponding motion-compensated luma signal to generate a filtered motion-compensated chroma signal. Subsequently, the original signal is subtracted from the predicted signal to remove temporal redundancy and generate the corresponding residual signal. The residual signal is transformed and quantized, then entropy-encoded and output as a bitstream. To obtain the reconstructed signal, the reconstructed residual signal can be obtained through inverse quantization and inverse transform. The reconstructed residual is then added to the motion-compensated prediction. Furthermore, loop filtering processes (e.g., deblocking, ALF, and SAO) are applied to the reconstructed video signal for output. As will be discussed later, the filter coefficients of the proposed CC-MCP filter can be derived directly from adjacent reconstructed luma and chroma samples at the decoder, or derived at the encoder and sent to the decoder. Furthermore, to maximize the coding gain of the proposed method, additional syntax can be signaled at a given block level (e.g., CTU, CU, or PU level) to indicate whether the proposed CC-MCP filter is applied to the current block for motion compensation.

[0148] Figure 10 The proposed receiver is shown. Figure 9 The diagram shows the block diagram of the bitstream decoder generated by the encoder. At the decoder, the bitstream is first parsed by an entropy decoder. The residual coefficients are then inversely quantized and inversely transformed to obtain the reconstructed residuals. For time prediction, a prediction signal is first generated by obtaining motion-compensated blocks using prediction information from the signal transmission (i.e., MV and reference index). Then, if CC-MCP is enabled for that block, the motion-compensated chroma signal is further processed by the proposed CC-MCP filtering; otherwise, the motion-compensated chroma signal is not filtered. The motion-compensated signal (filtered or unfiltered) and the reconstructed residuals are then added to obtain the reconstructed video. The reconstructed video may be further passed through a loop filter before being stored in a reference image memory for display and / or used to decode future video signals.

[0149] In the second method, the proposed CC-MCP uses high-frequency information from reconstructed luminance samples as guidance to improve the quality of motion-compensated chrominance samples, as shown in the block diagram below. Figure 12 As shown. At the encoder, firstly, similar to a traditional video encoder, the motion estimation and compensation module generates motion-compensated luminance and chrominance signals by matching the current block with a block in the reference image using the optimal MV. Subsequently, the original luminance signal is subtracted from the predicted luminance signal to remove temporal luminance redundancy and generate a corresponding residual luminance signal. The residual luminance signal is then transformed and quantized, and finally entropy-coded and output as a bitstream.

[0150] To obtain the reconstructed luminance signal, the reconstructed residual luminance signal can be obtained through inverse quantization and inverse transform. Then, the reconstructed residual luminance is added to the motion-compensated luminance prediction. Next, a CC-MCP filter is introduced, where the motion-compensated chrominance signal is filtered using the proposed CC-MCP filter based on the corresponding reconstructed luminance signal to generate a filtered motion-compensated chrominance signal. Subsequently, the original chrominance signal is subtracted from the predicted chrominance signal to remove temporal chrominance redundancy and generate the corresponding residual chrominance signal. The residual chrominance signal is then transformed and quantized, and finally entropy-coded and output as a bitstream.

[0151] To obtain the reconstructed chromaticity signal, the reconstructed residual chromaticity signal can be obtained through inverse quantization and inverse transform. Then, the reconstructed chromaticity residual is added to the motion-compensated chromaticity prediction.

[0152] Furthermore, loop filtering processes, such as deblocking, ALF, and SAO, are applied to the reconstructed video signal (containing the reconstructed luma and chroma signals) for output. As discussed later, the filter coefficients of the proposed CC-MCP filter can be derived directly from adjacent reconstructed luma and chroma samples at the decoder, or derived at the encoder and sent to the decoder. Additionally, to maximize the coding gain of the proposed method, additional syntax can be signaled at a given block level (e.g., CTU, CU, or PU level) to indicate whether the proposed CC-MCP filter is applied to the current block for motion compensation.

[0153] At the decoder, the bitstream is first parsed by an entropy decoder. The residual coefficients are then inversely quantized and inversely transformed to obtain the reconstructed residuals. For time prediction, the prediction signal is first generated by acquiring motion-compensated blocks using the prediction information transmitted by the signal (i.e., MV and reference index).

[0154] Then, the motion-compensated luminance signal and the reconstructed residual luminance are added together to obtain the reconstructed luminance signal.

[0155] Then, if CC-MCP is found to be enabled for that block from the bitstream, the motion-compensated chroma signal is further processed by the proposed CC-MCP filter; otherwise, the motion-compensated chroma signal is not filtered.

[0156] Then, the motion-compensated chromaticity signal (filtered or unfiltered) and the reconstructed residual chromaticity are added together to obtain the reconstructed chromaticity signal.

[0157] First, the reconstructed video (including the reconstructed luminance signal and the reconstructed chrominance signal) may pass through a loop filter before being stored in the reference image memory for display and / or used to decode future video signals.

[0158] CC-MCP filtering of motion-compensated chroma signals

[0159] Because the human visual system is more sensitive to changes in brightness than to changes in color, video coding systems typically allocate more bits to the luminance component than to the chrominance component, for example, by adjusting the QP delta value between the luminance and chrominance components. Therefore, the chrominance component is generally smoother than the luminance component. As a result, more transform coefficients are quantized to zero, and there will be more blurred edges and textures in the reconstructed chrominance signal. This can reduce the prediction efficiency of chrominance, thus requiring more overhead in encoding the chrominance residual. Although an ALF filter can be applied to reduce distortion between the reference and original chrominance signals, it cannot recover the lost high-frequency information in the reconstructed chrominance signal due to the low-pass nature of the ALF filter.

[0160] In this disclosure, blurred edges and textures in the chroma channel of a time-predicted signal can be recovered or repaired using corresponding neighboring samples in the luminance channel. Specifically, a method is provided for applying cross-component filtering during the motion compensation stage, which uses high-frequency information from the motion-compensated luminance signal as guidance to improve the quality of the motion-compensated chroma signal. Specifically, it is assumed that... and Indicator coordinates The original reconstructed chromaticity sample and the filtered reconstructed chromaticity sample at the location; Indicates application to reconstructing brightness samples The corresponding The coefficients of the high-pass filter in the adjacent region, where , The proposed CC-MCP filter can be calculated based on the following equation.

[0161]

[0162] In another embodiment, in addition to the linear filter coefficients (i.e., in (9) In addition to the above, it is recommended to introduce additional offset and nonlinear terms into the proposed CC-MCP filter design. Specifically, based on the same notation in (9), the chromaticity prediction samples generated from the CC-MCP process can be formulated as follows.

[0163]

[0164] in It's an offset, it's... The nonlinear term is represented by the juxtaposed brightness samples. A series of power (i.e.) k = 2, …, K-1 ) and.

[0165] The second method proposes applying cross-component filtering during the motion compensation stage, using high-frequency information from the reconstructed luminance signal as guidance to improve the quality of the motion-compensated chrominance signal. Specifically, it assumes... and Indicator coordinates The original motion-compensated chroma sample and the filtered motion-compensated chroma sample at the location; Indicates application to reconstructing brightness samples The corresponding (sum of motion-compensated brightness samples and reconstructed residual brightness samples) The coefficients of the high-pass filter in the adjacent region, where , The proposed CC-MCP filter can be calculated as follows.

[0166]

[0167] In another embodiment, in addition to the linear filter coefficients (i.e., in (11) In addition to the above, it is recommended to introduce additional offset and nonlinear terms into the proposed CC-MCP filter design. Specifically, based on the same symbols in (11), the chromaticity prediction samples generated from the CC-MCP process can be formulated as follows.

[0168]

[0169] in It's an offset, it's... The nonlinear term is represented by the juxtaposed reconstructed brightness samples. A series of power (i.e.) k = 2, …, K-1 ) and.

[0170] The third method proposes applying cross-component filtering during the motion compensation stage, using the reconstructed luminance signal as a guide to improve the quality of the motion-compensated chrominance signal. Specifically, it assumes... and Indicator coordinates The original motion-compensated chroma sample and the filtered motion-compensated chroma sample at the location; Indicates application to reconstructing brightness samples The corresponding (sum of motion-compensated brightness samples and reconstructed residual brightness samples) The coefficients of the filter in the adjacent region, where , The proposed CC-MCP filter can be calculated as follows.

[0171]

[0172] In another embodiment, in addition to the linear filter coefficients (i.e., in (13) In addition to the above, it is recommended to introduce additional offset and nonlinear terms into the proposed CC-MCP filter design. Specifically, based on the same notation in (13), the chromaticity prediction samples generated from the CC-MCP process can be formulated as follows.

[0173]

[0174] in It's an offset, it's... The nonlinear term is represented by the juxtaposed reconstructed brightness samples. A series of power (i.e.) k = 2, …, K-1 ) and.

[0175] Decoder-side derivation of CC-MCP filter coefficients

[0176] The following sections present three decoder-side methods consistent with the three methods presented in the "CC-MCP Filtering Process for Motion-Compensated Chroma Signals" section, wherein the coefficients of the proposed CC-MCP filter are derived at the decoder without signaling. Specifically, for the first method presented in the "CC-MCP Filtering Process for Motion-Compensated Chroma Signals" section, when CC-MCP filtering is applied to a block, this method derives coefficients from the neighboring reconstructed chroma samples of the current block and their corresponding luminance and chroma prediction samples. Figure 11 The proposed method for deriving the decoder-side filter coefficients is illustrated. Specifically, given a block... and its predefined adjacent regions (For example, (From the reconstructed chroma samples in the image), we can use the encoded MV of the current block to find the corresponding luminance prediction sample. and colorimetric prediction samples Then the LMMSE method can be used, by... and The filter coefficients are derived by using them as input to the CC-MCP filter and then minimized. The difference between the output of the CC-MCP filter and the output of the CC-MCP filter, i.e.

[0177] Subsequently, the derived filter can be applied to enhance the chromaticity prediction signal of the current block, as shown in Equation (9).

[0178] When considering the offset and nonlinear terms in (10), the derivation of the CC-MCP filter coefficients based on LMMSE can be expressed by the following formula.

[0179]

[0180] On the other hand, since the proposed method uses motion-compensated samples from the adjacent region of the target for LMMSE derivation, it may be more beneficial to apply the proposed decoder-side derivation method when the reconstructed signal of the current image contains higher quality reconstruction information than the reconstructed signal of the reference image. Therefore, in one embodiment of this disclosure, the proposed decoder-side derivation method is applied only when the reference image uses a smaller QP value than the current image.

[0181] For the second method proposed in the section "CC-MCP Filtering Process for Motion-Compensated Chroma Signals," specifically, when CC-MCP filtering is applied to a block, this method derives coefficients from the neighboring reconstructed chroma samples of the current block and their corresponding reconstructed luminance samples and chroma prediction samples. Specifically, given a block... and its predefined adjacent regions (For example, (In the reconstructed chroma samples), we can use the encoded MV of adjacent blocks to find the corresponding reconstructed luminance samples. and colorimetric prediction samples In some other embodiments, chroma prediction samples can be obtained by using the encoded MV of the current block. Then the LMMSE method can be used, by... and The filter coefficients are derived by using them as input to the CC-MCP filter and then minimized. The difference between the output of the CC-MCP filter and the output of the CC-MCP filter is expressed by the following formula.

[0182]

[0183] Subsequently, the derived filter can be applied to enhance the chromaticity prediction signal of the current block, as shown in (11).

[0184] When considering the offset and nonlinear terms in (12), the derivation of the CC-MCP filter coefficients based on LMMSE can be expressed by the following formula.

[0185]

[0186] In one embodiment of this disclosure, the proposed decoder-side derivation method is applied only when the reference image uses a smaller QP value than the current image.

[0187] The third method proposed in the section "CC-MCP Filtering Process for Motion-Compensated Chroma Signals" specifically derives coefficients from the neighboring reconstructed chroma samples and their corresponding reconstructed luminance samples of the current block when CC-MCP filtering is applied to a block. Specifically, given a block... and its predefined adjacent regions (For example, From the reconstructed chromaticity samples, we can find the corresponding reconstructed luminance samples. Then the LMMSE method can be used, by... The filter coefficients are derived by using them as input to the CC-MCP filter and then minimized. The difference between the output of the CC-MCP filter and the output of the CC-MCP filter is expressed by the following formula.

[0188]

[0189] Subsequently, the derived filter can be applied to enhance the chromaticity prediction signal of the current block, as shown in (13).

[0190] When considering the offset and nonlinear terms in (14), the derivation of the CC-MCP filter coefficients based on LMMSE can be expressed by the following formula.

[0191]

[0192] In one embodiment of this disclosure, the proposed decoder-side derivation method is applied only when the reference image uses a smaller QP value than the current image.

[0193] Explicit signaling of CC-MCP filter coefficients

[0194] In the above method, the CC-MCP filter coefficients are derived from neighboring reconstructed samples, which may be inaccurate because neighboring reconstructed samples may always be highly correlated with samples in the current block. To address this issue, in one embodiment, it is proposed to derive the CC-MCP filter coefficients at the encoder and explicitly transmit them as a signal to the decoder.

[0195] When using this signaling-based scheme in a practical video coding system, the adaptation of CC-MCP filter coefficients can be applied at various coding levels, such as sequence level, image / strip level, and / or block level; and each adaptation level can offer different trade-offs between coding efficiency and coding / decoding complexity. For example, if the filter coefficients are tuned at the sequence level, the encoder needs to derive the filter coefficients for the entire video sequence, and all filter coefficients, along with the decision on whether to apply motion compensation filtering, can be done in the sequence-level parameter set, such as the video parameter set (VPS) and sequence parameter set (SPS). If the filter coefficients are tuned at the image level, the encoder needs to derive the filter coefficients for a single image, and all filter coefficients, along with the decision on whether to apply motion compensation filtering, can be done in the image-level parameter set, such as the image parameter set (PPS). If the filter coefficients are tuned at the strip level, the encoder needs to derive the filter coefficients for each individual strip, and all filter coefficients, along with the decision on whether to apply motion compensation filtering, can be carried in the strip header. Furthermore, since the motivation of this disclosure is to recover high-frequency information in the motion-compensated chroma signal, the proposed filtering method may only be beneficial to regions with rich edge and texture information. With this in mind, a region-based adaptive filter coefficient method can also be applied, where the motion-compensated filter is signaled for different regions and applied only to regions containing rich high-frequency details. In this way, the high-pass filter will not be applied to predicted samples in flat regions, which reduces encoding / decoding complexity. Whether a region is flat can be determined by the encoder / decoder based on motion-compensated luminance samples or reconstructed luminance samples.

[0196] Using CC-MCP to handle unidirectional and bidirectional forecasts

[0197] In modern video coding and decoding standards, there are two main prediction types used for motion-compensated prediction: one-way prediction and two-way prediction. For one-way prediction, a single prediction is applied, where each block can be predicted using at most one motion-compensated block from a single reference image; for two-way prediction, a two-way prediction is applied, where a block can be predicted by averaging two motion-compensated blocks from two reference images. All the CC-MCP schemes described above are discussed based on the assumption that the prediction signal for the current video block to be encoded comes from a single prediction direction (i.e., one-way prediction). For two-way prediction blocks, the proposed motion-compensated filtering scheme can be applied in different ways.

[0198] In the first approach, a CC-MCP filter is proposed to directly enhance the output chroma prediction samples by applying it only once. Specifically, in this method, the encoder / decoder first generates motion-compensated predictions for the coded video by averaging two prediction signals from two reference images; then, the proposed CC-MCP is applied to enhance the quality of the resulting chroma prediction signal.

[0199] In the second method, two CC-MCP filtering processes are applied to enhance the motion-compensated prediction signals from the two reference images, respectively. Specifically, for bidirectional prediction blocks, the method first generates two prediction blocks from two lists of reference images; then, CC-MCP is applied to enhance the quality of the two prediction blocks respectively, and the two prediction blocks are finally averaged to generate the output prediction signal.

[0200] Adaptive filtering of motion compensation signal of luminance component

[0201] Using the Motion Compensation Component (MCP) of the luminance component, the video signal in the current image is predicted from the reconstructed signal of a time-adjacent reference image, which may contain unexpected noise and quantization errors. Therefore, some high-frequency information present in the original luminance signal may be lost or distorted in the reconstructed image. To compensate for this distortion, a high-pass filter is proposed for the luminance component of the motion-compensated signal to improve its quality. It is assumed that the proposed filter for the motion-compensated luminance signal is... A 2D finite impulse response (FIR) filter composed of taps, and the filter coefficients are... It means that, among them The proposed high-pass filtering process for motion-compensated luminance signals can be calculated using 2D convolution as follows.

[0202]

[0203] in and These are the motion-compensated luminance signals before and after the filtering process; and This represents the convolution operation.

[0204] Similar to (10), additional offset and nonlinear terms can be introduced into the proposed adaptive brightness MCP filtering process, as shown below.

[0205]

[0206] in It's an offset, it's... The nonlinear term is represented by the brightness sample. A series of power (i.e.) k = 2, …, K-1 ) and.

[0207] There may be different approaches to deriving the coefficients of the proposed luma MCP filter. One approach proposes deriving the coefficients of the proposed luma MCP filter at the decoder from neighboring reconstructed luma samples of the current CU. Specifically, when applying this approach, it computes the coefficients of the luma MCP filter from neighboring reconstructed luma samples of the current block. Given a block... and its predefined adjacent regions (For example, (From the reconstructed luminance samples in the image), we can use the encoding MV of the current block to find the corresponding luminance prediction sample. Then, the LMMSE method can be used to minimize and The difference between them is used to derive the filter coefficients, as shown in the following formula.

[0208]

[0209] In addition, when considering the offset and nonlinear terms in (14), the proposed decoder-side filter coefficient derivation of the luminance MCP filter can be formulated as follows.

[0210]

[0211] In another approach, it is suggested to directly derive the luma MCP filter at the encoder and transmit the corresponding filter coefficients to the decoder via a signal. Based on the fact that the additional complexity of deriving the MCP filter coefficients at the decoder can now be eliminated, this approach reduces the decoding complexity of the proposed luma MCP filter. Furthermore, when applying this signaling-based approach, the adaptation of the luma MCP filter can be performed at different coding levels, such as VPS, SPS, PPS, image header / strip header, or even the code block level.

[0212] Figure 13A computing environment (or computing device) 1310 coupled to a user interface 1350 is shown. The computing environment 1310 may be part of a data processing server. In some embodiments, according to various examples of this disclosure, the computing device 1310 may perform any of the various methods or processes described above (e.g., encoding / decoding methods or processes). The computing environment 1310 includes a processor 1320, a memory 1330, and an I / O interface 1340.

[0213] Processor 1320 typically controls the overall operation of computing environment 1310, such as operations associated with display, data acquisition, data communication, and image processing. Processor 1320 may include one or more processors to execute instructions to perform all or some of the steps described above. Furthermore, processor 1320 may include one or more modules that facilitate interaction between processor 1320 and other components. The processor may be a central processing unit (CPU), microprocessor, microcontroller, GPU, etc.

[0214] Memory 1330 is configured to store various types of data to support the operation of computing environment 1310. Memory 1330 may include predefined software 1342. Examples of such data include instructions for any application or method operating on computing environment 1310, video datasets, image data, etc. Memory 1330 can be implemented using any type of volatile or non-volatile memory device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0215] I / O interface 1340 provides an interface between processor 1320 and peripheral interface modules (such as keyboard, click wheel, buttons, etc.). Buttons may include, but are not limited to, a home button, a start scan button, and a stop scan button. I / O interface 1340 can be coupled to encoders and decoders.

[0216] In some embodiments, a non-transitory computer-readable storage medium is also provided, including, for example, a plurality of programs in memory 1330 that can be executed by processor 1320 in computing environment 1310 to implement the methods described above, and / or storing a bitstream generated by the encoding method described above or a bitstream to be decoded by the decoding method described above. In one example, the plurality of programs can be executed by processor 1320 in computing environment 1310 to (e.g., from...) Figure 1GThe video encoder 20 in the computing environment 1310 receives a bitstream or data stream including encoded video information (e.g., video blocks representing encoded video frames, and / or one or more associated syntax elements, etc.), and can also be executed by the processor 1320 in the computing environment 1310 to perform the above-described decoding method based on the received bitstream or data stream. In another example, the plurality of programs can be executed by the processor 1320 in the computing environment 1310 to perform the above-described encoding method to encode video information (e.g., video blocks representing video frames, and / or one or more associated syntax elements, etc.) into a bitstream or data stream, and can also be executed by the processor 1320 in the computing environment 1310 to (e.g., to...) Figure 2B The video decoder 30 in the middle sends the bitstream or data stream. Alternatively, a non-transitory computer-readable storage medium may store data generated by the encoder (e.g., Figure 1G The video encoder 20 in the video is generated using, for example, the encoding method described above, for use by the decoder (e.g., Figure 2B The video decoder 30 in the video decoder uses a bitstream or data stream that includes encoded video information (e.g., video blocks representing encoded video frames, and / or one or more associated syntax elements, etc.) when decoding video data. The non-transitory computer-readable storage medium can be, for example, ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, etc.

[0217] In one embodiment, a bitstream generated by the above-described encoding method or a bitstream to be decoded by the above-described decoding method is provided. In another embodiment, a bitstream comprising encoded video information generated by the above-described encoding method or encoded video information to be decoded by the above-described decoding method is provided.

[0218] In an embodiment, a computing device is also provided, comprising: one or more processors (e.g., processor 1320); and a non-transitory computer-readable storage medium or memory 1330 therein storing a plurality of programs executable by one or more processors, wherein one or more processors are configured to perform the methods described above when executing the plurality of programs.

[0219] In one embodiment, a computer program product having instructions for storing or transmitting a bitstream is also provided, the bitstream including encoded video information generated by the encoding method described above or encoded video information to be decoded by the decoding method described above. In another embodiment, a computer program product including, for example, a plurality of programs in a memory 1330 is also provided, the plurality of programs being executable by a processor 1320 in a computing environment 1310 to perform the methods described above. For example, the computer program product may include a non-transitory computer-readable storage medium.

[0220] In an embodiment, the computing environment 1310 may be implemented by one or more ASICs, DSPs, digital signal processing devices (DSPDs), programmable logic devices (PLDs), FPGAs, GPUs, controllers, microcontrollers, microprocessors, or other electronic components for performing the methods described above.

[0221] In one embodiment, a method for storing a bitstream is also provided, comprising: storing the bitstream on a digital storage medium, wherein the bitstream includes encoded video information generated by the above-described encoding method or encoded video information to be decoded by the above-described decoding method.

[0222] In one embodiment, a method for transmitting a bitstream generated by the encoder described above is also provided. In another embodiment, a method for receiving a bitstream to be decoded by the decoder described above is also provided.

[0223] Figure 14 This is a flowchart illustrating an example of a video decoding method according to this disclosure. This method can be implemented for decoding inter-frame coded blocks.

[0224] In step 1401, on the decoder side, processor 1320 can acquire multiple reconstructed luma samples of the current inter-frame coded block. In step 1402, processor 1320 can acquire an adaptive cross-component filter. In step 1403, processor 1320 can acquire filtered motion-compensated chroma samples based on the adaptive cross-component filter and the multiple reconstructed luma samples.

[0225] In some examples, obtaining the adaptive cross-component filter includes deriving the adaptive cross-component filter based on adjacent reconstructed luminance samples and adjacent reconstructed chrominance samples.

[0226] In some examples, deriving an adaptive cross-component filter based on neighboring reconstructed luminance samples and neighboring reconstructed chrominance samples includes: obtaining output neighboring chrominance samples based on the adaptive cross-component filter and neighboring reconstructed luminance samples; and deriving one or more filter coefficients of the adaptive cross-component filter by minimizing the difference between the output neighboring chrominance samples and neighboring reconstructed chrominance samples.

[0227] In some examples, the one or more filter coefficients include linear filter coefficients, offset terms, or nonlinear terms.

[0228] In some examples, deriving an adaptive cross-component filter based on neighboring reconstructed luminance samples and neighboring reconstructed chrominance samples includes: in response to determining that the reference image of the current inter-frame coding block has a smaller quantization parameter (QP) value than the current image of the current inter-frame coding block, deriving the adaptive cross-component filter based on neighboring reconstructed luminance samples and neighboring reconstructed chrominance samples.

[0229] In some examples, processor 1320 may further acquire motion-compensated chromaticity samples; wherein acquiring filtered motion-compensated chromaticity samples based on an adaptive cross-component filter and multiple reconstructed luminance samples includes: acquiring filtered motion-compensated chromaticity samples based on an adaptive cross-component filter, multiple reconstructed luminance samples, and motion-compensated chromaticity samples.

[0230] In some instances, obtaining the adaptive cross-component filter involves deriving the adaptive cross-component filter based on adjacent reconstructed luminance samples, adjacent reconstructed chrominance samples, and motion information of the current inter-frame coded block.

[0231] In some examples, deriving the adaptive cross-component filter based on adjacent reconstructed luminance samples, the adjacent reconstructed chrominance samples, and the motion information of the current inter-frame coding block includes: obtaining multiple motion-compensated adjacent chrominance samples of the adjacent reconstructed chrominance samples based on the motion information of the current inter-frame coding block; obtaining output adjacent chrominance samples based on the adaptive cross-component filter, the adjacent reconstructed luminance samples, and the multiple motion-compensated adjacent chrominance samples; and deriving one or more filter coefficients of the adaptive cross-component filter by minimizing the difference between the output adjacent chrominance samples and the adjacent reconstructed chrominance samples.

[0232] In some examples, obtaining output adjacent chromaticity samples based on the adaptive cross-component filter, the adjacent reconstructed luminance samples, and the plurality of motion-compensated adjacent chromaticity samples includes: obtaining chromaticity refinement results by applying the adaptive cross-component filter to the adjacent reconstructed luminance samples; and obtaining the output adjacent chromaticity samples based on the chromaticity refinement results and the motion-compensated adjacent chromaticity samples.

[0233] In some examples, obtaining the filtered motion-compensated chromaticity sample based on the adaptive cross-component filter, the plurality of reconstructed luminance samples, and the motion-compensated chromaticity sample includes: obtaining a chromaticity refinement result by applying the adaptive cross-component filter to the adjacent reconstructed luminance samples; and obtaining the filtered motion-compensated chromaticity sample based on the chromaticity refinement result and the motion-compensated chromaticity sample.

[0234] In some examples, the one or more filter coefficients include linear filter coefficients, offset terms, or nonlinear terms.

[0235] In some examples, deriving the adaptive cross-component filter based on adjacent reconstructed luminance samples, adjacent reconstructed chrominance samples, and motion information of the current inter-frame coding block includes: in response to determining that the reference image of the current inter-frame coding block has a smaller quantization parameter (QP) value than the current image of the current inter-frame coding block, deriving the adaptive cross-component filter based on adjacent reconstructed luminance samples, adjacent reconstructed chrominance samples, and motion information of the current inter-frame coding block.

[0236] In some examples, processor 1320 may also receive one or more filter coefficients of the adaptive cross-component filter transmitted by an encoder, wherein the one or more filter coefficients are transmitted by signal at a specific level.

[0237] In some examples, obtaining the plurality of reconstructed luminance samples of the current inter-frame coding block includes: obtaining a plurality of motion-compensated luminance samples and a plurality of residual luminance samples of the current inter-frame coding block; and obtaining the plurality of reconstructed luminance samples based on the plurality of motion-compensated luminance samples and the plurality of residual luminance samples.

[0238] Figure 15 It shows the corresponding Figure 14 The flowchart shown illustrates the video encoding method within the video decoding method. This method can be implemented to encode inter-frame coded blocks.

[0239] In step 1501, on the encoder side, processor 1320 can acquire multiple reconstructed luma samples of the current inter-frame coded block. In step 1502, processor 1320 can acquire an adaptive cross-component filter. In step 1503, processor 1320 can acquire filtered motion-compensated chroma samples based on the adaptive cross-component filter and the multiple reconstructed luma samples.

[0240] In some examples, obtaining the adaptive cross-component filter includes deriving the adaptive cross-component filter based on adjacent reconstructed luminance samples and adjacent reconstructed chrominance samples.

[0241] In some examples, deriving an adaptive cross-component filter based on neighboring reconstructed luminance samples and neighboring reconstructed chrominance samples includes: obtaining output neighboring chrominance samples based on the adaptive cross-component filter and neighboring reconstructed luminance samples; and deriving one or more filter coefficients of the adaptive cross-component filter by minimizing the difference between the output neighboring chrominance samples and neighboring reconstructed chrominance samples.

[0242] In some examples, the one or more filter coefficients include linear filter coefficients, offset terms, or nonlinear terms.

[0243] In some examples, deriving an adaptive cross-component filter based on neighboring reconstructed luminance samples and neighboring reconstructed chrominance samples includes: in response to determining that the reference image of the current inter-frame coding block has a smaller quantization parameter (QP) value than the current image of the current inter-frame coding block, deriving the adaptive cross-component filter based on neighboring reconstructed luminance samples and neighboring reconstructed chrominance samples.

[0244] In some examples, processor 1320 may further acquire motion-compensated chromaticity samples; wherein acquiring filtered motion-compensated chromaticity samples based on an adaptive cross-component filter and multiple reconstructed luminance samples includes: acquiring filtered motion-compensated chromaticity samples based on an adaptive cross-component filter, multiple reconstructed luminance samples, and motion-compensated chromaticity samples.

[0245] In some instances, obtaining the adaptive cross-component filter involves deriving the adaptive cross-component filter based on adjacent reconstructed luminance samples, adjacent reconstructed chrominance samples, and motion information of the current inter-frame coded block.

[0246] In some examples, deriving the adaptive cross-component filter based on adjacent reconstructed luminance samples, the adjacent reconstructed chrominance samples, and the motion information of the current inter-frame coding block includes: obtaining multiple motion-compensated adjacent chrominance samples of the adjacent reconstructed chrominance samples based on the motion information of the current inter-frame coding block; obtaining output adjacent chrominance samples based on the adaptive cross-component filter, the adjacent reconstructed luminance samples, and the multiple motion-compensated adjacent chrominance samples; and deriving one or more filter coefficients of the adaptive cross-component filter by minimizing the difference between the output adjacent chrominance samples and the adjacent reconstructed chrominance samples.

[0247] In some examples, obtaining output adjacent chromaticity samples based on the adaptive cross-component filter, the adjacent reconstructed luminance samples, and the plurality of motion-compensated adjacent chromaticity samples includes: obtaining chromaticity refinement results by applying the adaptive cross-component filter to the adjacent reconstructed luminance samples; and obtaining the output adjacent chromaticity samples based on the chromaticity refinement results and the motion-compensated adjacent chromaticity samples.

[0248] In some examples, obtaining the filtered motion-compensated chromaticity sample based on the adaptive cross-component filter, the plurality of reconstructed luminance samples, and the motion-compensated chromaticity sample includes: obtaining a chromaticity refinement result by applying the adaptive cross-component filter to the adjacent reconstructed luminance samples; and obtaining the filtered motion-compensated chromaticity sample based on the chromaticity refinement result and the motion-compensated chromaticity sample.

[0249] In some examples, the one or more filter coefficients include linear filter coefficients, offset terms, or nonlinear terms.

[0250] In some examples, deriving the adaptive cross-component filter based on adjacent reconstructed luminance samples, adjacent reconstructed chrominance samples, and motion information of the current inter-frame coding block includes: in response to determining that the reference image of the current inter-frame coding block has a smaller quantization parameter (QP) value than the current image of the current inter-frame coding block, deriving the adaptive cross-component filter based on adjacent reconstructed luminance samples, adjacent reconstructed chrominance samples, and motion information of the current inter-frame coding block.

[0251] In some examples, processor 1320 may also signal one or more filter coefficients of an adaptive cross-component filter, wherein the one or more filter coefficients are signaled at a specific level.

[0252] In some examples, obtaining the plurality of reconstructed luminance samples of the current inter-frame coding block includes: obtaining a plurality of motion-compensated luminance samples and a plurality of residual luminance samples of the current inter-frame coding block; and obtaining the plurality of reconstructed luminance samples based on the plurality of motion-compensated luminance samples and the plurality of residual luminance samples.

[0253] In some examples, a video encoding apparatus is provided. This apparatus includes a processor 1320 and a memory 1330, the memory 1330 being configured to store instructions executable by the processor; wherein the processor is configured to execute instructions such as... Figures 14 to 15 Any of the methods shown.

[0254] In some other examples, a non-transitory computer-readable storage medium is provided, in which instructions are stored. When processor 1320 executes the instructions, the instructions cause the processor to perform actions such as Figures 12 to 15 Any of the methods shown. In one example, multiple programs can be executed by processor 1320 in computing environment 1310 to receive (e.g., from) Figure 1G The video encoder 20 in the computing environment 1310 includes a bitstream or data stream that encodes video information (e.g., video blocks representing encoded video frames and / or one or more associated syntax elements, etc.) and can also be executed by the processor 1320 in the computing environment 1310 to perform the above-described decoding method based on the received bitstream or data stream. In another example, multiple programs can be executed by the processor 1320 in the computing environment 1310 to perform the above-described encoding method, thereby encoding video information (e.g., video blocks representing video frames and / or one or more associated syntax elements, etc.) into a bitstream or data stream, and can also be executed by the processor 1320 in the computing environment 1310 to transmit the bitstream or data stream (e.g., transmit to...). Figure 2B The video decoder 30 in the image (or a non-transitory computer-readable storage medium may store a video decoder (e.g., a video decoder 30) in the image (e.g., a video decoder 30). Alternatively, a non-transitory computer-readable storage medium may store a video decoder (e.g., a video decoder 30) in the image ...). Figure 1GThe video encoder 20 in the video encoder uses, for example, the decoder described above (e.g., Figure 2B The video decoder 30 in the video decoder is a bitstream or data stream of encoded video information (e.g., video blocks representing video frames and / or one or more associated syntax elements) generated by the encoding method used when decoding video data. Non-transitory computer-readable storage media can be, for example, ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, etc.

[0255] The description in this disclosure has been presented for illustrative purposes and is not intended to be exhaustive or limited to this disclosure. Many modifications, variations, and alternative embodiments will be apparent to those skilled in the art from the teachings presented in the foregoing description and the associated drawings.

[0256] Unless otherwise specifically stated, the order of steps in the method according to this disclosure is intended to be illustrative only, and the steps of the method according to this disclosure are not limited to the specific order described above, but may be changed according to actual circumstances. Furthermore, at least one step in the method according to this disclosure may be adjusted, combined, or omitted as needed.

[0257] The examples chosen and described are intended to explain the principles of this disclosure and to enable others skilled in the art to understand the various embodiments of this disclosure, and preferably to utilize the basic principles and various embodiments with various modifications suitable for the intended particular purpose. Therefore, it will be understood that the scope of this disclosure is not limited to the specific examples of the disclosed embodiments, and that modifications and other embodiments are intended to be included within the scope of this disclosure.

Claims

1. A method for decoding inter-frame coded blocks, comprising: The decoder obtains multiple reconstructed luminance samples of the current inter-frame coded block; The adaptive cross-component filter is obtained through the decoder; as well as The decoder obtains filtered motion-compensated chromaticity samples based on the adaptive cross-component filter and the plurality of reconstructed luminance samples.

2. The method according to claim 1, wherein obtaining the adaptive cross-component filter comprises: The adaptive cross-component filter is derived based on adjacent reconstructed luminance samples and adjacent reconstructed chrominance samples.

3. The method of claim 2, wherein deriving the adaptive cross-component filter based on the adjacent reconstructed luminance samples and the adjacent reconstructed chrominance samples comprises: Based on the adaptive cross-component filter and the adjacent reconstructed luminance samples, output adjacent chrominance samples are obtained; and One or more filter coefficients of the adaptive cross-component filter are derived by minimizing the difference between the output neighboring chromaticity samples and the neighboring reconstructed chromaticity samples.

4. The method of claim 3, wherein the one or more filter coefficients include linear filter coefficients, offset terms, or nonlinear terms.

5. The method of claim 2, wherein deriving the adaptive cross-component filter based on adjacent reconstructed luminance samples and adjacent reconstructed chrominance samples comprises: In response to determining that the reference image of the current inter-frame coding block has a smaller quantization parameter (QP) value than the current image of the current inter-frame coding block, the adaptive cross-component filter is derived based on adjacent reconstructed luminance samples and adjacent reconstructed chrominance samples.

6. The method of claim 1, further comprising: The decoder obtains motion-compensated chromaticity samples; The process of obtaining the filtered motion-compensated chromaticity sample based on the adaptive cross-component filter and the plurality of reconstructed luminance samples includes: The filtered motion-compensated chromaticity sample is obtained based on the adaptive cross-component filter, the plurality of reconstructed luminance samples, and the motion-compensated chromaticity sample.

7. The method of claim 6, wherein obtaining the adaptive cross-component filter comprises: The adaptive cross-component filter is derived based on adjacent reconstructed luminance samples, adjacent reconstructed chrominance samples, and the motion information of the current inter-frame coding block.

8. The method of claim 7, wherein deriving the adaptive cross-component filter based on the adjacent reconstructed luminance samples, the adjacent reconstructed chrominance samples, and the motion information of the current inter-frame coding block comprises: Based on the motion information of the current inter-frame coding block, obtain multiple motion-compensated adjacent chroma samples of the adjacent reconstructed chroma samples; Based on the adaptive cross-component filter, the adjacent reconstructed luminance samples, and the multiple motion-compensated adjacent chrominance samples, the output adjacent chrominance samples are obtained. and One or more filter coefficients of the adaptive cross-component filter are derived by minimizing the difference between the output neighboring chromaticity samples and the neighboring reconstructed chromaticity samples.

9. The method according to claim 8, wherein obtaining the output adjacent chromaticity samples based on the adaptive cross-component filter, the adjacent reconstructed luminance samples, and the plurality of motion-compensated adjacent chromaticity samples comprises: The chromaticity refinement result is obtained by applying the adaptive cross-component filter to the adjacent reconstructed luminance samples; and The output adjacent chromaticity samples are obtained based on the chromaticity refinement results and motion-compensated adjacent chromaticity samples.

10. The method of claim 8, wherein obtaining the filtered motion-compensated chromaticity sample based on the adaptive cross-component filter, the plurality of reconstructed luminance samples, and the motion-compensated chromaticity sample comprises: The chromaticity refinement result is obtained by applying the adaptive cross-component filter to the multiple reconstructed luminance samples; and The filtered motion-compensated chromaticity sample is obtained based on the chromaticity refinement result and the motion-compensated chromaticity sample.

11. The method of claim 8, wherein the one or more filter coefficients comprise linear filter coefficients, offset terms, or nonlinear terms.

12. The method of claim 7, wherein deriving the adaptive cross-component filter based on adjacent reconstructed luminance samples, adjacent reconstructed chrominance samples, and motion information of the current inter-frame coding block comprises: In response to determining that the reference image of the current inter-frame coding block has a smaller quantization parameter (QP) value than the current image of the current inter-frame coding block, the adaptive cross-component filter is derived based on adjacent reconstructed luminance samples, adjacent reconstructed chrominance samples, and motion information of the current inter-frame coding block.

13. The method of claim 1, further comprising: The decoder receives one or more filter coefficients of the adaptive cross-component filter transmitted by the encoder via signals, wherein the one or more filter coefficients are transmitted via signals at a specific level.

14. The method of claim 1, wherein obtaining the plurality of reconstructed luminance samples of the current inter-frame coded block comprises: Obtain multiple motion-compensated luminance samples and multiple residual luminance samples of the current inter-frame coded block; and The multiple reconstructed brightness samples are obtained based on the multiple motion-compensated brightness samples and the multiple residual brightness samples.

15. A method for decoding inter-frame coded blocks, comprising: The encoder obtains multiple reconstructed luminance samples of the current inter-frame coding block; An adaptive cross-component filter is obtained through the encoder; as well as The encoder obtains filtered motion-compensated chromaticity samples based on the adaptive cross-component filter and the plurality of reconstructed luminance samples.

16. The method of claim 15, wherein obtaining the adaptive cross-component filter comprises: The adaptive cross-component filter is derived based on adjacent reconstructed luminance samples and adjacent reconstructed chrominance samples.

17. The method of claim 16, wherein deriving the adaptive cross-component filter based on the adjacent reconstructed luminance samples and the adjacent reconstructed chrominance samples comprises: Based on the adaptive cross-component filter and the adjacent reconstructed luminance samples, output adjacent chrominance samples are obtained; and One or more filter coefficients of the adaptive cross-component filter are derived by minimizing the difference between the output neighboring chromaticity samples and the neighboring reconstructed chromaticity samples.

18. The method of claim 17, wherein the one or more filter coefficients comprise linear filter coefficients, offset terms, or nonlinear terms.

19. The method of claim 16, wherein deriving the adaptive cross-component filter based on adjacent reconstructed luminance samples and adjacent reconstructed chrominance samples comprises: In response to determining that the reference image of the current inter-frame coding block has a smaller quantization parameter (QP) value than the current image of the current inter-frame coding block, the adaptive cross-component filter is derived based on adjacent reconstructed luminance samples and adjacent reconstructed chrominance samples.

20. The method of claim 15, further comprising: The encoder obtains motion-compensated chromaticity samples; The process of obtaining the filtered motion-compensated chromaticity sample based on the adaptive cross-component filter and the plurality of reconstructed luminance samples includes: The filtered motion-compensated chromaticity sample is obtained based on the adaptive cross-component filter, the plurality of reconstructed luminance samples, and the motion-compensated chromaticity sample.

21. The method of claim 20, wherein obtaining the adaptive cross-component filter comprises: The adaptive cross-component filter is derived based on adjacent reconstructed luminance samples, adjacent reconstructed chrominance samples, and the motion information of the current inter-frame coding block.

22. The method of claim 21, wherein deriving the adaptive cross-component filter based on adjacent reconstructed luminance samples, the adjacent reconstructed chrominance samples, and the motion information of the current inter-frame coding block comprises: Based on the motion information of the current inter-frame coding block, obtain multiple motion-compensated adjacent chroma samples of the adjacent reconstructed chroma samples; Based on the adaptive cross-component filter, the adjacent reconstructed luminance samples, and the multiple motion-compensated adjacent chrominance samples, the output adjacent chrominance samples are obtained. and One or more filter coefficients of the adaptive cross-component filter are derived by minimizing the difference between the output neighboring chromaticity samples and the neighboring reconstructed chromaticity samples.

23. The method of claim 22, wherein obtaining the output adjacent chromaticity samples based on the adaptive cross-component filter, the adjacent reconstructed luminance samples, and the plurality of motion-compensated adjacent chromaticity samples comprises: The chromaticity refinement result is obtained by applying the adaptive cross-component filter to the adjacent reconstructed luminance samples; and The output adjacent chromaticity samples are obtained based on the chromaticity refinement results and motion-compensated adjacent chromaticity samples.

24. The method of claim 22, wherein obtaining the filtered motion-compensated chromaticity sample based on the adaptive cross-component filter, the plurality of reconstructed luminance samples, and the motion-compensated chromaticity sample comprises: The chromaticity refinement result is obtained by applying the adaptive cross-component filter to the multiple reconstructed luminance samples; and The filtered motion-compensated chromaticity sample is obtained based on the chromaticity refinement result and the motion-compensated chromaticity sample.

25. The method of claim 22, wherein the one or more filter coefficients include linear filter coefficients, offset terms, or nonlinear terms.

26. The method of claim 21, wherein deriving the adaptive cross-component filter based on adjacent reconstructed luminance samples, adjacent reconstructed chrominance samples, and motion information of the current inter-frame coding block comprises: In response to determining that the reference image of the current inter-frame coding block has a smaller quantization parameter (QP) value than the current image of the current inter-frame coding block, the adaptive cross-component filter is derived based on adjacent reconstructed luminance samples, adjacent reconstructed chrominance samples, and motion information of the current inter-frame coding block.

27. The method of claim 15, further comprising: The encoder transmits one or more filter coefficients of the adaptive cross-component filter by signal, wherein the one or more filter coefficients are transmitted by signal at a specific level.

28. The method of claim 15, wherein obtaining the plurality of reconstructed luminance samples of the current inter-frame coded block comprises: Obtain multiple motion-compensated luminance samples and multiple residual luminance samples of the current inter-frame coded block; and The multiple reconstructed brightness samples are obtained based on the multiple motion-compensated brightness samples and the multiple residual brightness samples.

29. A method for receiving a bitstream, wherein the bitstream includes encoded video information to be decoded by the method of any one of claims 1 to 14.

30. A method for transmitting a bit stream, wherein the bit stream comprises encoded video information generated by the method of claims 15 to 28.

31. A video decoding apparatus, comprising: One or more processors; as well as A memory coupled to the one or more processors and configured to store instructions executable by the one or more processors. The one or more processors wherein, when executing the instructions, are configured to perform the method according to any one of claims 1 to 14.

32. A non-transitory computer-readable storage medium for storing computer-executable instructions that, when executed by one or more computer processors, cause the one or more computer processors to perform the method according to any one of claims 1 to 14.

33. A video encoding apparatus, comprising: One or more processors; as well as A memory coupled to the one or more processors and configured to store instructions executable by the one or more processors. The one or more processors wherein, when executing the instructions, are configured to perform the method according to any one of claims 15 to 28.

34. A non-transitory computer-readable storage medium for storing computer-executable instructions that, when executed by one or more computer processors, cause the one or more computer processors to perform the method according to any one of claims 15 to 28.

35. A non-transitory computer-readable storage medium for storing a bit stream to be decoded by the method according to any one of claims 1 to 14.

36. A non-transitory computer-readable storage medium for storing a bit stream generated by the method according to any one of claims 15 to 28.

Images