Method and apparatus for filtered intra block copy

Abstract

The present disclosure provides a method for video decoding, comprising: determining an initial prediction block of a current block to be predicted based on a first block vector, where the first block vector is a fractional block vector between the current block and a reference block; determining a reference template based on a current template associated with the current block and the second block vector; obtaining a set of filter coefficients corresponding to a filter shape based at least on sample values from both the current template and the reference template; and deriving a final prediction block of the current block including the prediction sample value of the current block by applying the filter coefficient set and the filter shape to the initial prediction block.

Description

Cross-references to related applications

[0001] This application is based on and claims priority to provisional application No. 63 / 506,871, filed on June 8, 2023, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to video encoding / decoding and compression. More specifically, this application relates to methods and apparatus for improving the encoding / decoding efficiency of filtered intra-block copy (FIBC). Background Technology

[0003] Various electronic devices support digital video, such as digital televisions, laptops or desktop computers, tablets, digital cameras, digital recording devices, digital media players, video game consoles, smartphones, video conferencing equipment, and video streaming devices. Electronic devices transmit and receive, or otherwise transmit, digital video data across communication networks, and / or store digital video data on storage devices. Due to the limited bandwidth capacity of communication networks and the limited storage resources of storage devices, video data can be compressed using one or more video codec standards before being transmitted or stored. Examples of video codec standards include Universal Video Codec (VVC), Joint Explore Test Model (JEM), High Efficiency Video Codec (HEVC / H.265), Advanced Video Codec (AVC / H.264), and Moving Picture Experts Group (MPEG) codecs. Video codecs typically utilize predictive methods (e.g., inter-frame prediction, intra-frame prediction, etc.) that take advantage of the inherent redundancy in the video data. Video codecs aim to compress video data into a form using a lower bitrate while avoiding or minimizing degradation in video quality. Summary of the Invention

[0004] Embodiments of this disclosure provide methods and apparatus for improving the encoding and decoding efficiency of image / video blocks using FIBC technology.

[0005] According to one aspect of this disclosure, a method for video decoding is provided, comprising: determining an initial prediction block for a current block to be predicted based on a first block vector, wherein the first block vector is a fractional block vector between the current block and a reference block; determining a reference template based on a current template and a second block vector associated with the current block; obtaining a set of filter coefficients corresponding to a filter shape based at least on sample values ​​from both the current template and the reference template; and deriving a final prediction block of the current block, including the predicted sample values ​​of the current block, by applying the set of filter coefficients and the filter shape to the initial prediction block.

[0006] According to one aspect of this disclosure, a method for video coding is provided, comprising: determining an initial prediction block for a current block to be predicted based on a first block vector, wherein the first block vector is a fractional block vector between the current block and a reference block; determining a reference template based on a current template and a second block vector associated with the current block; obtaining a set of filter coefficients corresponding to a filter shape based at least on sample values ​​from both the current template and the reference template; deriving a final prediction block of the current block, including the predicted sample values ​​of the current block, by applying the set of filter coefficients and the filter shape to the initial prediction block; and generating a bitstream based on the final prediction block.

[0007] According to one aspect of this disclosure, an apparatus for video encoding is provided, comprising: one or more processors; and one or more storage devices storing computer-executable instructions, which, when executed, cause the one or more processors to perform the operations of the methods of this disclosure.

[0008] According to one aspect of this disclosure, a computer program product having instructions for storing a bitstream is provided, wherein the bitstream comprises video data decoded by the video decoding method of this disclosure; or the bitstream comprises video data generated by the video encoding method of this disclosure.

[0009] According to one aspect of this disclosure, a computer-readable storage medium is provided that stores instructions, which, when executed by a computing device having one or more processors, cause the one or more processors to perform the methods of this disclosure and store a bitstream to be decoded by the methods of this disclosure, or to perform the methods of this disclosure and store a bitstream generated by the methods of this disclosure.

[0010] According to one aspect of this disclosure, a computer-readable medium for storing bitstreams is provided, wherein the bitstreams are decoded by performing the operations of the methods of this disclosure, or the bitstreams are obtained by performing the operations of the methods of this disclosure.

[0011] According to one aspect of this disclosure, a method for receiving a bitstream to be decoded by the method of this disclosure is provided.

[0012] According to one aspect of this disclosure, a method for transmitting a bit stream generated by the method of this disclosure is provided.

[0013] It will be understood that the above general description and the following detailed description are merely examples and do not limit the content of this disclosure. Attached Figure Description

[0014] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate examples consistent with this disclosure and, together with this description, serve to explain the principles of this disclosure.

[0015] Figure 1 This is a block diagram illustrating an exemplary system for encoding and decoding video blocks according to some implementations of the present disclosure.

[0016] Figure 2 This is a block diagram illustrating an exemplary video encoder according to some implementations of this disclosure.

[0017] Figure 3 This is a block diagram illustrating an exemplary video decoder according to some implementations of this disclosure.

[0018] Figure 4A , Figure 4B , Figure 4C , Figure 4D and Figure 4E This is a block diagram illustrating how, according to some implementations of this disclosure, a frame is recursively divided into multiple video blocks of different sizes and shapes.

[0019] Figure 5 A schematic diagram of the spatial candidate locations is shown.

[0020] Figure 6 A schematic diagram of candidate pairs considered for redundancy checking of spatial candidates is shown.

[0021] Figure 7 A schematic diagram of scaling the motion vector for a time candidate is shown.

[0022] Figure 8 A schematic diagram of the candidate positions for time-based candidates is shown.

[0023] Figure 9 A schematic diagram of the merging mode (MMVD) search for points using motion vector difference is shown.

[0024] Figure 10 The selection of unidirectional predictive motion vectors for the Geometric Partitioning Mode (GPM) is shown.

[0025] Figure 11 The top and left adjacent blocks used in CIIP weight export are shown.

[0026] Figure 12 The current CTU processing order and its reference samples available in the current and left CTUs are shown.

[0027] Figure 13 The filling candidates for replacing the zero vector in the IBC list are shown.

[0028] Figure 14 The reference region used for IBC is shown when CTU(m,n) is encoded.

[0029] Figure 15 The IBC reference area of ​​the content captured by the camera is shown.

[0030] Figures 16A to 16B The method for dividing the angle pattern is shown.

[0031] Figure 17A , Figure 17B and Figure 17C Available IPM candidates and Figure 17D An example of GPM with intra-frame and intra-frame prediction is shown.

[0032] Figure 18 The edges on the template are shown.

[0033] Figure 19 The intra-frame template matching search area used is shown.

[0034] Figure 20 A template for OBMC based on template matching is shown.

[0035] Figure 21 The template and reference sample points of the template are shown in the reference image.

[0036] Figure 22 The template of the block with sub-block motion information using the current block and the reference sample points of the template are shown.

[0037] Figure 23 The luminance block used to derive the direct block vector is shown.

[0038] Figure 24 The method for partitioning intra-coded blocks and the corresponding weights for angular and planar modes are shown.

[0039] Figure 25 A schematic diagram of the filter shape of the reference block and the training region is shown.

[0040] Figure 26 Examples of predictions for different positions within the current block are shown.

[0041] Figure 27 A schematic diagram of the spatial terms corresponding to adjacent brightness samples is shown.

[0042] Figure 28 A schematic diagram showing examples of different shapes / numbers of filter taps is provided.

[0043] Figure 29A schematic diagram showing examples of different shapes / numbers of filter taps is provided.

[0044] Figure 30 A schematic diagram showing examples of different shapes / numbers of filter taps is provided.

[0045] Figure 31 A schematic diagram showing the possible locations of the candidate regions is provided.

[0046] Figure 32 A schematic diagram of the possible candidate locations is shown.

[0047] Figure 33 A schematic diagram of the filter shape of the reference block and the training region is shown.

[0048] Figure 34 The workflow of a method for video decoding according to one or more aspects of this disclosure is shown.

[0049] Figure 35 The workflow of a method for video encoding according to one or more aspects of this disclosure is shown.

[0050] Figure 36 This is a schematic diagram illustrating a computing environment coupled with a user interface according to some implementations of the present disclosure. Detailed Implementation

[0051] Referring now to specific implementations, examples of which are illustrated in the accompanying drawings. Numerous non-limiting details are set forth in the following detailed description to aid in understanding the subject matter presented herein. However, various alternatives may be used without departing from the scope of the claims, and the subject matter may be practiced without these specific details. For example, the subject matter presented herein can be implemented on many types of electronic devices with digital video capabilities.

[0052] It should be noted that the terms "first," "second," etc., used in the description, claims, and drawings of this disclosure are used to distinguish objects and not to describe any specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of this disclosure described herein can be implemented in orders other than those shown in the drawings or described in this disclosure.

[0053] Figure 1 This is a block diagram illustrating an exemplary system 10 for encoding and decoding video blocks in parallel, according to some implementations of the present disclosure. Figure 1As shown, system 10 includes source device 12, which generates and encodes video data that will later be decoded by target device 14. Source device 12 and target device 14 can include any electronic device from a wide variety of electronic devices, including cloud servers, server computers, desktop or laptop computers, tablet computers, smartphones, set-top boxes, digital televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, etc. In some implementations, source device 12 and target device 14 are equipped with wireless communication capabilities.

[0054] In some implementations, target device 14 may receive encoded video data to be decoded via link 16. Link 16 may include any type of communication medium or device capable of moving encoded video data from source device 12 to target device 14. In one example, link 16 may include a communication medium enabling source device 12 to transmit encoded video data directly to target device 14 in real time. The encoded video data may be modulated according to a communication standard (e.g., a wireless communication protocol) and transmitted to target device 14. The communication medium may include any wireless or wired communication medium, such as radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium may form part of a packet-based network (e.g., a local area network, a wide area network, or a global network such as the Internet). The communication medium may include a router, switch, base station, or any other means that may facilitate communication from source device 12 to target device 14.

[0055] In some other implementations, encoded video data can be sent from output interface 22 to storage device 32. The target device 14 can then access the encoded video data in storage device 32 via input interface 28. Storage device 32 can include any data storage medium of various distributed or locally accessed data storage media, such as hard disk drives, Blu-ray discs, digital universal discs (DVDs), compact disc read-only memory (CD-ROMs), flash memory, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded video data. In another example, storage device 32 can correspond to a file server or another intermediate storage device that can hold the encoded video data generated by source device 12. Target device 14 can access the stored video data from storage device 32 via streaming or downloading. The file server can be any type of computer capable of storing and sending encoded video data to target device 14. Exemplary file servers include web servers (e.g., for websites), file transfer protocol (FTP) servers, network attached storage (NAS) devices, or local disk drives. Target device 14 can access the encoded video data via any standard data connection suitable for accessing encoded video data stored on a file server. Standard data connections include wireless channels (e.g., Wi-Fi connections), wired connections (e.g., digital subscriber line (DSL), cable modems, etc.), or a combination of both. Transfer of encoded video data from storage device 32 can be streaming, downloading, or a combination of both.

[0056] like Figure 1 As shown, source device 12 includes a video source 18, a video encoder 20, and an output interface 22. Video source 18 may include sources or combinations of such sources, such as: a video capture device (e.g., a camera), a video archive containing previously captured video, a video feed interface for receiving video from a video content provider, and / or a computer graphics system for generating computer graphics data as source video. As an example, if video source 18 is a camera in a security monitoring system, source device 12 and target device 14 may form a camera phone or video phone. However, the implementations described in this application are generally applicable to video encoding and decoding and can be applied to wireless and / or wired applications.

[0057] Captured, pre-captured, or computer-generated video can be encoded by video encoder 20. The encoded video data can be directly transmitted to target device 14 via output interface 22 of source device 12. Alternatively, the encoded video data can be stored on storage device 32 for later access by target device 14 or other devices for decoding and / or playback. Output interface 22 may further include a modem and / or transmitter.

[0058] Target device 14 includes an input interface 28, a video decoder 30, and a display device 34. Input interface 28 may include a receiver and / or a modem, and receives encoded video data via link 16. The encoded video data transmitted via link 16 or provided on storage device 32 may include various syntax elements generated by video encoder 20 for use by video decoder 30 in decoding the video data. Such syntax elements may be included within encoded video data transmitted on a communication medium, stored on a storage medium, or stored on a file server.

[0059] In some implementations, target device 14 may include display device 34, which may be an integrated display device or an external display device configured to communicate with target device 14. Display device 34 displays decoded video data to a user and may include any of a variety of display devices, such as a liquid crystal display (LCD), a plasma display, an organic light-emitting diode (OLED) display, or another type of display device.

[0060] Video encoder 20 and video decoder 30 can operate according to proprietary or industry standards (e.g., VVC, HEVC, MPEG-4 Part 10, AVC) or extensions of such standards. It should be understood that this application is not limited to a specific video encoding / decoding standard and can be applied to other video encoding / decoding standards. It is generally understood that the video encoder 20 of source device 12 can be configured to encode video data according to any of these current or future standards. Similarly, it is generally understood that the video decoder 30 of target device 14 can be configured to decode video data according to any of these current or future standards.

[0061] The video encoder 20 and video decoder 30 can be implemented as any circuit of a variety of suitable encoder and / or decoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic devices, software, hardware, firmware, or any combination thereof. When implemented in part in software, the electronic device may store instructions for the software in a suitable non-transitory computer-readable medium and use one or more processors to execute the instructions for the hardware to perform the video encoding / decoding operations disclosed in this disclosure. Each of the video encoder 20 and video decoder 30 may be included in one or more encoders or decoders, either of which may be integrated as part of a combined encoder / decoder (CODEC) in the respective device.

[0062] In some implementations, components of source device 12 (e.g., video source 18, video encoder 20, or the following references) Figure 2 The components described include at least a portion of the components in the video encoder 20 and the output interface 22, and / or the components of the target device 14 (e.g., the input interface 28, the video decoder 30, or the following references). Figure 3At least a portion of the components included in the video decoder 30 and the display device 34 described herein can operate in a cloud computing service network such as Software as a Service (SaaS), Platform as a Service (PaaS), or Infrastructure as a Service (IaaS), which can provide software, platform, and / or infrastructure. In some implementations, one or more components of the source device 12 and / or the target device 14 that are not included in the cloud computing service network can be provided in one or more client devices, and these client devices can communicate with server computers in the cloud computing service network via wireless communication networks (e.g., cellular communication networks, short-range wireless communication networks, or Global Navigation Satellite System (GNSS) communication networks) or wired communication networks (e.g., local area network (LAN) communication networks or power line communication (PLC) networks). In one embodiment, at least a portion of the operations described herein can be implemented as a cloud-based service provided by one or more server computers, which are implemented by at least a portion of the components of the source device 12 and / or at least a portion of the components of the target device 14 in the cloud computing service network; and one or more other operations described herein can be implemented by one or more client devices. In some implementations, the cloud computing service network can be a private cloud, a public cloud, or a hybrid cloud. Without departing from the scope of this disclosure, terms such as “cloud,” “cloud computing,” and “cloud-based” may be used interchangeably as appropriate. It should be understood that this disclosure is not limited to implementations within the aforementioned cloud computing service networks. Instead, this disclosure can also be implemented in any other type of computing environment currently known or developed in the future.

[0063] Figure 2 This is a block diagram illustrating an exemplary video encoder 20 according to some implementations described in this application. The video encoder 20 can perform intra-frame predictive coding and inter-frame predictive coding on video blocks within a video frame. Intra-frame predictive coding relies on spatial prediction to reduce or remove spatial redundancy in the video data within a given video frame or picture. Inter-frame predictive coding relies on temporal prediction to reduce or remove temporal redundancy in the video data within neighboring video frames or pictures of a video sequence. It should be noted that in the field of video encoding and decoding, the term "frame" can be used as a synonym for the terms "image" or "picture".

[0064] like Figure 2As shown, the video encoder 20 includes a video data memory 40, a prediction processing unit 41, a decoded picture buffer (DPB) 64, an adder 50, a transform processing unit 52, a quantization unit 54, and an entropy coding unit 56. The prediction processing unit 41 further includes a motion estimation unit 42, a motion compensation unit 44, a segmentation unit 45, an intra-frame prediction processing unit 46, and an intra-frame block copying (BC) unit 48. In some implementations, the video encoder 20 also includes an inverse quantization unit 58, an inverse transform processing unit 60, and an adder 62 for video block reconstruction. A loop filter 63, such as a deblocking filter, can be located between the adder 62 and the DPB 64 to filter block boundaries to remove block artifacts from the reconstructed video. In addition to the deblocking filter, another loop filter (e.g., a sample adaptive offset (SAO) filter, a cross-component sample adaptive offset (CCSAO) filter, and / or an adaptive loop filter (ALF)) can be used to filter the output of the adder 62. It should be noted that, regarding the CCSAO technology, this application is not limited to the embodiments described herein, but can also be applied to situations where an offset is selected for any other component among the luminance, Cb, and Cr chrominance components based on any one of the luminance, Cb, and Cr chrominance components to modify that other component based on the selected offset. Furthermore, it should be noted that the first component mentioned herein can be any one of the luminance, Cb, and Cr chrominance components, the second component mentioned herein can be any other one of the luminance, Cb, and Cr chrominance components, and the third component mentioned herein can be the remaining components among the luminance, Cb, and Cr chrominance components. In some examples, the loop filter can be omitted, and the decoded video block can be directly provided to the DPB 64 by the adder 62. The video encoder 20 can take the form of a fixed or programmable hardware unit, or can be distributed among one or more of the fixed or programmable hardware units shown.

[0065] The video data storage device 40 can store video data encoded by components of the video encoder 20. This data can be stored, for example, from... Figure 1 The video source 18 shown receives video data from the video data memory 40. The DPB 64 is a buffer that stores reference video data (e.g., reference frames or pictures) used by the video encoder 20 (e.g., in intra-frame or inter-frame predictive coding modes) when encoding the video data. The video data memory 40 and DPB 64 can be formed from any of a variety of memory devices. In various examples, the video data memory 40 may be on-chip along with other components of the video encoder 20, or off-chip relative to those components.

[0066] like Figure 2As shown, after receiving video data, the segmentation unit 45 within the prediction processing unit 41 segments the video data into video blocks. This segmentation may also include segmenting the video frame into strips, tiles (e.g., a collection of video blocks), or other larger coding units (CUs) according to a predefined splitting structure associated with the video data (e.g., a quadtree (QT) structure). A video frame is, or can be considered, a two-dimensional array or matrix of sample points with sample values. Sample points in the array may also be referred to as pixels or image elements (pel). The number of samples in the horizontal and vertical directions (or axes) of the array or image defines the size and / or resolution of the video frame. For example, a video frame can be divided into multiple video blocks using QT segmentation. A video block is again, or can be considered, a two-dimensional array or matrix of sample points with sample values, but its dimension is smaller than that of the video frame. The number of samples in the horizontal and vertical directions (or axes) of the video block defines the size of the video block. A video block can be further divided into one or more block partitions or sub-blocks (which can then re-form blocks) by iteratively using, for example, QT partitioning, binary tree (BT) partitioning, or ternary tree (TT) partitioning, or any combination thereof. It should be noted that, as used herein, the term “block” or “video block” can be a portion of a frame or picture, particularly a rectangular (square or non-square) portion. Referring, for example, to HEVC and VVC, a block or video block can be or corresponds to a coding tree unit (CTU), CU, prediction unit (PU), or transform unit (TU) and / or can be or corresponds to a corresponding block (e.g., coding tree block (CTB), coding block (CB), prediction block (PB), or transform block (TB)) and / or sub-block.

[0067] Prediction processing unit 41 can select one of several feasible predictive coding modes for the current video block based on error results (e.g., coding rate and distortion level), such as one or more inter-frame predictive coding modes from among multiple intra-frame predictive coding modes. Prediction processing unit 41 can provide the resulting intra-frame predictive coded block or inter-frame predictive coded block to adder 50 to generate a residual block and to adder 62 to reconstruct the coded block for subsequent use as part of a reference frame. Prediction processing unit 41 also provides syntax elements (e.g., motion vectors, intra-frame mode indicators, segmentation information, and other such syntax information) to entropy coding unit 56.

[0068] To select a suitable intra-predictive coding mode for the current video block, the intra-predictive processing unit 46 within the prediction processing unit 41 can perform intra-predictive coding of the current video block in relation to one or more neighboring blocks in the same frame as the current block to be encoded to provide spatial prediction. The motion estimation unit 42 and motion compensation unit 44 within the prediction processing unit 41 perform inter-predictive coding of the current video block in relation to one or more prediction blocks in one or more reference frames to provide temporal prediction. The video encoder 20 can perform multiple coding passes, for example, to select a suitable coding mode for each block of video data.

[0069] In some implementations, motion estimation unit 42 determines the inter-frame prediction mode for the current video frame by generating motion vectors based on a predetermined pattern within the video frame sequence. The motion vectors indicate the displacement of a video block within the current video frame relative to a prediction block within a reference video frame. Motion estimation performed by motion estimation unit 42 is the process of generating motion vectors that estimate the motion of video blocks. For example, the motion vectors may indicate the displacement of a video block within the current video frame or picture relative to a prediction block within a reference frame associated with the current block being encoded in the current frame. The predetermined pattern may designate video frames in the sequence as P-frames or B-frames. Intra-frame BC unit 48 may determine vectors (e.g., block vectors) for intra-frame BC coding in a similar manner to how motion estimation unit 42 determines the motion vectors for inter-frame prediction, or the block vectors may be determined using motion estimation unit 42.

[0070] Regarding pixel differences, the prediction block used for the video block can be, or can correspond to, a block or reference block of a reference frame that is considered to closely match the video block to be encoded. Pixel differences can be determined by sum of absolute differences (SAD), sum of squared differences (SSD), or other difference metrics. In some implementations, the video encoder 20 can compute values ​​for sub-integer pixel positions of the reference frame stored in the DPB 64. For example, the video encoder 20 can interpolate values ​​for quarter-pixel positions, eighth-pixel positions, or other fractional pixel positions of the reference frame. Therefore, the motion estimation unit 42 can perform motion search relative to full pixel positions and fractional pixel positions and output a motion vector with fractional pixel precision.

[0071] The motion estimation unit 42 calculates motion vectors for video blocks in an inter-frame predictive coded frame by comparing the position of the video block with the position of a predicted block in a reference frame selected from either a first reference frame list (list 0) or a second reference frame list (list 1), each of the first and second reference frame lists identifying one or more reference frames stored in the DPB 64. The motion estimation unit 42 sends the calculated motion vectors to the motion compensation unit 44, and then to the entropy coding unit 56.

[0072] Motion compensation performed by motion compensation unit 44 may involve acquiring or generating prediction blocks based on motion vectors determined by motion estimation unit 42. Upon receiving motion vectors for the current video block, motion compensation unit 44 may locate the prediction block pointed to by the motion vector in a reference frame list within a reference frame list, retrieve the prediction block from DPB 64, and forward the prediction block to adder 50. Adder 50 then forms a residual video block of pixel differences by subtracting the pixel values ​​of the prediction block provided by motion compensation unit 44 from the pixel values ​​of the currently encoded video block. The pixel differences forming the residual video block may include luminance component differences, chrominance component differences, or both. Motion compensation unit 44 may also generate syntax elements associated with video blocks of a video frame for use by video decoder 30 when decoding video blocks of a video frame. Syntax elements may include, for example, syntax elements defining motion vectors for identifying prediction blocks, any flags indicating prediction modes, or any other syntax information described herein. It should be noted that motion estimation unit 42 and motion compensation unit 44 may be highly integrated, but are shown separately for conceptual purposes.

[0073] In some implementations, the intra-BC unit 48 can generate vectors and acquire prediction blocks in a manner similar to that described above in conjunction with the motion estimation unit 42 and the motion compensation unit 44. However, these prediction blocks are in the same frame as the current block being encoded, and these vectors are referred to as block vectors rather than motion vectors. Specifically, the intra-BC unit 48 can determine the intra-prediction mode to be used for encoding the current block. In some examples, the intra-BC unit 48 can, for example, use various intra-prediction modes to encode the current block during individual encoding passes and test their performance through rate-distortion analysis. Next, the intra-BC unit 48 can select a suitable intra-prediction mode from the various tested intra-prediction modes to use and generate an intra-mode indicator accordingly. For example, the intra-BC unit 48 can use rate-distortion analysis to calculate rate-distortion values ​​for the various tested intra-prediction modes and select the intra-prediction mode with the best rate-distortion characteristics from the tested modes as the suitable intra-prediction mode to use. Rate-distortion analysis generally determines the amount of distortion (or error) between the coded block and the original uncoded block that was encoded to produce the coded block, as well as the bit rate (i.e., the number of bits) used to produce the coded block. Intra-frame BC unit 48 can calculate the ratio based on the distortion and rate for various coded blocks to determine which intra-frame prediction mode exhibits the optimal rate-distortion value for the block.

[0074] In other examples, the intra-frame BC unit 48 may use, in whole or in part, the motion estimation unit 42 and the motion compensation unit 44 to perform such functions for intra-frame BC prediction according to the implementation described herein. In any case, for intra-frame block copying, in terms of pixel differences, the predicted block may be a block considered to closely match the block to be encoded, the pixel differences may be determined by SAD, SSD, or other difference metrics, and identifying the predicted block may include calculating values ​​for sub-integer pixel positions.

[0075] Regardless of whether the predicted block comes from the same frame predicted intra-frame or from different frames predicted inter-frame, the video encoder 20 can form a residual video block by subtracting the pixel values ​​of the predicted block from the pixel values ​​of the current video block being encoded. The pixel difference forming the residual video block can include both luma component difference and chroma component difference.

[0076] As described above, as an alternative to inter-frame prediction performed by motion estimation unit 42 and motion compensation unit 44, or intra-block copy prediction performed by intra-BC unit 48, intra-frame prediction processing unit 46 can perform intra-frame prediction on the current video block. Specifically, intra-frame prediction processing unit 46 can determine an intra-frame prediction mode for encoding the current block. To this end, intra-frame prediction processing unit 46 can use various intra-frame prediction modes to encode the current block, for example, during individual encoding passes, and intra-frame prediction processing unit 46 (or, in some examples, mode selection unit) can select a suitable intra-frame prediction mode from the tested intra-frame prediction modes for use. Intra-frame prediction processing unit 46 can provide information indicating the intra-frame prediction mode selected for the block to entropy coding unit 56. Entropy coding unit 56 can encode the information indicating the selected intra-frame prediction mode into the bitstream.

[0077] After prediction processing unit 41 determines a prediction block for the current video block via inter-frame prediction or intra-frame prediction, adder 50 forms a residual video block by subtracting the prediction block from the current video block. The residual video data in the residual block may be included in one or more TUs and provided to transform processing unit 52. Transform processing unit 52 uses a transform (e.g., discrete cosine transform (DCT) or a conceptually similar transform) to transform the residual video data into residual transform coefficients.

[0078] The transform processing unit 52 can send the resulting transform coefficients to the quantization unit 54. The quantization unit 54 quantizes the transform coefficients to further reduce the bit rate. The quantization process can also reduce the bit depth associated with some or all of the coefficients. The degree of quantization can be modified by adjusting the quantization parameters. In some examples, the quantization unit 54 can subsequently perform a scan on the matrix including the quantized transform coefficients. Alternatively, the entropy coding unit 56 can perform the scan.

[0079] After quantization, entropy coding unit 56 entropies the quantized transform coefficients into a video bitstream using, for example, context-adaptive variable-length coding (CAVLC), context-adaptive binary arithmetic coding (CABAC), syntax-based context-adaptive binary arithmetic coding (SBAC), probabilistic interval segmented entropy (PIPE) coding, or another entropy coding method or technique. The encoded bitstream can then be sent to, for example,... Figure 1 The video decoder 30 shown, or archived in, as Figure 1 The data is stored in storage device 32 for later transmission to or retrieval by video decoder 30. Entropy coding unit 56 can also entropy code the motion vectors and other syntax elements used for the current video frame being encoded.

[0080] The inverse quantization unit 58 and the inverse transform processing unit 60 apply inverse quantization and inverse transform, respectively, to reconstruct residual video blocks in the pixel domain for generating reference blocks to predict other video blocks. As noted above, the motion compensation unit 44 can generate motion-compensated prediction blocks from one or more reference blocks of a frame stored in the DPB 64. The motion compensation unit 44 can also apply one or more interpolation filters to the prediction blocks to compute sub-integer pixel values ​​for use in motion estimation.

[0081] Adder 62 adds the reconstructed residual block to the motion-compensated prediction block generated by motion compensation unit 44 to generate a reference block for storage in DPB 64. The reference block can then be used as a prediction block by intra-frame BC unit 48, motion estimation unit 42, and motion compensation unit 44 for inter-frame prediction of another video block in subsequent video frames.

[0082] Figure 3 This is a block diagram illustrating an exemplary video decoder 30 according to some implementations of this application. The video decoder 30 includes a video data memory 79, an entropy decoding unit 80, a prediction processing unit 81, an inverse quantization unit 86, an inverse transform processing unit 88, an adder 90, and a DPB 92. The prediction processing unit 81 further includes a motion compensation unit 82, an intra-frame prediction unit 84, and an intra-frame prediction (BC) unit 85. The video decoder 30 can perform operations in conjunction with the above. Figure 2 The encoding process described for video encoder 20 is essentially the inverse of the decoding process. For example, motion compensation unit 82 can generate prediction data based on motion vectors received from entropy decoding unit 80, while intra-frame prediction unit 84 can generate prediction data based on intra-frame prediction mode indicators received from entropy decoding unit 80.

[0083] In some examples, the units of the video decoder 30 may be assigned tasks to perform the implementation of this application. Furthermore, in some examples, the implementation of this disclosure may be distributed across one or more units of the video decoder 30. For example, the intra-frame BC unit 85 may perform the implementation of this application alone or in combination with other units of the video decoder 30 (e.g., motion compensation unit 82, intra-frame prediction unit 84, and entropy decoding unit 80). In some examples, the video decoder 30 may not include the intra-frame BC unit 85, and the functionality of the intra-frame BC unit 85 may be performed by other components of the prediction processing unit 81 (e.g., motion compensation unit 82).

[0084] Video data memory 79 can store video data, such as encoded video bitstreams, to be decoded by other components of video decoder 30. The video data stored in video data memory 79 can be obtained, for example, from storage device 32, from a local video source (e.g., a camera), via wired or wireless network communication of video data, or by accessing a physical data storage medium (e.g., a flash drive or hard disk). Video data memory 79 may include an encoded picture buffer (CPB) that stores encoded video data from the encoded video bitstream. The DPB 92 of video decoder 30 stores reference video data for use by video decoder 30 (e.g., in intra-frame or inter-frame predictive coding modes) when decoding video data. Video data memory 79 and DPB 92 can be formed of any memory device from a variety of memory devices, such as dynamic random access memory (DRAM) (including synchronous DRAM (SDRAM)), magnetoresistive RAM (MRAM), resistive RAM (RRAM), or other types of memory devices. For illustrative purposes, video data memory 79 and DPB 92 are shown in... Figure 3 The video data memory 79 and DPB 92 are depicted as two distinct components of the video decoder 30. However, it will be apparent to those skilled in the art that the video data memory 79 and DPB 92 may be provided by the same memory device or separate memory devices. In some examples, the video data memory 79 may be on-chip along with other components of the video decoder 30, or off-chip relative to those components.

[0085] During the decoding process, the video decoder 30 receives an encoded video bitstream representing video blocks of encoded video frames and associated syntax elements. The video decoder 30 may receive syntax elements at the video frame level and / or the video block level. The entropy decoding unit 80 of the video decoder 30 performs entropy decoding on the bitstream to generate quantization coefficients, motion vectors or intra-prediction mode indicators, and other syntax elements. The entropy decoding unit 80 then forwards the motion vectors or intra-prediction mode indicators, and other syntax elements to the prediction processing unit 81.

[0086] When a video frame is encoded as an intra-predictive coded (I) frame or as an intra-coded prediction block in other types of frames, the intra-prediction unit 84 of the prediction processing unit 81 can generate prediction data for the video block of the current video frame based on the intra-prediction mode transmitted by the signal and reference data from the previous decoded block of the current frame.

[0087] When a video frame is encoded as an inter-frame predictive coded (i.e., B or P) frame, the motion compensation unit 82 of the prediction processing unit 81 generates one or more prediction blocks for the current video frame based on the motion vectors and other syntax elements received from the entropy decoding unit 80. Each of the prediction blocks can be generated from a reference frame within a reference frame list. The video decoder 30 can construct the reference frame list, i.e., list 0 and list 1, based on the reference frames stored in the DPB 92 using a default construction technique.

[0088] In some examples, when a video block is encoded according to the intra-BC mode described herein, the intra-BC unit 85 of the prediction processing unit 81 generates a prediction block for the current video block based on the block vector and other syntax elements received from the entropy decoding unit 80. The prediction block can be located within a reconstructed region of the same image as the current video block, as defined by the video encoder 20.

[0089] Motion compensation unit 82 and / or intra-frame prediction (BC) unit 85 determine prediction information for video blocks in the current video frame by parsing motion vectors and other syntax elements, and then use this prediction information to generate prediction blocks for the current video block being decoded. For example, motion compensation unit 82 uses some of the received syntax elements to determine the prediction mode (e.g., intra-frame prediction or inter-frame prediction) for encoding video blocks in the video frame, the inter-frame prediction frame type (e.g., B or P), construction information for one or more reference frames in the reference frame list for the frame, motion vectors for each inter-frame prediction encoded video block in the frame, the inter-frame prediction state for each inter-frame prediction encoded video block in the frame, and other information for decoding video blocks in the current video frame.

[0090] Similarly, the intra-BC unit 85 can use some of the syntax elements (e.g., flags) from the received syntax elements to determine whether the current video block was predicted using the intra-BC mode, which video blocks of the frame are in the reconstruction region and should be stored in the DPB 92, the block vector for each intra-BC predicted video block of the frame, the intra-BC prediction state for each intra-BC predicted video block of the frame, and other information for decoding the video blocks in the current video frame.

[0091] The motion compensation unit 82 can also perform interpolation using interpolation filters, such as those used by the video encoder 20 during encoding of video blocks, to calculate interpolated values ​​for sub-integer pixels of the reference block. In this case, the motion compensation unit 82 can determine the interpolation filters used by the video encoder 20 based on the received syntax elements and use these interpolation filters to generate the prediction block.

[0092] The dequantization unit 86 dequantizes the quantized transform coefficients provided in the bitstream and entropy-decoded by the entropy decoding unit 80 using the same quantization parameters calculated by the video encoder 20 for each video block in the video frame to determine the degree of quantization. The inverse transform processing unit 88 applies an inverse transform (e.g., inverse DCT, inverse integer transform, or a conceptually similar inverse transform process) to the transform coefficients in order to reconstruct the residual block in the pixel domain.

[0093] After the motion compensation unit 82 or the intra-frame BC unit 85 generates a prediction block for the current video block based on vectors and other syntax elements, the adder 90 reconstructs the decoded video block for the current video block by adding the residual block from the inverse transform processing unit 88 to the corresponding prediction block generated by the motion compensation unit 82 and the intra-frame BC unit 85. A loop filter 91 (e.g., a deblocking filter, SAO filter, CCSAO filter, and / or ALF) may be located between the adder 90 and the DPB 92 for further processing of the decoded video block. In some examples, the loop filter 91 may be omitted, and the decoded video block may be directly provided to the DPB 92 by the adder 90. The decoded video block in a given frame is then stored in the DPB 92, which stores reference frames for subsequent motion compensation of the next video block. The DPB 92 or a separate memory device may also store the decoded video for later presentation on a display device (e.g., ...). Figure 1 On the display device 34).

[0094] In a typical video encoding and decoding process, a video sequence usually consists of an ordered set of frames or pictures. Each frame may include three sample arrays, denoted as SL, SCb, and SCr. SL is a two-dimensional array of luma samples. SCb is a two-dimensional array of chroma samples (Cb). SCr is a two-dimensional array of chroma samples (Cr). In other instances, a frame may be monochromatic and therefore consist of only a two-dimensional array of luma samples.

[0095] like Figure 4AAs shown, the video encoder 20 (or more specifically, the segmentation unit 45) generates an encoded representation of a frame by first segmenting the frame into a set of CTUs. A video frame may include an integer number of CTUs ordered consecutively from left to right and top to bottom in raster scan order. Each CTU is the largest logical coding unit, and the width and height of the CTU are signaled by the video encoder 20 in a sequence parameter set such that all CTUs in the video sequence have the same size, one of 128×128, 64×64, 32×32, and 16×16. However, it should be noted that this application is not limited to a specific size. Figure 4B As shown, each CTU may include a CTB for luma samples, two corresponding coding tree blocks for chroma samples, and syntax elements for encoding the samples of the coding tree blocks. The syntax elements describe the properties of different types of units in the coded pixel block and how the video sequence can be reconstructed at the video decoder 30, including inter-frame prediction or intra-frame prediction, intra-frame prediction mode, motion vectors, and other parameters. In monochrome images or images with three separate color planes, the CTU may include a single coding tree block and syntax elements for encoding the samples of that coding tree block. The coding tree block may be an N×N sample block.

[0096] To achieve better performance, the video encoder 20 can recursively perform tree splitting on the coding tree blocks of the CTU, such as binary tree splitting, ternary tree splitting, quadtree splitting, or combinations thereof, and divide the CTU into smaller CUs. Figure 4C As described, the 64×64 CTU 400 is first divided into four smaller CUs, each with a block size of 32×32. Of these four smaller CUs, CU 410 and CU 420 are each divided into four CUs with a block size of 16×16. The two 16×16 CUs, 430 and CU 440, are each further divided into four CUs with a block size of 8×8. Figure 4D Depicting as shown Figure 4C The final result of the CTU 400 partitioning process, as depicted in the diagram, is a quadtree data structure, where each leaf node of the quadtree corresponds to a CU of a corresponding size ranging from 32×32 to 8×8. Similar to... Figure 4B The CTU depicted in the image can include, for example, two corresponding coded blocks (CBs) of the same size frame's luma and chroma samples, as well as syntax elements for encoding the samples of the coded blocks. In monochrome images or images with three separate color planes, a CU can include a single coded block and a syntax structure for encoding the samples of the coded block. It should be noted that... Figure 4C and Figure 4DThe quadtree partitioning depicted is for illustrative purposes only, and a CTU can be split into multiple CUs based on quadtree / ternary / binary partitioning to adapt to varying local characteristics. In multi-type tree structures, a CTU is partitioned according to a quadtree structure, and each quadtree leaf CU can be further partitioned according to binary and ternary tree structures. Figure 4E As shown, a coded block with width W and height H has five possible segmentation types: quadruple segmentation, horizontal binary segmentation, vertical binary segmentation, horizontal triple segmentation, and vertical triple segmentation.

[0097] In some implementations, the video encoder 20 may further segment the coded blocks of the CU into one or more (M×N) PBs. A PB is a rectangular (square or non-square) sample block to which the same prediction (inter-frame or intra-frame) is applied. The PU of the CU may include a PB of luma samples, two corresponding PBs of chroma samples, and syntax elements for predicting the PBs. In monochrome images or images with three separate color planes, a PU may include a single PB and a syntax structure for predicting the PBs. The video encoder 20 may generate predicted luma blocks, predicted Cb blocks, and predicted Cr blocks for each PU of the CU, representing the luma PB, Cb PB, and Cr PB.

[0098] Video encoder 20 can generate prediction blocks for a PU using intra-frame prediction or inter-frame prediction. If video encoder 20 uses intra-frame prediction to generate prediction blocks for the PU, it can generate the prediction blocks for the PU based on decoded samples of the frame associated with the PU. If video encoder 20 uses inter-frame prediction to generate prediction blocks for the PU, it can generate the prediction blocks for the PU based on decoded samples of one or more frames other than the frame associated with the PU.

[0099] After the video encoder 20 generates a predicted luminance block, a predicted Cb block, and a predicted Cr block for one or more PUs of the CU, the video encoder 20 can generate a luminance residual block for the CU by subtracting the predicted luminance block of the CU from the original luminance coding block of the CU, such that each sample in the luminance residual block of the CU indicates the difference between a luminance sample in one of the predicted luminance blocks of the CU and a corresponding sample in the original luminance coding block of the CU. Similarly, the video encoder 20 can generate Cb residual blocks and Cr residual blocks for the CU, respectively, such that each sample in the Cb residual block of the CU indicates the difference between a Cb sample in one of the predicted Cb blocks of the CU and a corresponding sample in the original Cb coding block of the CU, and each sample in the Cr residual block of the CU indicates the difference between a Cr sample in one of the predicted Cr blocks of the CU and a corresponding sample in the original Cr coding block of the CU.

[0100] In addition, such as Figure 4CAs shown, the video encoder 20 can use quadtree partitioning to decompose the luminance residual block, Cb residual block, and Cr residual block of the CU into one or more luminance transform blocks, Cb transform blocks, and Cr transform blocks, respectively. A transform block is a rectangular (square or non-square) sample block to which the same transform is applied. A TU of the CU can include a transform block of luminance samples, two corresponding transform blocks of chrominance samples, and syntax elements for transforming the samples of the transform block. Therefore, each TU of the CU can be associated with a luminance transform block, a Cb transform block, and a Cr transform block. In some examples, the luminance transform block associated with a TU can be a sub-block of the CU's luminance residual block. A Cb transform block can be a sub-block of the CU's Cb residual block. A Cr transform block can be a sub-block of the CU's Cr residual block. In a monochrome image or an image with three separate color planes, a TU can include a single transform block and syntax structures for transforming the samples of that transform block.

[0101] The video encoder 20 can apply one or more transforms to the luminance transform block of the TU to generate a luminance coefficient block for the TU. The coefficient block can be a two-dimensional array of transform coefficients. The transform coefficients can be scalars. The video encoder 20 can apply one or more transforms to the Cb transform block of the TU to generate a Cb coefficient block for the TU. The video encoder 20 can apply one or more transforms to the Cr transform block of the TU to generate a Cr coefficient block for the TU.

[0102] After generating coefficient blocks (e.g., luminance coefficient blocks, Cb coefficient blocks, or Cr coefficient blocks), video encoder 20 can quantize the coefficient blocks. Quantization typically refers to the process of quantizing transform coefficients to potentially reduce the amount of data used to represent the transform coefficients, thereby providing further compression. After quantizing the coefficient blocks, video encoder 20 can entropy-encode the syntax elements indicating the quantized transform coefficients. For example, video encoder 20 can perform CABAC on the syntax elements indicating the quantized transform coefficients. Finally, video encoder 20 can output a bitstream comprising a bit sequence that forms a representation of encoded frames and associated data; the bitstream is stored in storage device 32 or transmitted to target device 14.

[0103] After receiving the bitstream generated by the video encoder 20, the video decoder 30 can parse the bitstream to obtain syntax elements. The video decoder 30 can reconstruct frames of video data, at least in part, based on the syntax elements obtained from the bitstream. The process of reconstructing the video data is generally the inverse of the encoding process performed by the video encoder 20. For example, the video decoder 30 can perform an inverse transform on the coefficient block associated with the TU of the current CU to reconstruct the residual block associated with the TU of the current CU. The video decoder 30 also reconstructs the coded blocks of the current CU by adding the samples of the prediction block of the PU for the current CU to the corresponding samples of the transform block of the TU of the current CU. After reconstructing the coded blocks for each CU of the frame, the video decoder 30 can reconstruct the frame.

[0104] As mentioned above, video encoding and decoding primarily use two modes (i.e., intra-frame prediction and inter-frame prediction) to achieve video compression. It should be noted that IBC can be considered either intra-frame prediction or a third mode. Between the two modes, inter-frame prediction contributes more to coding efficiency than intra-frame prediction because it uses motion vectors to predict the current video block based on a reference video block.

[0105] However, with continuously improving video data capture technologies and finer video tile sizes used to preserve details in video data, the amount of data required to represent the motion vectors for the current frame has also increased significantly. One way to overcome this challenge benefits from the fact that not only do a set of neighboring CUs in both the spatial and temporal domains have similar video data for prediction purposes, but the motion vectors between these neighboring CUs are also similar. Therefore, the motion information of spatially neighboring CUs and / or temporally co-located CUs can be used as an approximation of the motion information (e.g., motion vectors) of the current CU (also known as the "Motion Vector Prediction Value" (MVP) of the current CU) by exploring their spatial and temporal correlations.

[0106] Instead of the above combination Figure 2 The method described involves encoding the actual motion vector of the current CU, determined by the motion estimation unit 42, into the video bitstream, and subtracting the predicted motion vector value of the current CU from the actual motion vector of the current CU to produce the motion vector difference (MVD) for the current CU. By doing so, it is not necessary to encode the motion vector determined by the motion estimation unit 42 for each CU of the frame into the video bitstream, and the amount of data used to represent motion information in the video bitstream can be significantly reduced.

[0107] Similar to the process of selecting a prediction block in a reference frame during inter-frame prediction of a coded block, both the video encoder 20 and the video decoder 30 need to employ a set of rules to construct a motion vector candidate list (also known as a "merging list") for the current CU using those potential candidate motion vectors associated with spatially neighboring CUs and / or temporally co-located CUs, and then select one member from the motion vector candidate list as the motion vector prediction value for the current CU. By doing so, it is not necessary to send the motion vector candidate list itself from the video encoder 20 to the video decoder 30, and the index of the selected motion vector prediction value within the motion vector candidate list is sufficient for both the video encoder 20 and the video decoder 30 to encode and decode the current CU using the same motion vector prediction value from the motion vector candidate list.

[0108] Typically, apart from further extending, adding and / or improving several prediction tools (e.g., extended merge prediction, MMVD and GPM), the basic inter-frame prediction scheme applied in VVC is almost identical to the basic inter-frame prediction scheme in HEVC.

[0109] Extended merge forecast

[0110] With continuous improvements in video data capture technology and the increasing fineness of video block sizes used to preserve video data details, the amount of data required to represent the motion vector of the current image has also increased significantly. One way to overcome this challenge is to use motion information (e.g., motion vectors) of spatially adjacent CUs, temporally co-located CUs, etc., of the current CU as an approximation (e.g., prediction) of the motion information of the current CU, which is also referred to as the "Motion Vector Prediction Value (MVP)" of the current CU. The "motion vector" used throughout this disclosure includes not only motion vectors between CUs from different frames (e.g., between temporally co-located CUs in inter-frame prediction) but also block vectors between CUs within the same frame (e.g., between spatially adjacent CUs in intra-frame prediction).

[0111] Similar to the process of selecting a prediction block from a reference picture during inter-frame prediction of a coded block, both video encoder 20 and video decoder 30 need to employ a set of rules to construct an MVP candidate list for the current CU, and then select an MVP candidate from the MVP candidate list as the MVP of the current CU. By doing so, it is not necessary to transmit the MVP candidate list itself between video encoder 20 and video decoder 30, and the index of the MVP candidate selected from the MVP candidate list is sufficient for video encoder 20 and video decoder 30 to use the same MVP candidate selected from the MVP candidate list to encode and decode the current CU.

[0112] In VVC, the MVP candidate list is constructed by sequentially including the following five MVP types:

[0113] —The spatial MVP from the spatially adjacent CUs (i.e., spatial candidates);

[0114] —The time MVP from the time-isolated CU (i.e., the time candidate);

[0115] —History-based MVP (HMVP) from a First-In-First-Out (FIFO) table;

[0116] —Paired average MVP; and

[0117] —Zero MVP.

[0118] The size of the MVP candidate list is signaled in the sequence parameter set header, and the maximum allowed size of the MVP candidate list is 6. For each CU encoded in merge mode, the index of the best MVP candidate is encoded using truncated unary binarization. The first bit of the index is encoded using the context, and the remaining bits of the index are encoded using bypass encoding.

[0119] The export process for each type of MVP is provided below. Like HEVC, VVC also supports exporting a list of MVP candidates in parallel for all CUs within a given region.

[0120] Derive MVP from spatial candidates

[0121] In VVC, based on spatial candidates (e.g., with...) Figure 5 The MVP derived from the current CU 101 (adjacent CUs) is the same as the MVP derived from spatial candidates in HEVC, except that the positions of the first two spatial candidates are swapped. From the CU located at... Figure 5 Up to four spatial candidates are selected from the spatial candidates depicted in the image (i.e., top position B0, left position A0, top right position B1, bottom left position A1, and top left position B2). The derivation process is performed in the order of the CUs at positions B0, A0, B1, A1, and B2. The CU at position B2 is considered only if one or more CUs at positions B0, A0, B1, and A1 are unavailable (e.g., because the one or more CUs belong to other stripes or tiles) or are intra-coded.

[0122] After adding the CU at position B0 as a candidate to the merged candidate list, redundancy checks are performed on the addition of the remaining candidates to the merged candidate list. This ensures that candidates with the same motion information are excluded from the merged candidate list, thereby improving encoding and decoding efficiency. To reduce computational complexity, not all possible candidate pairs are considered in the redundancy check. Instead, only... Figure 6Pairs linked by arrowed lines are considered, and a candidate is added to the merged candidate list only if the motion information of the candidate in the pair used for redundancy checking differs from the motion information of the candidate to be added. The spatial MVP derived from the candidates in the merged candidate list is added to the MVP candidate list.

[0123] Export MVP based on time candidates

[0124] During the process of deriving the MVP based on the time candidate, only one time candidate is added to the merge candidate list. Specifically, when deriving the MVP based on this time candidate, for the current CU (e.g., ...), Figure 7 curr_CU 303), based on belonging to the same image (e.g., Figure 7 The corresponding CU of col_pic 302 in (e.g., Figure 7 The scaled motion vector (col_CU 301) is used as a temporal candidate to derive the MVP candidate list, and this scaled motion vector is added as a temporal MVP candidate. The list of reference images and their indices for deriving the co-located CU are explicitly signaled in the strip header. Figure 7 As shown, the scaled motion vector is obtained (i.e., scaled) based on the motion vector of the co-located CU using the picture sequence count (POC) distance (i.e., tb and td), where tb is defined as the current picture (e.g., ...). Figure 7 Reference image for curr_pic 304 (e.g., Figure 7 The difference between curr_ref 305 in the image and the current image is the POC difference, while td is defined as the reference image of the co-located image (e.g., Figure 7 The POC difference between col_ref 306 and the corresponding image. The reference image index for the time candidate is set to zero.

[0125] like Figure 8 As depicted, the position of the corresponding CU (i.e., the co-occurring CU) in the current CU 401 is selected between positions C0 and C1. If the CU at position C0 in the co-occurring picture is unavailable, is intra-coded, or is located outside the current line of the CTU, then the CU at position C1 is used as the co-occurring CU for deriving the temporal MVP candidate. Otherwise, the CU at position C0 is used as the co-occurring CU for deriving the temporal MVP candidate.

[0126] Export HMVP candidates

[0127] After the spatial MVP and temporal MVP, HMVP candidates are added to the MVP candidate list. Motion information from previously coded blocks is stored in the HMVP table and used as the MVP for the current CU. The table with multiple HMVP candidates is maintained during encoding / decoding. The table is reset (cleared) when a new row of the CTU is encountered. Whenever a non-sub-block inter-coded CU exists, the associated motion information is added as the last entry in the HMVP table as a new HMVP candidate.

[0128] The size of the HMVP table is set to 6. When a new HMVP candidate is inserted into the HMVP table, a FIFO constraint is used, where a redundancy check is first applied to check if the same HMVP exists in the HMVP table. If found, the same HMVP is removed from the HMVP table, all subsequent HMVP candidates are shifted forward, and the same HMVP is added to the last entry in the HMVP table.

[0129] HMVP candidates can be used during the MVP candidate list construction process. The latest few HMVP candidates in the HMVP table are checked sequentially and inserted into the MVP candidate list after the temporal MVP candidates. Redundancy checks are applied to the HMVP candidates relative to the spatial and / or temporal MVP candidates.

[0130] To reduce the number of redundant verification operations, the following simplifications are introduced:

[0131] —Redundancy checks are performed on the last two entries in the HMVP table relative to the spatial MVP candidates derived from the spatial candidates at positions A1 and B1, respectively; and

[0132] —The process of building the MVP candidate list based on the HMVP candidate is terminated once the total number of available MVP candidates reaches the maximum allowed size of the MVP candidate list minus 1.

[0133] Derive Pairwise Average MVP Candidates

[0134] Pairwise averaged MVP candidates are generated by averaging the derived MVPs using a predefined pair of the first two merge candidates from an existing merge candidate list. The first merge candidate in the predefined pair can be defined as p0Cand, and the second merge candidate in the predefined pair can be defined as p1Cand. For each reference image list individually, the average motion vector is calculated based on the availability of motion vectors for p0Cand and p1Cand. If both motion vectors are available for a reference image list, they are averaged even if they point to different reference images, and the reference image of the averaged motion vector is set as the reference image of p0Cand. If only one motion vector is available for a reference image list, that motion vector is used directly. If no motion vector is available for a reference image list, the motion vector and reference image index for that reference image list remain invalid.

[0135] Zero MVP

[0136] If the MVP candidate list is not full after adding paired average MVP candidates, insert zero MVPs at the end of the MVP candidate list until the maximum allowed size of the MVP candidate list is reached.

[0137] MMVD

[0138] As mentioned above, in merge mode, motion information (i.e., MVP candidates) is implicitly derived from the MVP candidate list constructed for the current CU and directly used as the MV of the current CU to generate prediction samples for the current CU. This may result in a certain error between the actual MV of the current CU and the implicitly derived MVP. To improve the accuracy of the MV of the current CU, MMVD is introduced in VVC, where the motion vector difference (MVD) of the current CU is added to the implicitly derived MVP to obtain the MV of the current CU. After sending the regular merge flag, the MMVD flag is signaled to specify whether the MMVD mode is used for the current CU.

[0139] In MMVD mode, after selecting an MVP candidate from the first two MVP candidates in the MVP candidate list, MMVD information is signaled. The MMVD information includes an MMVD candidate flag to specify which of the first two MVP candidates is selected as the basis for the MV, a distance index to indicate the motion amplitude information of the MVD, and a direction index to indicate the motion direction information of the MVD.

[0140] The distance index indicating the motion amplitude information of the specified MVD is compared with the reference image of the current CU (e.g., Figure 9In the L0 reference image 501 or L1 reference image 503, the MVP candidate is pointed to by the selected MVP candidate (e.g., Figure 9 The dashed circle in the diagram represents a predefined offset of the starting point, from which the MVD can be derived, and this MVD can be added to the selected MVP candidate. The relationship between the distance index and the predefined offset is specified in Table 1 below. Table 1

[0141] The direction index specifies the sign of the MVD, which represents the direction of the MVD relative to the starting point. Table 2 specifies the relationship between the direction index and the predefined signs. It should be noted that the meaning of the MVD sign can vary depending on the information of the selected MVP candidate. When the selected MVP candidate is a one-way predictive MV or a two-way predictive MV (i.e., the POC of both reference images of the current image (e.g., the reference images of List 0 and List 1, which are also referred to as the L0 reference image and L1 reference image, respectively) is greater than or less than the POC of the current image, the sign in Table 2 specifies the sign of the MVD added to the selected MVP candidate. When the selected MVP candidate is a bidirectional prediction MV (where the two MVs point to different sides of the current image (i.e., the POC of one reference image of the current image is greater than the POC of the current image, while the POC of the other reference image of the current image is less than the POC of the current image)), if the POC distance for the L0 reference image (i.e., the POC distance between the L0 reference image and the current image) is greater than the POC distance for the L1 reference image (i.e., the POC distance between the L1 reference image and the current image), then the sign in Table 2 specifies the sign of the MVD for list 0 (MVD0) added to the MVP for list 0 of the selected MVP candidate (MVP0), and the sign of the MVD for list 1 (MVD1) added to the MVP for list 1 of the selected MVP candidate (MVP1) is opposite to the sign in Table 2; otherwise, if the POC distance for the L1 reference image is greater than the POC distance for the L0 reference image, then the sign in Table 2 specifies the sign of the MVD1 added to MVP1, and the sign of the MVD0 added to MVP0 is opposite to the sign in Table 2. Table 2

[0142] MVD is scaled based on the POC distance. If the POC distances for the L0 and L1 reference images are the same, no scaling of the MVD is required. Otherwise, if the POC distance for the L0 reference image is greater than the POC distance for the L1 reference image, MVD1 is scaled. If the POC distance for the L1 reference image is greater than the POC distance for the L0 reference image, MVD0 is scaled.

[0143] GPM

[0144] In VVC, GPM is supported for inter-frame prediction. A CU-level flag is used to signal to GPM as a merging mode. Other merging modes include regular merging, MMVD, CIIP, and sub-block merging. For each possible CU size W×H (W=2...),... m And H = 2 n ,in GPM supports a total of 64 partitions, and the possible CU sizes W×H do not include 8×64 and 64×8.

[0145] When using GPM, the CU is split into two parts by a geometrically positioned straight line. The position of the split line is mathematically derived from the angle and offset parameters of the specific partition. Each part of the CU obtained through geometric segmentation is predicted inter-frame using its own motion; and only unidirectional prediction is allowed for each partition, i.e., each part has one motion vector and one reference index. Unidirectional prediction motion constraints are applied to ensure that, like traditional bidirectional prediction, only two motion-compensated predictions are needed for each CU.

[0146] If GPM is used for the current CU, then the geometric partition index (indicating the angle and offset of the geometric partition) and two merge indexes (one merge index for each partition) are further signaled.

[0147] The unidirectional prediction candidate list is directly derived from the merge candidate list constructed according to the extended merge prediction process described above. Let n represent the index of the unidirectional prediction motion vector in the unidirectional prediction candidate list. The LX motion vector of the nth merge candidate in the merge candidate list (where X equals the parity of n) is used as the nth unidirectional prediction motion vector of GPM. Figure 10 In this context, these motion vectors are labeled with "x". If the corresponding LX motion vector of the nth merge candidate in the merge candidate list does not exist, the L(1-X) motion vector of the same merge candidate is used instead as the unidirectional predicted motion vector of GPM.

[0148] CIIP

[0149] In VVC, when encoding a CU in merge mode, if the CU contains at least 64 luma samples (i.e., the width of the CU multiplied by the height of the CU is equal to or greater than 64), and if both the width and height of the CU are less than 128 luma samples, an additional flag is signaled to indicate whether CIIP mode is applied to the current CU. In CIIP mode, prediction signals are obtained by combining inter-frame prediction signals with intra-frame prediction signals. The inter-frame prediction signals in CIIP mode are derived using the same inter-frame prediction process applied in regular merge mode; and the intra-frame prediction signals in CIIP mode are derived using the regular intra-frame prediction process utilizing planar mode. Then, a weighted average is used to combine the intra-frame prediction signals and the inter-frame prediction signals, wherein the combination is based on the top and left adjacent blocks of the current CU 1601 (e.g., ...). Figure 11 The weight values ​​are calculated for the encoding pattern shown in the figure.

[0150] —If the top adjacent block is available and is intra-coded, set isIntraTop to 1; otherwise, set isIntraTop to 0.

[0151] —If the left adjacent block is available and is intra-coded, set isIntraLeft to 1; otherwise, set isIntraLeft to 0.

[0152] —If (isIntraLeft+isIntraTop) equals 2, then set the weight value to 3;

[0153] —Otherwise, if (isIntraLeft+isIntraTop) equals 1, then set the weight value to 2;

[0154] —Otherwise, set the weight value to 1.

[0155] —Derive the prediction signal P in CIIP mode as follows. CIIP : (1)

[0156] in It is the inter-frame prediction signal in CIIP mode. It is an intra-frame prediction signal in CIIP mode. It represents the weight value, and >> indicates a right shift operation.

[0157] Intra-block copying in Universal Video Codec (VVC)

[0158] Intra-Block Copy (IBC) is a tool used in the HEVC extension on SCC. It is well known to significantly improve the encoding and decoding efficiency of screen content material. Since IBC mode is implemented as a block-level coding mode, block matching (BM) is performed at the encoder to find the optimal block vector (or motion vector) for each CU. Here, the block vector is used to indicate the displacement from the current block to a reference block that has already been reconstructed within the current frame. The luma block vector of an IBC-encoded CU has integer precision. The chroma block vector is also rounded to integer precision. When combined with AMVR, IBC mode can switch between 1-frame-element and 4-frame-element motion vector precision. IBC-encoded CUs are considered a third prediction mode in addition to intra-frame or inter-frame prediction modes. IBC mode is suitable for CUs with a width and height of 64 luma samples or less.

[0159] On the encoder side, hash-based motion estimation is performed for IBC. The encoder performs RD checks on blocks with a width or height no greater than 16 luminance samples. For non-merging modes, a block vector search is first performed using a hash-based search. If the hash search does not return valid candidates, a local search based on block matching is performed.

[0160] In hash-based search, hash key matching (32-bit CRC) between the current block and reference blocks is extended to all allowed block sizes. Hash key calculation for each location in the current image is based on 4x4 sub-blocks. For larger current blocks, a reference block is determined to have a matching hash key if all hash keys in all 4x4 sub-blocks match the hash key at the corresponding reference location. If multiple reference blocks are found to match the current block's hash key, the block vector cost of each matching reference is calculated, and the reference block with the lowest cost is selected.

[0161] In block matching search, the search scope is set to cover both the previous CTU and the current CTU.

[0162] At the CU level, flags are used to signal the IBC mode, and these flags can be used to signal either IBC AMVP mode or IBC skip / merge mode as follows:

[0163] IBC Skip / Merge Mode: The merge candidate index is used to indicate which block vectors from the list of neighboring candidate IBC coding blocks are used to predict the current block. The merge list includes spatial candidates, HMVP candidates, and paired candidates.

[0164] IBC AMVP mode: Encodes block vector differences in the same way as motion vector differences. The block vector prediction method uses two candidates as prediction values, one from the left neighbor and one from the top neighbor (if IBC encoded). When either neighbor is unavailable, a default block vector is used as the prediction value. A signaling flag indicates the block vector prediction index.

[0165] IBC Reference Area

[0166] To reduce memory consumption and decoder complexity, IBC in VVC only allows the reconstruction of predefined regions, including the current CTU region and some regions of the left CTU. Figure 12 The reference area for the IBC mode is shown, where each block represents a 64x64 luminance sample cell.

[0167] Depending on the current encoding CU's location within the current CTU, the following applies:

[0168] If the current block falls within the top-left 64x64 block of the current CTU, then in addition to the samples already reconstructed in the current CTU, the CPR mode can also reference the reference samples in the bottom-right 64x64 block of the left CTU. The current block can also use the CPR mode to reference the reference samples in the bottom-left 64x64 block of the left CTU and the reference samples in the top-right 64x64 block of the left CTU.

[0169] If the current block falls within the upper right 64x64 block of the current CTU, then in addition to the samples already reconstructed in the current CTU, if the brightness position (0, 64) relative to the current CTU has not yet been reconstructed, the current block can also use CPR mode to reference the reference samples in the lower left and lower right 64x64 blocks of the left CTU; otherwise, the current block can also reference the reference samples in the lower right 64x64 block of the left CTU.

[0170] If the current block falls within the lower-left 64x64 block of the current CTU, then in addition to the samples already reconstructed in the current CTU, if the brightness position (64, 0) relative to the current CTU has not yet been reconstructed, the current block can also use CPR mode to reference reference samples in the upper-right and lower-right 64x64 blocks of the left CTU. Otherwise, the current block can also use CPR mode to reference reference samples in the lower-right 64x64 block of the left CTU.

[0171] If the current block falls within the bottom right 64x64 block of the current CTU, it can only use CPR mode to reference already reconstructed samples in the current CTU.

[0172] Regarding the hardware implementation, this restriction allows the use of local on-chip memory to implement the IBC mode.

[0173] Interaction between IBC and other codec tools

[0174] The interaction between IBC mode and other inter-frame coding / decoding tools in VVC (such as Paired Merge Candidate, History-Based Motion Vector Prediction (HMVP), Combined Intra / Inter-Frame Prediction Mode (CIIP), Merge Mode with Motion Vector Difference (MMVD), and Geometric Partitioning Mode (GPM)) is as follows:

[0175] IBC can be used with pairwise merge candidates and HMVP. New pairwise IBC merge candidates can be generated by averaging two IBC merge candidates. For HMVP, IBC movements are inserted into a history buffer for future reference.

[0176] IBC cannot be used in combination with the following inter-frame tools: affine motion, CIIP, MMVD, and GPM.

[0177] When using DUAL_TREE partitions, IBC is not allowed for chroma-coded blocks.

[0178] Unlike in HEVC screen content codec extensions, the current image is no longer included as one of the reference images in reference image list 0 for IBC prediction. The process of deriving motion vectors for IBC mode excludes all adjacent blocks in inter-frame modes, and vice versa. The following IBC design aspects are applied:

[0179] IBC shares the same process as regular MV merging, including having pairwise merge candidates and historical motion predictions, but does not allow TMVP and zero vectors because they are invalid for IBC mode.

[0180] Separate HMVP buffers (5 candidates each) are used for traditional MV and IBC.

[0181] Block vector constraints are implemented in the form of bitstream consistency constraints. The encoder needs to ensure that there are no invalid vectors in the bitstream, and that merging should not be used if a merging candidate is invalid (out of range or 0). Such bitstream consistency constraints are expressed using a virtual buffer as described below.

[0182] For deblocking, IBC is processed as an inter-frame mode.

[0183] If the current block is encoded using the IBC prediction mode, AMVR does not use quarter-image elements; instead, AMVR is signaled to indicate only whether the MV is between image elements or 4-integer image elements.

[0184] The number of IBC merge candidates can be signaled separately in the strip header from the number of regular, sub-block, and geometric merge candidates.

[0185] The concept of a virtual buffer is used to describe the permissible reference region and effective block vector of an IBC prediction mode. Representing the CTU size as ctbSize, the virtual buffer ibcBuf has a width wIbcBuf = 128x128 / ctbSize and a height hIbcBuf = ctbSize. For example, for a CTU size of 128x128, the size of ibcBuf is also 128x128; for a CTU size of 64x64, the size of ibcBuf is 256x64; and for a CTU size of 32x32, the size of ibcBuf is 512x32.

[0186] The size of the VPDU is min(ctbSize, 64) in each dimension, W v =min(ctbSize, 64).

[0187] The virtual IBC buffer ibcBuf is maintained as follows.

[0188] At the beginning of decoding each CTU line, refresh the entire ibcBuf with an invalid value of -1.

[0189] When starting to decode VPDU(xVPDU, yVPDU), set ibcBuf[x][y] = -1 relative to the top left corner of the image, where x = xVPDU%wIbcBuf, ..., xVPDU%wIbcBuf + W v -1;y=yVPDU%ctbSize,…,yVPDU%ctbSize+W v -1.

[0190] After decoding, the CU contains (x, y), relative to the top left corner of the image.

[0191] ibcBuf[ x % wIbcBuf ][ y % ctbSize ] = recSample[ x ][ y ]

[0192] For a block covering coordinates (x, y), it is valid if the following is true for the block vector bv = (bv[0], bv[1]); otherwise, it is invalid:

[0193] ibcBuf[ (x + bv[0])% wIbcBuf] [ (y + bv[1]) % ctbSize ] should not be equal to -1.

[0194] Intra-block copying in Enhanced Compression Model (ECM)

[0195] In ECM, IBC is improved in the following ways.

[0196] IBC Merge / AMVP List Construction

[0197] The IBC merge / AMVP list construction has been modified as follows:

[0198] An IBC merge / AMVP candidate can only be inserted into the IBC merge / AMVP candidate list if it is valid.

[0199] Candidates in the upper right, lower left, and upper left spaces, as well as a pairwise average candidate, can be added to the IBC merge / AMVP candidate list.

[0200] Template-based adaptive reordering (ARMC-TM) was applied to the IBC merge list.

[0201] The HMVP table size for IBC was increased to 25. After deriving up to 20 IBC merge candidates using full pruning, they were reordered together. Following reordering, the top 6 candidates with the lowest template matching cost were selected as the final candidates in the IBC merge list.

[0202] The zero vector candidate filling the IBC merge / AMVP list is replaced with the set of BVP candidates located in the IBC reference region. The zero vector is invalid as a block vector in the IBC merge mode, and therefore, it is discarded as a BVP in the IBC candidate list.

[0203] Three candidates are located at the nearest corner of the reference region, and three additional candidates are determined in the middle of the three sub-regions (A, B, and C), whose coordinates are determined by the width and height of the current block and... and Parameters are determined, such as Figure 13 As shown in the image.

[0204] IBC with template matching

[0205] Template matching is used for IBC in both IBC merge mode and IBC AMVP mode.

[0206] Compared to the IBC-TM merge list used in the regular IBC merge mode, the IBC-TM merge list is modified so that, as in the regular TM merge mode, candidates are selected based on a pruning method with motion distances between candidates. The zero-motion termination is replaced by motion vectors pointing to the left (-W, 0), up (0, -H), and upper left (-W, -H), where W is the width of the current CU and H is the height of the current CU.

[0207] In IBC-TM merging mode, template matching is used to refine the selected candidates before the RDO or decoding process. This allows IBC-TM merging mode to compete with the regular IBC merging mode and signals the TM merging flag.

[0208] In the IBC-TM AMVP mode, up to three candidates are selected from the IBC-TM merge list. Each of these three selected candidates is refined using a template matching method and ranked according to the template matching costs they yield. Then, as usual, only the top two are considered during motion estimation.

[0209] Template matching refinement for both IBC-TM merging and AMVP patterns is fairly straightforward, as the IBC motion vectors are constrained to be (i) integers and (ii) within the reference region, such as Figure 12 As shown in the diagram. Therefore, in IBC-TM merge mode, all thinning is performed with integer precision, and in IBC-TM AMVP mode, they are performed with integer or 4-image-element precision depending on the AMVR value. Such thinning only accesses samples without interpolation. In both cases, the motion vectors refined in each thinning step and the template used must adhere to the constraints of the reference region.

[0210] IBC Reference Area

[0211] The IBC reference area is extended to the two CTU rows above. Figure 14 The reference region used for encoding CTU(m,n) is shown. Specifically, for a CTU(m,n) to be encoded, the reference region contains CTUs with indices (m-2,n-2)…(W,n-2), (0,n-1)…(W,n-1), (0,n)…(m,n), where W represents the maximum horizontal index within the current tile, strip, or image. This setup ensures that for a CTU size of 128, IBC does not require additional memory in the current ETM platform. The per-sample block vector search (or local search) range is limited horizontally to [-(C<<1),C>>2] and vertically to [-C,C>>2] to accommodate the reference region expansion, where C represents the CTU size.

[0212] IBC merging mode with block vector difference

[0213] In ECM, an IBC merging mode with block vector difference is used. The distance set is {1 image element, 2 image elements, 4 image elements, 8 image elements, 12 image elements, 16 image elements, 24 image elements, 32 image elements, 40 image elements, 48 ​​image elements, 56 image elements, 64 image elements, 72 image elements, 80 image elements, 88 image elements, 96 image elements, 104 image elements, 112 image elements, 120 image elements, 128 image elements}, and the BVD direction is two horizontal directions and two vertical directions.

[0214] A basic candidate is selected from the top five candidates in the reordered IBC merge list. Then, based on the SAD cost between the template (one row above and one column to the left of the current block) and its reference for each refinement position, all possible MBVD refinement positions (20x4) for each basic candidate are reordered. Finally, the top 8 refinement positions with the lowest template SAD cost are retained as available positions and thus used for MBVD index encoding.

[0215] IBC adaptation for content captured by the camera

[0216] When adapting IBC for content captured by the camera, the IBC reference range is reduced from 2 CTU lines to 2x128 lines, such as Figure 15 As shown in the diagram. On the encoder side, to reduce complexity, the local search range is set horizontally to [-8,8] and vertically to [-8,8], centered on the first block of vector predictions in the current CU. This encoder modification is not applied to the SCC sequence.

[0217] CIIP combined with TIMD and TM

[0218] In CIIP mode, prediction samples are generated by weighting the inter-prediction signals obtained by merging candidate predictions using CIIP-TM and the intra-prediction signals obtained by using intra-prediction modes derived from TIMD. This method is only applied to coded blocks with an area less than or equal to 1024.

[0219] The TIMD export method is used to export intra-prediction modes from CIIP. Specifically, it selects the intra-prediction mode with the smallest SATD value from the TIMD mode list and maps it to one of 67 regular intra-prediction modes.

[0220] Additionally, it is proposed that if the exported intra-prediction mode is an angle mode, the weights of the two tests (wIntra, wInter) should be modified. For near-horizontal modes (2 <= angle mode index < 34), such as Figure 16A The current block is divided vertically as shown; for near-vertical mode (34 <= angle mode index <= 66), as... Figure 16B The current block is divided horizontally as shown.

[0221] The different sub-blocks (wIntra, wInter) are shown in Table 3. Table 3. Weights used for modifying angle modes

[0222] Using CIIP-TM, a CIIP-TM merge candidate list is constructed for the CIIP-TM pattern. Merge candidates are refined through template matching. CIIP-TM merge candidates are also reordered using the ARMC method, just like regular merge candidates. The maximum number of CIIP-TM merge candidates is two.

[0223] Multiple Hypothesis Prediction (MHP)

[0224] In multi-hypothesis inter-frame prediction mode, in addition to the traditional bidirectional prediction signal, one or more additional motion compensation prediction signals are signaled. The resulting total prediction signal is obtained by weighted summation of samples. The bidirectional prediction signal is then utilized. and the first additional inter-frame prediction signal / hypothesis The obtained prediction signal The following was obtained: (2)

[0225] Based on the mapping presented in Table 4, the weighting factors Specifyed by the new syntax element add_hyp_weight_idx: Table 4. add_hyp_weight_idx and Mapping between

[0226] Similar to the above, more than one additional prediction signal can be used. The resulting overall prediction signal is iteratively accumulated with each additional prediction signal. (3)

[0227] The obtained overall prediction signal is used as the last one. (That is, the one with the largest index n) Within this mode, up to two additional prediction signals can be used (i.e., n is limited to 2).

[0228] The motion parameters of each additional prediction hypothesis can be explicitly signaled by specifying a reference index, a motion vector prediction value index, and a motion vector difference, or implicitly signaled by specifying a merging index. A separate multi-hypothesis merging flag distinguishes between these two signaling modes.

[0229] For inter-frame AMVP mode, MHP is applied only when unequal weights are selected in BCW in bidirectional prediction mode.

[0230] Combining MHP and BDOF is possible; however, BDOF is only applied to the bidirectional prediction signal portion of the predicted signal (i.e., the ordinary first two assumptions).

[0231] Geometric Partitioning Pattern (GPM) in ECM

[0232] GPM with Combined Motion Vector Difference (MMVD)

[0233] The GPM in VVC is extended by applying motion vector refinement on top of the existing GPM unidirectional MV. First, a signaling flag is sent to the GPM CU to indicate whether this mode is used. If this mode is used, each geometric partition of the GPM CU can further determine whether to signal the MVD. If the MVD is signaled for a geometric partition, the motion of the partition is further refined using the signaled MVD information after selecting a GPM merge candidate. All other procedures remain the same as in the GPM.

[0234] MVD is signaled as a pair of distances and directions, similar to MMVD. There are nine candidate distances (1 / 4 image element, 1 / 2 image element, 1 image element, 2 image elements, 3 image elements, 4 image elements, 6 image elements, 8 image elements, 16 image elements), and eight candidate directions (four horizontal / vertical directions and four diagonal directions) in GPM with MMVD (GPM-MMVD). Additionally, when pic_fpel_mmvd_enabled_flag equals 1, MVD is shifted left by 2, as in MMVD.

[0235] GPM with Template Matching (TM)

[0236] Template matching is applied to GPM. When GPM mode is enabled for a CU, a CU-level flag is signaled to indicate whether TM is applied to both geometric partitions. TM is used to refine motion information for each geometric partition. When TM is selected, a template is constructed using adjacent samples on the left, top, or upper left, based on the partition angle, as shown in Table 5. Motion is then refined by minimizing the difference between the current template and the template in the reference image using the same search mode with the merge mode having the half-image element interpolation filter disabled. Table 5

[0237] Table 5 shows the templates for the first and second geometric partitions, where A indicates the use of the top sample point, L indicates the use of the left sample point, and L+A indicates the use of both the left and top samples.

[0238] The GPM candidate list was constructed as follows:

[0239] 1. The interleaved list 0 MV candidates and list 1 MV candidates are directly derived from the regular merged candidate list, where list 0 MV candidates have a higher priority than list 1 MV candidates. A pruning method with an adaptive threshold based on the current CU size is applied to remove redundant MV candidates.

[0240] 2. The staggered List 1 MV candidates and List 0 MV candidates are further derived directly from the regular merged candidate list, where List 1 MV candidates have a higher priority than List 0 MV candidates. The same pruning method with adaptive thresholds is also applied to remove redundant MV candidates.

[0241] 3. Fill the zero MV candidate list until the GPM candidate list is full.

[0242] GPM-MMVD and GPM-TM specifically enable a single GPM CU. This is done by first signaling the GPM-MMVD syntax. When both GPM-MMVD control flags are false (i.e., GPM-MMVD is disabled for both GPM partitions), the GPM-TM flag is signaled to indicate whether template matching should be applied to both GPM partitions. Otherwise (if at least one GPM-MMVD flag is true), the GPM-TM flag is presumed to be false.

[0243] GPM with inter-frame prediction and intra-frame prediction

[0244] In GPM with inter-frame prediction and intra-frame prediction, the final prediction samples are generated by weighting the inter-frame prediction samples and intra-frame prediction samples of each GPM-separated region. Inter-frame prediction samples are derived from the inter-frame GPM, while intra-frame prediction samples are derived from the intra-frame prediction mode (IPM) candidate list and an index signaled by the encoder. The IPM candidate list size is predefined as 3. Available IPM candidates are the parallel angle mode (parallel mode) for GPM block boundaries, the vertical angle mode (vertical mode) for GPM block boundaries, and the planar mode, such as... Figures 17A to 17C As shown. Furthermore, as... Figure 17D The GPM shown, with both intra-frame prediction and intra-frame prediction, is limited to reduce signaling overhead for IPM and avoid increasing the size of the intra-frame prediction circuitry on the hardware decoder. Additionally, direct motion vectors and IPM storage are introduced on the GPU's mixing region to further improve encoding and decoding performance.

[0245] In IPM export based on DIMD and adjacent modes, parallel modes are registered first. Therefore, if no identical IPM candidates exist in the list, the maximum two IPM candidates from the decoder-side intra-mode export (DIMD) method and / or adjacent block export can be registered. As for adjacent mode export, there are at most five available adjacent block positions; however, they are limited by the angle of the GPM block boundary as shown in Table 6, which has been used for GPM with template matching (GPM-TM). Table 6

[0246] Table 6 shows the positions of available neighboring blocks for IPM candidate derivation based on the angles of the GPM block boundaries. A and L represent the top and left sides of the predicted block.

[0247] Intra-GPM frames can be combined with GPM frames that have motion vector differences (GPM-MMVD). TIMD is used as an IPM candidate within GPM frames to further improve encoding and decoding performance. Parallel mode can be registered first, followed by TIMD, DIMD, and IPM candidates for adjacent blocks.

[0248] Template-match-based reordering for GPM splitting patterns

[0249] In template-match-based reordering for GPM split patterns, given the motion information of the current GPM block, the corresponding TM cost value for the GPM split pattern is calculated. Then, all GPM split patterns are reordered in ascending order based on their TM cost values. Instead of sending GPM split patterns, a signal is sent using Golomb-Rice codes to indicate the exact index of the GPM split pattern in the reordering list.

[0250] The reordering method for GPM split patterns is a two-step process performed after generating the corresponding reference templates for the two GPM partitions in the coding unit, as follows:

[0251] • Extend the GPM partition edge to the reference templates of the two GPM partitions to generate 64 reference templates and calculate the corresponding TM cost for each of the 64 reference templates;

[0252] • Reorder the GPM split patterns in ascending order based on their TM cost values ​​and mark the top 32 as available split patterns.

[0253] The edges on the template extend from the edges of the current CU, such as Figure 18 As shown, however, the GPM blending process is not used in template areas that cross the edge.

[0254] After using TM cost ascending reordering, signal the index.

[0255] Intra-frame template matching

[0256] Intra-Template Matching Prediction (Intra TMP) is a special intra-prediction mode that copies the best prediction block from the reconstructed portion of the current frame, with its L-shaped template matching the current template. For a predefined search range, the encoder searches for the template most similar to the current template in the reconstructed portion of the current frame and uses the corresponding block as the prediction block. The encoder then signals the use of this mode, and the same prediction operation is performed on the decoder side.

[0257] By comparing the L-shaped causal neighbors of the current block with Figure 19 The predicted signal is generated by matching another block within a predefined search region, which includes:

[0258] R1: Current CTU

[0259] R2: Top Left CTU

[0260] R3: Above CTU

[0261] R4: Left CTU

[0262] The sum of absolute differences (SAD) is used as the cost function.

[0263] Within each region, the decoder searches for the template with the minimum SAD relative to the current template and uses its corresponding block as the prediction block.

[0264] The dimensions of all regions (SearchRange_w, SearchRange_h) are set to be proportional to the block dimensions (BlkW, BlkH) to have a fixed number of SAD comparisons per pixel. That is:

[0265]

[0266]

[0267] Here, 'a' is a constant that controls the trade-off between gain and complexity. In practice, 'a' equals 5.

[0268] For CUs with a width and height less than or equal to 64, enable the intra-frame template matching tool. The maximum CU size used for intra-frame template matching is configurable.

[0269] When DIMD is not used in the current CU, the intra-template matching prediction mode signals this at the CU level via a dedicated flag.

[0270] Fusion for Template-Based Intra-Frame Mode Export (TIMD)

[0271] For each intra-prediction mode in the MPM, the SATD between the predicted and reconstructed samples of the template is calculated. The top two intra-prediction modes with the minimum SATD are selected as TIMD modes. These two TIMD modes are weighted and fused after applying the PDPC procedure, and such weighted intra-prediction is used to encode the current CU. Position-dependent intra-prediction combination (PDPC) is included in the derivation of the TIMD modes.

[0272] The costs of the two chosen modes are compared to the threshold. In the test, cost factor 2 is applied as follows:

[0273] .

[0274] If the condition is true, then apply fusion; otherwise, use only mode 1.

[0275] The weights of the patterns are calculated from their SATD costs as follows:

[0276]

[0277]

[0278] The division operation is performed using the same lookup table (LUT)-based integration scheme used by CCLM.

[0279] Localized lighting compensation (LIC)

[0280] LIC is an inter-frame prediction technique that models the local illumination variation between the current block and its predicted block as a function of the local illumination variation between the current block template and the reference block template. The parameters of this function can be scaled... and offset This indicates that it forms a linear equation, that is, To compensate for lighting variations, p[x] is the reference sample point pointed to by the MV at position x on the reference image. When surround motion compensation is enabled, surround offset will be used to crop the MV. This is because it can be derived based on the current block template and the reference block template. and Therefore, apart from signaling the LIC flag for AMVP mode to indicate the use of the LIC, they do not require signaling overhead.

[0281] The local illumination compensation proposed in JVET-O0066 was used for unidirectional prediction of inter-frame CU with the following modifications:

[0282] Intra-frame neighbor samples can be used in LIC parameter export;

[0283] Disable LIC for blocks with fewer than 32 luminance samples;

[0284] For both non-subblock mode and affine mode, LIC parameter export is performed based on the template block samples corresponding to the current CU, rather than the partial template block samples corresponding to the first top-left 16x16 element; and

[0285] Samples of a reference block template are generated by using a MC with block MV, without rounding them to integer image element precision.

[0286] OBMC

[0287] When applying OBMC, neighboring block motion information with weighted prediction is used to refine the top and left boundary pixels of the CU, as described in JVET-L0101.

[0288] The following conditions should not be used for OBMC:

[0289] When OBMC is disabled at the SPS level;

[0290] When the current block has intra-frame mode or IBC mode;

[0291] When the current block applies a LIC; and

[0292] When the area of ​​the current luminance block is less than or equal to 32.

[0293] Sub-block boundary OBMC is performed by applying the same blending to the top, left, bottom, and right sub-block boundary pixels using motion information from adjacent sub-blocks. For sub-block-based codecs, enable:

[0294] Affine AMVP mode;

[0295] Affine merging mode and sub-block-based temporal motion vector prediction (SbTMVP); and

[0296] Bilateral matching based on sub-blocks.

[0297] When using OBMC mode in CIIP mode with LMCS, inter-frame blending is performed before LMCS mapping of inter-frame samples. LMCS is applied to the blended inter-frame samples, which are then combined with intra-frame samples where LMCS is applied in CIIP mode.

[0298] in, This represents the sample points predicted by the motion of the current block in the original domain. This represents the sample points predicted in the mapping domain. This represents the sample points predicted by the motion of neighboring blocks in the original domain, and and It's the weight.

[0299] OBMC based on template matching

[0300] In the template matching-based OBMC scheme, instead of directly using weighted prediction, the predicted value of the CU boundary sample derivation method is determined based on the template matching cost, including using only the motion information of the current block, or using the motion information of adjacent blocks, or a hybrid mode.

[0301] In this scheme, for each 4x4 block at the top CU boundary, the template size is equal to 4x1. If N adjacent blocks have the same motion information, the template size is increased to 4N×1 because MC operations can be processed in one go. For each left block of size 4x4 at the left CU boundary, the left template size is equal to 1x4 or 1x4N (…). Figure 20 ).

[0302] For each 4x4 top block (or N groups of 4x4 blocks), follow these steps to derive the predicted values ​​for the boundary samples.

[0303] Taking block A as the current block and its adjacent block AboveNeighbor_A as an example, the operations on the left-hand blocks are performed in the same way.

[0304] First, based on the following three types of motion information, the three template matching costs (Cost1, Cost2, Cost3) are measured by the SAD between the reconstructed samples of the templates derived from the MC process and their corresponding reference samples:

[0305] Cost1 is calculated based on the motion information of A.

[0306] Cost2 is calculated based on the motion information of AboveNeighbor_A.

[0307] Cost3 is calculated based on a weighted prediction of the motion information of A and AboveNeighbor_A, where the weighting factors are 3 / 4 and 1 / 4, respectively.

[0308] Secondly, a method is selected to calculate the final prediction result of the boundary sample points by comparing Cost1, Cost2 and Cost3.

[0309] The original MC result using the motion information of the current block is represented as Pixel1, and the MC result using the motion information of neighboring blocks is represented as Pixel2. The final prediction result is represented as NewPixel.

[0310] If Cost1 is the minimum, then NewPixel(i,j) = Pixel1(i,j).

[0311] if If so, then use mixed mode 1.

[0312] For a luminance block, the number of mixed pixel rows is 4.

[0313]

[0314]

[0315]

[0316]

[0317] For chroma blocks, the number of mixed pixel rows is 1.

[0318]

[0319] if If so, then use hybrid mode 2.

[0320] For a luminance block, the number of mixed pixel rows is 2.

[0321]

[0322]

[0323] For chroma blocks, the number of mixed pixel rows / columns is 1.

[0324]

[0325] Otherwise, use mixed mode 3.

[0326] For a luminance block, the number of mixed pixel rows is 4.

[0327]

[0328]

[0329]

[0330] For chroma blocks, the number of mixed pixel rows is 1.

[0331]

[0332] Adaptive reordering of merge candidates with template matching (ARMC-TM)

[0333] The reordering method is applied to the regular merge pattern, the template matching (TM) merge pattern, and the affine merge pattern (excluding SbTMVP candidates). For the TM merge pattern, the merge candidates are reordered before the refinement process.

[0334] After constructing the merge candidate list, the merge candidates are divided into several subgroups. For the regular merge mode and the TM merge mode, the subgroup size is set to 5. For the affine merge mode, the subgroup size is set to 3. The merge candidates in each subgroup are reordered incrementally based on the cost value of template matching. For simplicity, the merge candidates in the last, but not the first, subgroup are not reordered.

[0335] The template matching cost of merging candidates is measured by the sum of absolute differences (SAD) between the template samples of the current block and their corresponding reference samples. The template includes a set of reconstructed samples adjacent to the current block. The reference samples of the template are located using the motion information of the merging candidates.

[0336] When merging candidates utilize bidirectional prediction, the reference samples for the template of the merging candidates are also generated through bidirectional prediction, such as... Figure 21 As shown in the image.

[0337] For sub-block-based merge candidates with a sub-block size equal to Wsub × Hsub, the upper template includes several sub-templates of size Wsub × 1, and the left template includes several sub-templates of size 1 × Hsub. For example... Figure 22 As shown, the motion information of the sub-blocks in the first row and first column of the current block is used to derive the reference samples for each sub-template.

[0338] Direct block vectors for chroma blocks

[0339] Direct block vectors are used for chroma blocks in a dual-tree slice. When the chroma dual-tree is activated, a signal flag is sent to indicate whether the chroma block is encoded using IBC mode. Figure 23 If one of the luma blocks in the five locations shown is encoded in IBC or intra-frame TMP mode, its block vector is scaled and used as the block vector for the chroma block. Template matching is used to perform block vector scaling.

[0340] While existing IBC schemes offer significant improvements to intra-frame coding and decoding in ECM, there is room for further performance enhancement. Simultaneously, some aspects of existing Convolutional Cross-Component Model (CCCM) patterns need simplification for efficient codec hardware implementations or improvement for better coding and decoding efficiency. Furthermore, a better balance needs to be struck between implementation complexity and its coding and decoding efficiency benefits.

[0341] In order to address the problems mentioned above, this disclosure provides methods for further improving the existing design of IBC. The main features of the techniques presented in this disclosure are generally summarized below.

[0342] The CCCM tool is used to filter IBC predictions. Filtered intra-block copying (FIBC) is a special intra-prediction mode that applies filters to IBC-based prediction blocks to increase prediction accuracy and adapt the characteristics of the copied blocks to the local neighborhood.

[0343] In FIBC, training samples can be adjacent to the current block. It is known that references from local regions can improve the accuracy of predictions.

[0344] In FIBC, training samples may not be adjacent to the current block. It is known that references from non-local regions can also improve the accuracy of predictions.

[0345] In FIBC, only one assumption can be used: that is, the best matching block that results in the minimum matching cost is selected as the final prediction.

[0346] In FIBC, multiple assumptions can also be used.

[0347] It should be understood that the accompanying drawings in this disclosure can be combined with all the examples mentioned in this disclosure, and the disclosed methods can be applied independently or in combination.

[0348] Filtered intra-block copy (FIBC)

[0349] According to one or more embodiments of this disclosure, the CCCM tool is used to filter IBC predictions. Different methods can be used to achieve this goal. Existing CCCM modes apply various filters to predict chroma sample values ​​based on corresponding luma sample values. Unlike CCCM, FIBC is a special intra-frame prediction mode that applies filters to IBC-based prediction blocks to predict target luma or chroma samples of the current block based on corresponding luma or chroma samples of a reference block, thereby increasing prediction accuracy and adapting the characteristics of the copied block to the local neighborhood.

[0350] According to one or more embodiments of this disclosure, the IBC prediction is further filtered. Different methods can be used to achieve this goal. FIBC is a special intra-frame prediction mode that applies filters to IBC-based prediction blocks to increase prediction accuracy and adapt the characteristics of the copied blocks to the local neighborhood.

[0351] According to one or more embodiments of this disclosure, reconstructed luminance / chrominance samples on the template region of a reference block are used as input to a filter during the training phase, and corresponding reconstructed luminance / chrominance samples in the template region of the current block are the target. In one example, Figure 25 A filter shape (cross-shaped) and training region for a reference block are shown. It should be understood that for this filter shape, both the template region and the boundary region of the template region can be part of the training region of the reference block. Reconstructed samples in the boundary regions can be used for training when available, and they are filled with the nearest available samples when unavailable. Conversely, the training region of the current block can be determined as the template region of the current block. During the filtering phase (where the filter coefficients have been trained / determined through the training phase), the filter can be applied to the corresponding sample values ​​of the reference block and the boundary region of the reference block to predict each of the sample values ​​of the current block.

[0352] According to one or more embodiments of this disclosure, the predicted samples can be used as input to a filter during the prediction process. In one example, Figure 26 The diagram shows the prediction samples used to generate the prediction process, where gray areas (as shown by a pattern with sparse dots) are the predicted samples, small patches without patterns represent reconstructed samples, and patterns with dense dots represent the locations to be predicted, which will become the predicted samples after being predicted.

[0353] According to one or more embodiments of this disclosure, filter coefficients (i.e., parameters) are derived using regression-based MSE minimization techniques (i.e., LDL decomposition) that exist in ECM and are utilized by other tools such as CCCM.

[0354] According to one or more embodiments of this disclosure, a convolutional N-tap (N is an integer and greater than 1) filter may include an (N-1-M) tap space term, an M nonlinear term, and a bias term. The (N-1-M) tap space term corresponds to terms derived from, for example, ... Figure 27 The values ​​of neighboring samples of the reconstructed reference block shown are, for example, luminance samples (i.e., L0, L1, ..., L8). In this example, the formula for each new predicted luminance sample is as follows:

[0355] in Is with Correlated coefficients This is the offset (i.e., 1 << (bitDepth-1)). The reference brightness sample value of the top-left sample adjacent to the current block can be used as... Values. The location and number of spatial and nonlinear terms can differ. Figure 28Examples of different shapes / numbers of filter taps are shown. For another example, different positions and numbers are used, as shown in the table below.

[0356] According to one or more embodiments of this disclosure, the filter shape may be rectangular. (N and M are integers and greater than 1). Figure 29 Examples of filter taps of different shapes / numbers are shown. According to one or more embodiments of this disclosure, the corresponding center point (C) location may be different. The corresponding center point (C) may also be referred to as the location to be predicted. Examples of different locations of the center point (C) are shown in... Figure 29 As shown in the image.

[0357] like Figure 29 As shown, the positions to be predicted in each filter shape 0-11 are identified by the letter "C". For example, shapes 0 and 1 are both rectangles with a width of 3 lines and a height of 3 lines, with shape 0 identified as having a position to be predicted at the lower right of the rectangle (i.e., the third row and third column), and shape 1 identified as having a different position to be predicted at the center of the rectangle (i.e., the second row and second column). Similarly, shape 2 is a rectangle with a width of 3 lines and a height of 4 lines, and is identified as having a position to be predicted at the third row and second column. Shapes 3 and 4 are both rectangles with a width of 4 lines and a height of 4 lines, with shape 3 identified as having a position to be predicted at the third row and third column, and shape 4 identified as having a different position to be predicted at the lower right of the rectangle (i.e., the fourth row and fourth column). Shapes 5 and 6 are both rectangles with a width of 2 lines and a height of 8 lines. Shape 5 is identified as having a predicted position at the bottom right of the rectangle (i.e., row 8 and column 2), and shape 6 is identified as having different predicted positions at row 5 and column 2. Shapes 7 and 8 are both rectangles with a width of 8 lines and a height of 2 lines. Shape 7 is identified as having a predicted position at the bottom right of the rectangle (i.e., row 2 and column 8), and shape 8 is identified as having different predicted positions at row 2 and column 5. Shape 9 is a rectangle with a width of 6 lines and a height of 2 lines, and is identified as having a predicted position at the bottom right of the rectangle (i.e., row 2 and column 6). Shapes 10 and 11 are both rectangles with a width of 2 lines and a height of 6 lines. Shape 10 is identified as having a predicted position at row 4 and column 2, and shape 11 is identified as having different predicted positions at the bottom right of the rectangle (i.e., row 6 and column 2).

[0358] The width / height of the filter shape and the location to be predicted in this disclosure are not limited to... Figure 29The shapes shown are illustrated. Appropriate filter shapes from among the different filter shapes described in this disclosure can be used to perform FIBC to further increase prediction accuracy.

[0359] According to one or more embodiments of this disclosure, the filter shape may be rectangular and exclude the lower right sample point, and the numbers used are... (N and M are integers and greater than 1). Figure 30 Examples of different shapes / numbers of filter taps are shown, where C is the position to be predicted.

[0360] like Figure 30 As shown, filter shapes 0-12 are rectangles with different widths and / or different heights, and do not have a lower right sample / position, which is the location to be predicted and identified by the letter "C" (see the unpatterned block). For example, shape 0 has a width and height of 3 lines; shape 1 has a width and height of 3 lines and 2 lines; shape 2 has a width and height of 3 lines and 4 lines; shape 3 has a width and height of 4 lines and 3 lines; shape 4 has a width and height of 4 lines; shape 5 has a width and height of 2 lines and 8 lines; shape 6 has a width and height of 2 lines and 3 lines; shape 7 has a width and height of 2 lines and 4 lines; shape 8 has a width and height of 8 lines and 2 lines; shape 9 has a width and height of 6 lines and 2 lines; shape 10 has a width and height of 4 lines and 2 lines; shape 11 has a width and height of 2 lines and 5 lines; and shape 12 has a width and height of 2 lines and 6 lines. The width and height of the rectangle without a lower right sample in this disclosure are not limited to... Figure 30 The shape shown.

[0361] FIBC methods can be used as follows Figures 29-30 The filter shape shown is used for video decoding / encoding.

[0362] According to one or more embodiments of this disclosure, the number of filter taps may be predefined or signaled / switched at the SPS / DPS / VPS / SEI / APS / PPS / PH / SH / area / CTU / CU / subblock / sample level.

[0363] According to one or more embodiments of this disclosure, the template size and shape may be the same as those in the intra-frame TMP, and the template size used for training is four rows above and to the left of the current block, depending on their availability.

[0364] According to one or more embodiments of this disclosure, the template size used for training is up to 5 rows above and to the left of the current block, depending on their availability.

[0365] According to one or more embodiments of this disclosure, the template size and shape may be the same as in CCCM, and the template size used for training is 6 rows above and to the left of the current block, depending on their availability.

[0366] According to one or more embodiments of this disclosure, the template size for training can be N rows above and to the left of the current block, depending on their availability, where N is an integer.

[0367] According to one or more embodiments of this disclosure, the template size for training can be N rows above the current block, depending on their availability, where N is an integer.

[0368] According to one or more embodiments of this disclosure, the template size for training can be N rows to the left of the current block, depending on their availability, where N is an integer.

[0369] According to one or more embodiments of this disclosure, the size of the template used for training can depend on the filter shape. In one example, if the height of the filter shape is greater than its width, the size of the template used for training can be N rows above the current block, depending on their availability, where N is an integer. Similarly, in another example, if the width of the filter shape is greater than its height, the size of the template used for training can be N rows to the left of the current block, depending on their availability, where N is an integer. That is, instead of an L-shape, the template used to train the filter can be a rectangular shape, and the width of the template (in the case above the current block) or the height of the template (in the case to the left of the current block) can depend on the width or height of the current block. Therefore, the corresponding template associated with the reference block will be above or to the left of the reference block, which has the same size and shape. Such templates for training can be used in FIBC methods.

[0370] According to one or more embodiments of this disclosure, the reference sample / template region of the reference block / template region of the current block may be predefined or signaled / switched at different encoding levels (such as SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / subblock / sample level).

[0371] According to one or more embodiments of this disclosure, location information can be used to calculate model parameters, including utilizing horizontal / vertical / diagonal distances and their nonlinear terms. One or more pieces of location information can be used for this purpose. In one example, the location-based parameters are related to the vertical and horizontal coordinates (Xc, Yc) of the center brightness sample point and are calculated relative to the top-left coordinates (Xtl, Ytl) of the block, for example, Xc-Xtl+Yc-Ytl. In another example, the location-based parameters are related to the vertical and horizontal coordinates (Xc, Yc) of the center brightness sample point and are calculated relative to the top-left coordinates (Xtl, Ytl) of the block, for example, Xc-Xtl+Yc-Ytl, Xc-Xtl, Yc-Ytl. In another example, the position-based parameters are related to the vertical and horizontal coordinates (Xc, Yc) of the center brightness sample and are calculated relative to the top-left coordinate (Xtl, Ytl) of the block, for example, (Xc-Xtl+Yc-Ytl) / N, where N is a predefined number, for example, 2. In yet another example, the position-based parameters are related to the vertical and horizontal coordinates (Xc, Yc) of the center brightness sample and are calculated relative to the top-left coordinate (Xtl, Ytl) of the block, for example, (Xc-Xtl+Yc-Ytl) / N1, (Xc-Xtl) / N2, (Yc-Ytl) / N3, where N1 to N3 are predefined numbers, for example, 2, 3, and 4. In yet another example, the position-based nonlinear term is expressed as a power of 2 of the horizontal / vertical / diagonal distance, for example, , , , where (Xc, Yc) are the vertical and horizontal coordinates of the center brightness sample point, and (Xtl, Ytl) are the top-left coordinates.

[0372] According to one or more embodiments of this disclosure, an enable flag can be signaled in the bitstream to indicate the FIBC mode being used. The enable flag can be signaled at different encoding levels (such as SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / subblock / sample level).

[0373] According to one or more embodiments of this disclosure, a mode flag can be derived at the decoder to save bit overhead, rather than explicitly signaling the selected mode flag.

[0374] According to one or more embodiments of this disclosure, no additional control flags are required, and the FIBC mode will be derived under some predefined conditions (e.g., a specific mode, a specific block size, a specific partition). When the predefined conditions are matched, the FIBC mode will be derived based on previously decoded information.

[0375] According to one or more embodiments of this disclosure, samples in regions not adjacent to the current block can be used to derive the model of the current block. In one embodiment, a candidate region list with N candidates can be constructed by sequentially examining potential M×M regions. If an examined region is available, it is added to the candidate region list. For example, a candidate region list with 6 candidates can be constructed by sequentially examining potential 8x8 regions. The top-left position of a potential 8x8 region is predetermined. , where xStep = Max(width, 16) and yStep = Max(height, 16). Figure 31 Some possible locations of the candidate regions are shown.

[0376] According to one or more embodiments of this disclosure, a non-adjacent candidate region with N candidates can be constructed by considering the positions and inclusion order of the spatially non-adjacent candidates from a set of two spatially non-adjacent candidate regions in an inter-frame merging mode. If the examined region is available, it is added to the candidate region list. Figure 32 Some possible locations for the candidates are shown.

[0377] According to one or more embodiments of this disclosure, inherited parameters of FIBC from previously decoded TB / CB / strip / picture / sequence levels can be used in the current block. According to one or more embodiments of this disclosure, a control flag is signaled at the TB / CB / strip / picture / sequence level to indicate whether inherited FIBC signaling is enabled or disabled. When the signaling control flag is enabled, the inherited FIBC flag is further signaled to the decoder to indicate whether inherited FIBC is used at the signaled level.

[0378] According to one or more embodiments of this disclosure, derived parameters from previously decoded TB / CB / strip / picture / sequence level FIBCs can be stored and used as the current FIBC (referred to as the inherited FIBC). In one embodiment, a history-based FIBC (H-FIBC) table can be maintained, similar to an HMVP table. In one embodiment, an index value can be signaled in the bitstream to indicate which candidate model in the H-FIBC table is selected. In one embodiment, the corresponding table can be updated after decoding an FIBC encoded block. In one embodiment, the size of the H-FIBC table is N. N is an integer (e.g., 4, 5, 6, 7).

[0379] According to one or more embodiments of this disclosure, the FIBC flag can be inherited from an IBC HMVP candidate.

[0380] According to one or more embodiments of this disclosure, the FIBC flag can be inherited from the IBC space MVP from a spatially adjacent CU.

[0381] According to one or more embodiments of this disclosure, the FIBC flag can be inherited from the IBC time MVP from the time-corresponding CU.

[0382] Multiple Hypothesis FIBC

[0383] According to one or more embodiments of this disclosure, more than one prediction block candidate is used and weighted to generate the final prediction for the current block. Assume N prediction block candidates are used.

[0384] Predicted block candidate export

[0385] In one embodiment, candidate prediction blocks are searched and selected based on a criterion of minimizing template matching cost; that is, the top N candidates that result in the minimum template matching cost are selected. Template matching cost may not be limited to SAD (sum of absolute differences) and SSE (sum of squared errors).

[0386] In one embodiment, candidate prediction blocks can be selected based on a predefined pattern (i.e., a planar pattern).

[0387] In one embodiment, predicted block candidates can be selected based on predefined patterns of neighbors (i.e., top predefined pattern, left predefined pattern).

[0388] Fixed Multiple Hypothesis FIBC

[0389] In this embodiment, the weighting factors for generating the final prediction block are predefined and fixed on both the encoder and decoder sides. As an example, equal weighting factors, i.e., 1 / N, can be used for all candidate blocks.

[0390] Adaptive Multihypothesis Intra-FIBC

[0391] To adapt to the diverse characteristics of video content, an adaptive multi-hypothesis intra-FIBC method is also proposed.

[0392] In one embodiment, the weighting factor can be derived based on the template matching cost. The template matching costs of N candidates are represented as... , , ..., The weighting factors are calculated as follows. (4)

[0393] It should be noted that template matching costs can be measured using (but are not limited to) SAD and SSE.

[0394] In another embodiment, the weighting factor can be derived / switched based on the block size or syntax element signaled at different coding levels (such as SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / subblock / sample level).

[0395] In another embodiment, weighting factors can be derived on the encoder side and then signaled to the decoder in the bitstream. The N candidate prediction blocks are represented as... , , ..., And by representing the current block as X, the weighting factor can be solved using the following equation: (5)

[0396] Equation (5) can be solved using the Wiener-Hopf equation as an ALF. The derived filter coefficients are then quantized to integer type and signaled at the block level.

[0397] In another embodiment, weighting factors are derived based on a template, and these derived weighting factors are applied to prediction block candidates to generate the final prediction block. The template for the prediction candidate is represented as... , , ..., And by representing the current block as T, the weighting factor can then be derived using the following equation: (6)

[0398] Equation (6) can be solved using the Wiener-Hopf equation. Then, the final predicted block can be computed as... ,in This represents the i-th prediction block candidate.

[0399] The FIBC model leverages nonlocal correlation to improve prediction accuracy, where similar blocks are searched and used to generate the final prediction block. In this embodiment, a combination of nonlocal mean filtering and multi-hypothesis FIBC is proposed, as described below. In the first step, as in FIBC, N prediction block candidates are searched and identified. In the second step, weighting factors are calculated as follows. (7)

[0400] in, The distance between the template of the i-th prediction block candidate and the template of the current block is measured, h is used as the weighting degree, and Z[i] is the normalization constant: (8)

[0401] In order to calculate the weighting factors in equation (7), the strength of the weights must first be determined. Several methods are proposed in this disclosure to determine the weighting strength.

[0402] In the first method, a candidate list of weighted strength values, including some typical weighted strength values, is defined and fixed on both the encoder and decoder sides. On the encoder side, rate-distortion optimization is used to examine the weighted strength values, and the optimal weighted strength value is identified and signaled to the decoder side in the bitstream.

[0403] In the second method, the templates of the predicted block candidates and the template of the current block are used to estimate the weighted intensity value. The template of the predicted candidate is represented as... , , ..., And represent the current block as T. Then, the weighted strength value can be solved using the following equation: (9)

[0404] In the third method, the QP value and variance of the template of the current block can be used to estimate the weighted intensity value, that is, the relationship between the weighted intensity value, QP value and template variance can be fitted offline.

[0405] To better utilize the nonlocal correlations in FIBC, this embodiment employs singular value decomposition (SVD) to generate the final predicted block from the predicted block candidates. The width and height of the current block are represented as W and H, respectively, and the area of ​​the current block is represented as d = H × W.

[0406] Step 1. As performed in FIBC, search and identify K candidate prediction blocks. .

[0407] Step 2. The K predicted block candidates of the current block y are constructed into a block group G and arranged as a matrix: (10)

[0408] in It has A matrix of size (by arranging each candidate in group G as a column vector).

[0409] Step 3. For the matrix Perform SVD decomposition. (11)

[0410] Step 4. For the singular value matrix Apply soft-threshold operations. (12)

[0411] in Using threshold Shrink A function of the diagonal elements. For The k-th diagonal element in the array is formed by a nonlinear function. By level Shrink: (13)

[0412] It is a singular value that shrinks at the diagonal position. The matrix formed.

[0413] Step 5. Perform inverse SVD to obtain the filtered patch group. (14)

[0414] One of the key steps is determining the threshold value for each diagonal element in step 4. In this invention, the threshold value is calculated as follows. The threshold is estimated for each group of image patches using the following equation: (15)

[0415] in It is the standard deviation of the noise, and Let be the standard deviation of the original block in the k-th dimension of the SVD space of group G. The following estimates the deviation of the original block in the SVD space. (16)

[0416] in yes The k-th singular value. When When it is zero, skip the soft thresholding operation. Additionally, utilize... and The parameterized power function uses the bias of the prediction block to estimate the bias of the noise. (17)

[0417] in The calculation is as follows: (18)

[0418] Here Represents the candidate vector of the prediction block The i-th pixel.

[0419] Multiple Assumptions FIBC Signaling

[0420] In this disclosure, the proposed multi-hypothesis FIBC can be used as an alternative to the current FIBC mode, or the encoder can adaptively select either the FIBC mode or the multi-hypothesis FIBC mode.

[0421] In one embodiment, the proposed multiple hypothesis FIBC is used as an alternative to the current FIBC model, i.e., multiple hypotheses are always used for prediction.

[0422] In another embodiment, one of the multi-hypothesis FIBC methods described above is used in conjunction with the current FIBC mode. A signal flag is sent in the bitstream to indicate whether the multi-hypothesis FIBC mode is applied to the CU.

[0423] In another embodiment, more than one of the multi-hypothesis FIBC methods described in the preceding sections is used in conjunction with the current FIBC mode. First, a signal flag is sent in the bitstream to indicate whether the multi-hypothesis FIBC mode is applied. Then, an index is signaled to indicate which of the multi-hypothesis FIBC methods is applied to the CU.

[0424] Coordination of filters for FIBC and FTMP modes

[0425] TMP prediction can also be filtered using CCCM tools, a process known as Filtered Template Matching Prediction (FTMP) mode. The FTMP process is identical to the FIBC process, except that FTMP does not require a signaled block vector from the encoder to find the reference block. Instead, in FTMP, the reference block is determined on the decoder side by searching for the most similar L-shaped template in the reconstructed portion of the current frame, and this corresponding block is used as the reference block for the current block to be predicted. In other words, the L-shaped template associated with the reference block is the template most similar to the L-shaped template associated with the current block in the reconstructed portion of the frame. This operation for determining the reference block is the same as in intra-frame TMP mode. After determining the reference block, the same filtering process as in FIBC mode is applied to predict the target luma or chroma samples of the current block based on the corresponding luma or chroma samples of the reference block, respectively. For example, as... Figure 25 As shown, a cross-shaped filter can be applied to the corresponding sample values ​​(sample values ​​of the reference block and the boundary region of the reference block) to predict each sample value of the current block.

[0426] According to one or more embodiments of this disclosure, the same filter shape and / or template region can be applied to prediction in both FIBC and FTMP modes. For example, before deciding which mode to apply, the decoder and / or encoder can try both FIBC and FTMP modes with the same filter shape and / or template region to achieve better performance or reduced cost. Different methods can be used to achieve this goal.

[0427] In the first example, it is also proposed that the filter operations used in FTMP mode be applied to FIBC mode. In one example, the 6-tap filter (a cross-shaped filter with 5 spatial components and a bias term) used in FTMP mode and the template regions used for training (4 rows in width above the current block, depending on their availability, depending on the samples) can also be applied to FIBC mode in the same CU.

[0428] In the second example, it is also proposed that the filter operations used in FIBC mode be applied to FTMP mode. In one example, the 2-tap filter (a monosample filter with one spatial component and a bias term) used in FIBC mode and the template region used for training (one row to the left of the current block, depending on their availability) can also be applied to FTMP mode in the same CU.

[0429] Coordination of filters for FIBC, FTMP and CCCM modes

[0430] In Convolutional Cross-Component Model (CCCM) mode, filters are used to predict chroma sample values ​​based on corresponding luma sample values. In CCCM mode, the set of chroma sample values ​​in the reconstructed region of the current block to be predicted, along with the corresponding luma sample values, are used to determine the filter coefficients of the CCCM filter. In one example, the chroma sample values ​​to be predicted and their corresponding luma sample values ​​are isotopic sample values. Although the training results of the filter coefficients may differ, the same filter shape and / or template region can be reused across CCCM, FIBC, and FTMP for better performance / reduced cost. In one example, the filter shape and / or template region can be informed to the decoder by an encoder-based signal. In another example, the filter shape and / or template region can be derived by the encoder based on predetermined rules (e.g., from a predefined candidate set).

[0431] According to one or more embodiments of this disclosure, the same filter shape and / or template region can be applied to at least two of the FIBC, FTMP, and CCCM modes. Different methods can be used to achieve this goal.

[0432] In the first example, it is also proposed that the filter operations used in CCCM mode be applied to FIBC mode. In one example, the 7-tap filter (a cross with 5 spatial components, nonlinear terms, and bias terms) used in CCCM mode and the template region used for training (6 rows to the upper left of the current block, depending on their availability) can also be applied to FIBC mode in the same CU.

[0433] In the second example, it is proposed that one of the filter operations used in CCCM mode be applied to both FIBC and FTMP modes. In one example, an 11-tap filter (with 9 spatial components, a nonlinear term, and a bias term) is used for CCCM mode. Squares and template regions for training (6 rows to the left of the current block, depending on their availability) can also be applied to FIBC and FTMP modes.

[0434] Reference area and filling process in FIBC

[0435] According to one or more embodiments of this disclosure, filter coefficients are calculated by minimizing the MSE between predicted and reconstructed luminance and / or chrominance samples in the template region. In one example, Figure 25 The template region is shown, consisting of luminance / chrominance samples in the upper left of the CU (i.e., the reference block). An extension of the template region (i.e., the boundary region of the template region) is needed to support the "side samples" of the cross-shaped spatial filter, and this can be achieved using different methods.

[0436] In the first approach, it is proposed to fill in the gaps using the nearest available sample point when the sample point is unavailable.

[0437] The second approach proposes filling with the nearest available sample point, regardless of whether it is unavailable.

[0438] According to one or more embodiments of this disclosure, the reference region may extend to the right by a CU width and extend below the CU boundary by a CU height.

[0439] According to one or more embodiments of this disclosure, the reference region can be adjusted to include only available samples. In one example, the reference region consists of N rows of luminance / chrominance samples in the upper left corner of the CU. N is an integer and / or has a maximum upper limit (e.g., 4, 5, 6, 7).

[0440] Adaptive reordering of merge candidates with FIBC

[0441] According to one or more embodiments of this disclosure, during the process of extending the ARMC-TM to the IBC merge list, when the merge candidates are predicted using FIBC, the reference samples of the templates for the merge candidates are also generated by FIBC.

[0442] According to one or more embodiments of this disclosure, filter coefficients are calculated by minimizing the MSE between predicted luminance and / or chrominance samples in the reference region and reconstructed luminance and / or chrominance samples.

[0443] According to one or more embodiments of this disclosure, the template size and shape may not be included in the reference samples of the merged candidate templates. In one example, Figure 33 The reference area is shown, consisting of three rows of brightness samples in the upper left of the CU.

[0444] According to one or more embodiments of this disclosure, during the process of extending the ARMC-TM to the IBC merge list, reference samples of the templates for the merge candidates generated by IBC-LIC are used to reduce complexity when the merge candidates are predicted using FIBC.

[0445] According to one or more embodiments of this disclosure, during the process of extending the ARMC-TM to the IBC merging list, when the merging candidates are predicted using FIBC, the reference samples of the template of the merging candidates are generated by IBC without filtering (i.e., unfiltered IBC) to reduce complexity.

[0446] Merger candidates with FIBC

[0447] According to one or more embodiments of this disclosure, the IBC predictions from the merged candidates are further filtered. Different methods can be used to achieve this objective.

[0448] According to one or more embodiments of this disclosure, an enable flag can be signaled in the bitstream to indicate the FIBC merging mode used. The enable flag can be signaled at the SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / subblock / sample level.

[0449] According to one or more embodiments of this disclosure, a mode flag can be derived at the decoder to save bit overhead, rather than explicitly signaling the selected mode flag.

[0450] Direct block vectors for chromaticity blocks with FIBC

[0451] According to one or more embodiments of this disclosure, the direct block vector of the chroma block is further filtered. Different methods can be used to achieve this objective.

[0452] According to one or more embodiments of this disclosure, an enable flag can be signaled in the bitstream to indicate the FIBC merging mode used. The enable flag can be signaled at the SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / subblock / sample level.

[0453] According to one or more embodiments of this disclosure, the mode flag can be inherited from the luma block at the decoder to save bit overhead, instead of explicitly signaling the selected mode flag.

[0454] Combination of FIBC and IBC with fractional block vectors

[0455] In FIBC, after generating an initial prediction block based on the reference block using integer copying (i.e., copying the sample values ​​associated with the reference block using the integer part of the integer block vector or fractional block vector) or interpolation filtering (i.e., generating interpolated sample values ​​using the fractional block vector), a further filtering process is applied to the initial prediction block to refine it into a final prediction block. When the current block to be predicted in FIBC mode is associated with the integer block vector of the reference block, this can be achieved by using a reference template (e.g., such as...). Figure 25 and Figure 33 The reference template shown is used to derive the parameters (i.e., filter coefficients) used in the filtering process. This reference template is determined using the same integer block vector used to generate the initial prediction block for the current block. In one example, as... Figure 33 As shown, the reference template is determined based on the current template by using the same integer block vector between the reference block and the current block. In another example, when an IBC block encoded using fractional block vectors is combined with an FIBC mode (i.e., when the current block to be predicted in the FIBC mode is associated with the fractional block vector of the reference block), the reference template for deriving the filter parameters can be determined in a different way. That is, if the block vector is a fractional block vector, the same fractional block vector can be applied to determine the reference template, or a different block vector (such as an integer block vector obtained based on the fractional block vector) can be used to determine the reference template, which can result in fewer interpolation operations.

[0456] In one example, a reference template for FIBC parameter derivation is determined using the same fractional block vector used to determine the initial prediction block for the current block. In such a case, both the reference template and the initial prediction block are generated via interpolation. However, the interpolation filter used to generate the reference template can be the same as or different from the interpolation filter used to generate the initial prediction block for the current block. For example, given a 1 / 4 image element block vector for the current block, a 12-tap interpolation filter can be used to generate the initial prediction block for the current block, but when generating the reference template for FIBC, the same 12-tap interpolation filter or a different interpolation filter (e.g., a 4-tap, a 2-tap, or a 0-tap interpolation filter) can be used. If some samples are outside the effective IBC reference area, an interpolation filter with fewer taps can result in less computation and / or less padding. If a different interpolation filter is used for reference template generation, that interpolation filter can be predefined or signaled at different coding levels (e.g., based on sequence, image, slice, CTU level, or region).

[0457] In another example, the reference template for FIBC parameter derivation is determined by modifying the integer block vector from the fractional block vector of the initial prediction block used to generate the current block. This modification can be based on a truncation operation (e.g., directly discarding the fractional portion of the fractional block vector) or a rounding operation (e.g., rounding the fractional portion to the nearest integer value).

[0458] According to one or more embodiments of this disclosure, FIBC and IBC are mutually exclusive with fractional block vectors. Different methods can be used to achieve this goal.

[0459] In one example, the reference template for FIBC parameter export is determined by using the integer block vector used when determining the initial prediction block for the current block.

[0460] In another example, FIBC is disabled when the fractional block vector is determined to be used to determine the initial prediction block for the current block. In this case, the traditional IBC mode can be implemented to obtain the final prediction block.

[0461] Figure 34 The workflow of a method 3400 for video decoding according to one or more aspects of this disclosure is shown.

[0462] At step 3410, method 3400 includes determining an initial prediction block for the current block to be predicted based on a first block vector, wherein the first block vector is a fractional block vector between the current block and a reference block.

[0463] At step 3420, method 3400 includes determining a reference template based on the current template and the second block vector associated with the current block.

[0464] At step 3430, method 3400 includes obtaining a set of filter coefficients corresponding to the filter shape based at least on sample values ​​from both the current template and the reference template.

[0465] At step 3440, method 3400 includes deriving a final prediction block of the current block, which includes the prediction sample values ​​of the current block, by applying a set of filter coefficients and a filter shape to an initial prediction block.

[0466] In one example, determining the initial prediction block includes: determining a reference block based on a first block vector and the current block; and determining the initial prediction block by deriving interpolated sample values ​​of the reference block based on a first interpolation filter.

[0467] In one example, the second block vector is an integer block vector modified from the first block vector based on a truncation or rounding operation.

[0468] In one example, the second vector block is the same as the first vector block.

[0469] In one example, determining the reference template involves deriving the interpolated sample values ​​of the reference template based on the second interpolation filter.

[0470] In one example, the second interpolation filter is different from the first interpolation filter used to determine the initial prediction block.

[0471] In one example, the second interpolation filter has fewer taps than the first interpolation filter.

[0472] In one example, the second interpolation filter is either predefined or obtained at a different coding level.

[0473] In one example, method 3400 further includes: in response to determining that the first block vector is a fractional block vector, obtaining the final predicted block by using an intra-block copy (IBC) mode.

[0474] Figure 35 The workflow of a method 3500 for video encoding according to one or more aspects of this disclosure is shown.

[0475] At step 3510, method 3500 includes determining an initial prediction block for the current block to be predicted based on a first block vector, wherein the first block vector is a fractional block vector between the current block and a reference block.

[0476] At step 3520, method 3500 includes determining a reference template based on the current template and the second block vector associated with the current block.

[0477] At step 3530, method 3500 includes obtaining a set of filter coefficients corresponding to the filter shape based at least on sample values ​​from both the current template and the reference template.

[0478] At step 3540, method 3500 includes deriving a final prediction block of the current block, which includes the prediction sample values ​​of the current block, by applying a set of filter coefficients and a filter shape to an initial prediction block.

[0479] At step 3550, method 3500 includes generating a bitstream based on the final predicted block.

[0480] In one example, determining the initial prediction block includes: determining a reference block based on a first block vector and the current block; and determining the initial prediction block by deriving interpolated sample values ​​of the reference block based on a first interpolation filter.

[0481] In one example, the second block vector is an integer block vector modified from the first block vector based on a truncation or rounding operation.

[0482] In one example, the second vector block is the same as the first vector block.

[0483] In one example, determining the reference template involves deriving the interpolated sample values ​​of the reference template based on the second interpolation filter.

[0484] In one example, the second interpolation filter is different from the first interpolation filter used to determine the initial prediction block.

[0485] In one example, the second interpolation filter has fewer taps than the first interpolation filter.

[0486] In one example, the second interpolation filter is signaled at different coding levels.

[0487] In one example, method 3500 further includes: in response to determining that the first block vector is a fractional block vector, obtaining the final predicted block by using an intra-block copy (IBC) mode.

[0488] Figure 36 A computing environment 3610 coupled to a user interface 3650 is shown. The computing environment 3610 may be part of a data processing server. The computing environment 3610 includes a processor 3620, memory 3630, and input / output (I / O) interface 3640.

[0489] Processor 3620 typically controls the overall operation of computing environment 3610, such as operations associated with display, data acquisition, data communication, and image processing. Processor 3620 may include one or more processors for executing instructions to perform all or some of the steps in the methods described above. Furthermore, processor 3620 may include one or more modules that facilitate interaction between processor 3620 and other components. The processor may be a central processing unit (CPU), microprocessor, microcontroller, graphics processing unit (GPU), etc.

[0490] Memory 3630 is configured to store various types of data to support the operation of computing environment 3610. Memory 3630 may include predefined software 3632. Examples of such data include instructions for any application or method operating on computing environment 3610, video datasets, image data, etc. Memory 3630 can be implemented using any type of volatile or non-volatile memory device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0491] I / O interface 3640 provides an interface between processor 3620 and peripheral interface modules (such as keyboard, click wheel, buttons, etc.). Buttons may include, but are not limited to, home button, start scan button, and stop scan button. I / O interface 3640 can be coupled to encoder and decoder.

[0492] In an embodiment, a non-transitory computer-readable storage medium is also provided, including, for example, a memory 3630 containing a plurality of programs and / or storing a bitstream generated by the above-described encoding method or a bitstream to be decoded by the above-described decoding method. The plurality of programs can be executed by a processor 3620 in a computing environment 3610 to perform the above-described methods. In one example, the plurality of programs can be executed by a processor 3620 in a computing environment 3610 to (e.g., from...) Figure 2The video encoder 20 in the computing environment 3610 receives a bitstream or data stream including encoded video information (e.g., video blocks representing encoded video frames, and / or one or more associated syntax elements, etc.), and can also be executed by the processor 3620 in the computing environment 3610 to perform the above-described decoding method based on the received bitstream or data stream. In another example, the plurality of programs can be executed by the processor 3620 in the computing environment 3610 to perform the above-described encoding method to encode video information (e.g., video blocks representing video frames, and / or one or more associated syntax elements, etc.) into a bitstream or data stream, and can also be executed by the processor 3620 in the computing environment 3610 to (e.g., to...) Figure 3 The video decoder 30 in the middle sends the bitstream or data stream. Alternatively, a non-transitory computer-readable storage medium may store data generated by the encoder (e.g., Figure 2 The video encoder 20 in the video encoder uses, for example, the encoding method described above to generate the video for the decoder (e.g., Figure 3 The video decoder 30 in the video decoder uses a bitstream or data stream that includes encoded video information (e.g., video blocks representing encoded video frames, and / or one or more associated syntax elements, etc.) when decoding video data. Non-transitory computer-readable storage media can be, for example, ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage devices, etc.

[0493] In one embodiment, a bitstream generated by the above-described encoding method or a bitstream to be decoded by the above-described decoding method is provided. In another embodiment, a bitstream comprising encoded video information generated by the above-described encoding method or encoded video information to be decoded by the above-described decoding method is provided.

[0494] In one embodiment, a computing device is also provided, comprising: one or more processors (e.g., processor 3620); and a non-transitory computer-readable storage medium or memory 3630 therein storing a plurality of programs executable by the one or more processors, wherein the one or more processors are configured to perform the methods described above when executing the plurality of programs.

[0495] In one embodiment, a computer program product having instructions for storing or transmitting a bitstream is also provided, the bitstream including encoded video information generated by the encoding method described above or encoded video information to be decoded by the decoding method described above. In another embodiment, a computer program product including, for example, a plurality of programs in a memory 3630 is also provided, the plurality of programs being executable by a processor 3620 in a computing environment 3610 to perform the methods described above. For example, the computer program product may include a non-transitory computer-readable storage medium.

[0496] In an embodiment, the computing environment 3610 may be implemented by one or more ASICs, DSPs, digital signal processing devices (DSPDs), programmable logic devices (PLDs), FPGAs, GPUs, controllers, microcontrollers, microprocessors, or other electronic components for performing the methods described above.

[0497] In one embodiment, a method for storing a bitstream is also provided, comprising: storing the bitstream on a digital storage medium, wherein the bitstream includes encoded video information generated by the above-described encoding method or encoded video information to be decoded by the above-described decoding method.

[0498] In one embodiment, a method for transmitting a bitstream generated by the encoder described above is also provided. In another embodiment, a method for receiving a bitstream to be decoded by the decoder described above is also provided.

[0499] The description in this disclosure has been presented for illustrative purposes and is not intended to be exhaustive or limited to this disclosure. Many modifications, variations, and alternative embodiments will be apparent to those skilled in the art from the teachings presented in the foregoing description and the associated drawings.

[0500] Unless otherwise specifically stated, the order of steps in the method according to this disclosure is intended to be illustrative only, and the steps of the method according to this disclosure are not limited to the specific order described above, but may be changed according to actual circumstances. Furthermore, at least one step in the method according to this disclosure may be adjusted, combined, or omitted as needed.

[0501] The examples chosen and described are intended to explain the principles of this disclosure and to enable others skilled in the art to understand the various embodiments of this disclosure, and preferably to utilize the basic principles and various embodiments with various modifications suitable for the intended particular purpose. Therefore, it will be understood that the scope of this disclosure is not limited to the specific examples of the disclosed embodiments, and that modifications and other embodiments are intended to be included within the scope of this disclosure.

Claims

1. A method for video decoding, comprising: The initial prediction block for the current block to be predicted is determined based on the first block vector, wherein the first block vector is a fractional block vector between the current block and the reference block; A reference template is determined based on the current template and the second block vector associated with the current block; The set of filter coefficients corresponding to the filter shape is obtained based at least on sample values ​​from both the current template and the reference template; and The final prediction block of the current block is derived by applying the filter coefficient set and the filter shape to the initial prediction block. The final prediction block includes the predicted sample values ​​of the current block.

2. The method according to claim 1, wherein, Determining the initial prediction block further includes: The reference block is determined based on the first block vector and the current block; and The initial prediction block is determined by deriving the interpolated sample values ​​of the reference block based on the first interpolation filter.

3. The method according to claim 1, wherein, The second block vector is an integer block vector modified from the first block vector based on a truncation or rounding operation.

4. The method according to claim 1, wherein, The second block vector is the same as the first block vector.

5. The method according to claim 4, wherein, Determining the reference template also includes: The interpolated sample values ​​of the reference template are derived based on the second interpolation filter.

6. The method according to claim 5, wherein, The second interpolation filter is different from the first interpolation filter used to determine the initial prediction block.

7. The method according to claim 6, wherein, The second interpolation filter has fewer taps than the first interpolation filter.

8. The method according to claim 6, wherein, The second interpolation filter is predefined or obtained at a different coding level.

9. The method according to claim 1, further comprising: In response to determining that the first block vector is a fractional block vector, the final predicted block is obtained by using the intra-block copy (IBC) mode.

10. A method for video encoding, comprising: The initial prediction block for the current block to be predicted is determined based on the first block vector, wherein the first block vector is a fractional block vector between the current block and the reference block; A reference template is determined based on the current template and the second block vector associated with the current block; The set of filter coefficients corresponding to the filter shape is obtained based at least on sample values ​​from both the current template and the reference template; The final prediction block of the current block is derived by applying the filter coefficient set and the filter shape to the initial prediction block; the final prediction block includes the predicted sample values ​​of the current block. A bitstream is generated based on the final predicted block.

11. The method according to claim 10, wherein, Determining the initial prediction block further includes: The reference block is determined based on the first block vector and the current block; and The initial prediction block is determined by deriving the interpolated sample values ​​of the reference block based on the first interpolation filter.

12. The method according to claim 10, wherein, The second block vector is an integer block vector modified from the first block vector based on a truncation or rounding operation.

13. The method according to claim 10, wherein, The second block vector is the same as the first block vector.

14. The method according to claim 13, wherein, Determining the reference template also includes: The interpolated sample values ​​of the reference template are derived based on the second interpolation filter.

15. The method according to claim 14, wherein, The second interpolation filter is different from the first interpolation filter used to determine the initial prediction block.

16. The method according to claim 15, wherein, The second interpolation filter has fewer taps than the first interpolation filter.

17. The method according to claim 15, wherein, The second interpolation filter is signaled at different coding levels.

18. The method of claim 10, further comprising: In response to determining that the first block vector is a fractional block vector, the final predicted block is obtained by using the intra-block copy (IBC) mode.

19. An apparatus for video encoding and decoding, comprising: One or more processors; as well as One or more storage devices storing computer-executable instructions that, when executed, cause the one or more processors to perform the operation of the method according to any one of claims 1-18.

20. A computer program product having instructions for storing a bit stream, wherein, The bitstream comprises video data decoded by the method for video decoding of any one of claims 1-9; or the bitstream comprises video data generated by the method for video encoding of any one of claims 10-18.

21. A computer-readable storage medium storing instructions, said instructions causing said one or more processors, when executed by a computing device having one or more processors, to: Perform the method according to any one of claims 1-9, and store the bitstream to be decoded by the method according to any one of claims 1-9, or Perform the method according to any one of claims 10-18, and store the bit stream generated by the method according to any one of claims 10-18.

22. A computer-readable medium for storing a bitstream, wherein, The bitstream is to be decoded by performing the operation of any one of claims 1-9, or The bitstream is obtained by performing the operation of any one of claims 10-18.

23. A method for receiving a bitstream to be decoded by any one of claims 1-9.

24. A method for transmitting a bit stream generated by the method according to any one of claims 10-18.

Images