Intra-frame template matching prediction method and device

Abstract

Methods, apparatus, and non-transitory computer-readable storage media for video decoding and encoding are provided. In a video decoding method, in response to determining to use multi-hypothesis intra template matching prediction (TMP), a decoder may obtain one or more reference block candidates for a current block and obtain a final prediction for the current block based on the one or more reference block candidates.

Description

[0001] Cross-references to related applications

[0002] This application is filed and claims priority to U.S. Provisional Application No. 63 / 451,546, filed on March 10, 2023, entitled “Method and Apparatus for Intra-Frame Template Matching Prediction”, the entire contents of which are incorporated herein by reference. Technical Field

[0003] This disclosure relates to video encoding and decoding and compression, and in particular, but not limited to, methods and apparatus for improving the encoding and decoding efficiency of intra-frame template matching prediction modes. Background Technology

[0004] Various electronic devices (such as digital televisions, laptops or desktop computers, tablets, digital cameras, digital recording devices, digital media players, video game consoles, smartphones, video conferencing equipment, video streaming devices, etc.) support digital video. Electronic devices transmit and receive, or otherwise transmit, digital video data via communication networks, and / or store digital video data on storage devices. Due to the limited bandwidth capacity of communication networks and the limited memory resources of storage devices, video data can be compressed using one or more video codec standards before it is transmitted or stored. Examples of video codec standards include Universal Video Codec (VVC), Joint Explore Test Model (JEM), High Efficiency Video Codec (HEVC / H.265), Advanced Video Codec (AVC / H.264), Moving Picture Experts Group (MPEG) codec, etc. Video codecs typically employ prediction methods that utilize the inherent redundancy in video data (e.g., inter-frame prediction, intra-frame prediction, etc.). Video codecs aim to compress video data to a form using a lower bitrate while avoiding or minimizing degradation in video quality.

[0005] SUMMARY

[0006] This disclosure provides examples of techniques related to improving the encoding and decoding efficiency of intra-frame template matching prediction (TMP) methods during video encoding or decoding.

[0007] According to a first aspect of this disclosure, a video decoding method is provided. In this method, in response to determining that multiple hypothesis intra-template matching prediction (TMP) is used, the decoder can obtain one or more reference block candidates for the current block and obtain a final prediction for the current block based on the one or more reference block candidates.

[0008] According to a second aspect of this disclosure, a video coding method is provided. In this method, in response to determining to use multi-hypothesis intra-template matching prediction (TMP), the encoder can obtain one or more reference block candidates for the current block and obtain a final prediction for the current block based on the one or more reference block candidates.

[0009] According to a third aspect of this disclosure, a video decoding apparatus is provided. The apparatus may include one or more processors and a memory coupled to the one or more processors, the memory being configured to store instructions executable by the one or more processors. Furthermore, the one or more processors, when executing the instructions, are configured to perform the method according to the first aspect described above.

[0010] According to a fourth aspect of this disclosure, a video encoding apparatus is provided. The apparatus may include one or more processors and a memory coupled to the one or more processors, the memory being configured to store instructions executable by the one or more processors. Furthermore, the one or more processors, when executing the instructions, are configured to perform the method according to the second aspect described above.

[0011] According to a fifth aspect of this disclosure, a non-transitory computer-readable storage medium is provided for storing computer-executable instructions that, when executed by one or more computer processors, cause the one or more computer processors to perform the method according to the first aspect described above.

[0012] According to a sixth aspect of this disclosure, a non-transitory computer-readable storage medium is provided for storing computer-executable instructions that, when executed by one or more computer processors, cause the one or more computer processors to perform the method according to the second aspect described above.

[0013] According to a seventh aspect of this disclosure, a non-transitory computer-readable storage medium is provided for storing a bit stream to be decoded by the method described in the first aspect above.

[0014] According to an eighth aspect of this disclosure, a non-transitory computer-readable storage medium is provided for storing a bit stream generated by the method described in the second aspect above. Attached Figure Description

[0015] A more specific description of the examples of this disclosure will be presented with reference to the specific examples shown in the accompanying drawings. Given that these drawings depict only a few examples and should not be considered limiting, these examples will be described and explained with additional specificity and detail using the drawings.

[0016] Figure 1 A block diagram of an exemplary system for encoding and decoding video blocks according to some embodiments of the present disclosure is shown.

[0017] Figure 2 A block diagram of an exemplary video encoder according to some embodiments of the present disclosure is shown.

[0018] Figure 3 A block diagram of an exemplary video decoder according to some embodiments of the present disclosure is shown.

[0019] Figures 4A-4E A block diagram is shown illustrating how a frame is recursively divided into multiple video blocks of different sizes and shapes according to some embodiments of this disclosure.

[0020] Figure 5 A diagram illustrating a computing environment coupled to a user interface according to some embodiments of the present disclosure is shown.

[0021] Figure 6 A schematic diagram of the intra-frame template matching search region used according to some examples of this disclosure is shown.

[0022] Figure 7 The use of intra-frame TMP block vectors for IBC blocks is shown in some examples according to this disclosure.

[0023] Figure 8 Spatial locations for linear filter models are shown in some examples according to this disclosure.

[0024] Figure 9 Reference areas for deriving filter coefficients for linear filter models, according to some examples of this disclosure, are shown.

[0025] Figure 10 A schematic diagram is shown illustrating the positions of the left and upper samples of the CU involved in the prediction of a cross-component linear model (CCLM) according to some examples of this disclosure.

[0026] Figure 11 The reference region (and its padding) for deriving the filter coefficients of CCCM according to some examples of this disclosure are shown;

[0027] Figure 12 Four Sobel-based gradient modes for GLM are shown, according to some examples of this disclosure.

[0028] Figure 13 A schematic diagram of half-pixel interpolation and template matching search according to some examples of this disclosure is shown;

[0029] Figure 14 shows a schematic diagram of the intra-frame TMP in a sub-block.

[0030] Figure 15 A schematic diagram of intra-frame TMP based on a rectangular region is shown.

[0031] Figure 16 shows a schematic diagram of the two-stage combination of intra-frame and intra-frame TMP.

[0032] Figure 17 The diagram illustrates, according to some examples of this disclosure, the division of an intermediate prediction block into several sub-blocks and the execution of several sub-blocks based on the sub-blocks.

[0033] Figure 18 The illustration shows some examples of how a reference sub-block for the current pixel is found by minimizing the distance between the current sub-block and the reference sub-block using a sub-block centered on the pixel location, according to the present disclosure.

[0034] Figure 19 A flowchart of a video decoding method according to some examples of this disclosure is shown.

[0035] Figure 20 Examples of such disclosure are shown. Figure 19 The flowchart shows the video encoding method corresponding to the video decoding method shown. Detailed Implementation

[0036] Referring now to the detailed description, examples of which are illustrated in the accompanying drawings. Numerous non-limiting details are set forth in the following detailed description to aid in understanding the subject matter presented herein. However, various alternatives may be used without departing from the scope of the claims, and the subject matter may be practiced without these specific details. For example, the subject matter presented herein can be implemented on many classes of electronic devices with digital video capabilities.

[0037] The terminology used in this disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of this disclosure. The singular forms “a / an,” “the,” and “the” in this disclosure and the appended claims are also intended to include the plural forms unless otherwise expressly indicated throughout the disclosure. It should also be understood that the term “and / or” as used in this disclosure refers to and includes one or any one or all possible combinations of the listed related items.

[0038] Throughout this specification, references to "an embodiment," "an embodiment," "an example," "some embodiments," "some examples," "one or more examples," or similar language mean that a particular feature, structure, or characteristic described is included in at least one embodiment or example. Unless otherwise expressly stated, the features, structures, elements, or characteristics described in connection with one or more embodiments also apply to other embodiments.

[0039] Throughout this disclosure, "first," "second," "third," etc., are used as a nomenclature to refer only to relevant elements, such as equipment, components, parts, steps, etc., and do not imply any spatial or temporal order unless otherwise expressly stated. For example, "first equipment" and "second equipment" can refer to two separately formed devices, or two parts, components, or operating states of the same equipment, and can be named arbitrarily.

[0040] The terms "module," "submodule," "circuit," "sub-circuit," "circuitry," "sub-circuitry," "unit," or "subunit" can include memory (shared, dedicated, or grouped) that stores code or instructions executable by one or more processors. A module can include one or more circuits, with or without stored code or instructions. A module or circuit can include one or more components that are directly or indirectly connected. These components may or may not be physically attached to each other or adjacent to each other.

[0041] As used herein, depending on the context, the terms "if" or "when" can be understood to mean "at the time of" or "in response to". If these terms appear in the claims, they may not indicate that the relevant limitation or feature is conditional or optional. For example, a method may include the steps of: i) performing a function or action X' when or if condition X exists, and ii) performing a function or action Y' when or if condition Y exists. The method may simultaneously have the capability to perform both function or action X' and function or action Y'. Therefore, functions X' and Y' may be performed at different times in multiple executions of the method.

[0042] Units or modules can be implemented purely in software, purely in hardware, or a combination of both. For example, in a purely software implementation, a unit or module may include functionally related code blocks or software components that are directly or indirectly linked together to perform a specific function.

[0043] Figure 1 A block diagram of an exemplary system 10 for encoding and decoding video blocks in parallel, according to some embodiments of the present disclosure, is shown. Figure 1As shown, system 10 includes a source device 12 that generates and encodes video data that will later be decoded by a target device 14. The source device 12 and target device 14 can include any electronic device from a wide variety of electronic devices, including cloud servers, server computers, desktop or laptop computers, tablet computers, smartphones, set-top boxes, digital televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, etc. In some embodiments, the source device 12 and target device 14 are equipped with wireless communication capabilities.

[0044] In some implementations, target device 14 may receive encoded video data to be decoded via link 16. Link 16 may include any type of communication medium or device capable of moving encoded video data from source device 12 to target device 14. In one example, link 16 may include a communication medium enabling source device 12 to transmit encoded video data directly to target device 14 in real time. The encoded video data may be modulated according to a communication standard (e.g., a wireless communication protocol) and transmitted to target device 14. The communication medium may include any wireless or wired communication medium, such as radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium may form part of a packet-based network (e.g., a local area network, a wide area network, or a global network such as the Internet). The communication medium may include a router, switch, base station, or any other means that may facilitate communication from source device 12 to target device 14.

[0045] In some other implementations, encoded video data can be sent from output interface 22 to storage device 32. The target device 14 can then access the encoded video data in storage device 32 via input interface 28. Storage device 32 can include any data storage medium of various distributed or locally accessed data storage media, such as hard disk drives, Blu-ray discs, digital universal discs (DVDs), compact disc read-only memory (CD-ROMs), flash memory, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded video data. In another example, storage device 32 can correspond to a file server or another intermediate storage device that can hold the encoded video data generated by source device 12. Target device 14 can access the stored video data from storage device 32 via streaming or downloading. The file server can be any type of computer capable of storing and sending encoded video data to target device 14. Exemplary file servers include web servers (e.g., for websites), file transfer protocol (FTP) servers, network attached storage (NAS) devices, or local disk drives. Target device 14 can access the encoded video data via any standard data connection suitable for accessing encoded video data stored on the file server. Standard data connections include wireless channels (e.g., Wi-Fi connections), wired connections (e.g., digital subscriber line (DSL), cable modems, etc.), or a combination of both. Transfer of encoded video data from storage device 32 can be streaming, downloading, or a combination of both.

[0046] like Figure 1 As shown, source device 12 includes a video source 18, a video encoder 20, and an output interface 22. Video source 18 may include sources or combinations of such sources, such as: a video capture device (e.g., a camera), a video archive containing previously captured video, a video feed interface for receiving video from a video content provider, and / or a computer graphics system for generating computer graphics data as source video. As an example, if video source 18 is a camera in a security monitoring system, source device 12 and target device 14 may form a camera phone or video phone. However, the embodiments described in this application are generally applicable to video encoding and decoding and can be applied to wireless and / or wired applications.

[0047] The captured, pre-captured, or computer-generated video can be encoded by the video encoder 20. The encoded video data can be sent directly to the target device 14 via the output interface 22 of the source device 12. Alternatively, the encoded video data can be stored on the storage device 32 for later access by the target device 14 or other devices for decoding and / or playback. The output interface 22 may further include a modem and / or transmitter.

[0048] Target device 14 includes an input interface 28, a video decoder 30, and a display device 34. Input interface 28 may include a receiver and / or a modem, and receives encoded video data via link 16. The encoded video data transmitted via link 16 or provided on storage device 32 may include various syntax elements generated by video encoder 20 for use by video decoder 30 when decoding the video data. Such syntax elements may be included within encoded video data transmitted on a communication medium, stored on a storage medium, or stored on a file server.

[0049] In some embodiments, the target device 14 may include a display device 34, which may be an integrated display device or an external display device configured to communicate with the target device 14. The display device 34 displays decoded video data to a user and may include any of a variety of display devices, such as a liquid crystal display (LCD), a plasma display, an organic light-emitting diode (OLED) display, or another type of display device.

[0050] Video encoder 20 and video decoder 30 can operate according to proprietary or industry standards (e.g., VVC, HEVC, MPEG-4 Part 10, AVC) or extensions of such standards. It should be understood that this application is not limited to a specific video encoding / decoding standard and can be applied to other video encoding / decoding standards. It is generally understood that the video encoder 20 of source device 12 can be configured to encode video data according to any of these current or future standards. Similarly, it is also generally understood that the video decoder 30 of target device 14 can be configured to decode video data according to any of these current or future standards.

[0051] The video encoder 20 and video decoder 30 can be implemented as any circuit of a variety of suitable encoder and / or decoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic devices, software, hardware, firmware, or any combination thereof. When partially implemented in software, the electronic device may store instructions for the software in a suitable non-transitory computer-readable medium and use one or more processors to execute the instructions in the hardware to perform the video encoding / decoding operations disclosed in this disclosure. Each of the video encoder 20 and video decoder 30 may be included in one or more encoders or decoders, either of which may be integrated as part of a combined encoder / decoder (CODEC) in the respective device.

[0052] In some implementations, components of source device 12 (e.g., video source 18, video encoder 20, or the following references) Figure 2 The components described include at least a portion of the components in the video encoder 20 and the output interface 22) and / or the components of the target device 14 (e.g., the input interface 28, the video decoder 30, or the following references). Figure 3At least a portion of the components included in the video decoder 30 and the display device 34 described herein can operate in a cloud computing service network such as Software as a Service (SaaS), Platform as a Service (PaaS), or Infrastructure as a Service (IaaS), where the cloud computing service network can provide software, platform, and / or infrastructure. In some embodiments, one or more components of the source device 12 and / or the target device 14 not included in the cloud computing service network may be located in one or more client devices, and these client devices may communicate with server computers in the cloud computing service network via wireless communication networks (e.g., cellular communication networks, short-range wireless communication networks, or Global Navigation Satellite System (GNSS) communication networks) or wired communication networks (e.g., local area network (LAN) communication networks or power line communication (PLC) networks). In embodiments, at least a portion of the operations described herein may be implemented as cloud-based services provided by one or more server computers, wherein the one or more server computers are implemented by at least a portion of the components of the source device 12 and / or at least a portion of the components of the target device 14 in the cloud computing service network; and one or more other operations described herein may be implemented by one or more client devices. In some implementations, the cloud computing service network may be a private cloud, a public cloud, or a hybrid cloud. Without departing from the scope of this disclosure, terms such as “cloud,” “cloud computing,” and “cloud-based” are used interchangeably as appropriate. It should be understood that this disclosure is not limited to implementation within the aforementioned cloud computing service networks. Rather, this disclosure may be implemented in any other type of computing environment currently known or developed in the future.

[0053] Figure 2 A block diagram of an exemplary video encoder 20 according to some embodiments described in this application is shown. The video encoder 20 can perform intra-frame predictive coding and inter-frame predictive coding on video blocks within a video frame. Intra-frame predictive coding relies on spatial prediction to reduce or remove spatial redundancy in the video data within a given video frame or picture. Inter-frame predictive coding relies on temporal prediction to reduce or remove temporal redundancy in the video data within neighboring video frames or pictures of a video sequence. It should be noted that in the field of video encoding and decoding, the term "frame" can be used as a synonym for the terms "image" or "picture".

[0054] like Figure 2As shown, the video encoder 20 includes a video data memory 40, a prediction processing unit 41, a decoded picture buffer (DPB) 64, an adder 50, a transform processing unit 52, a quantization unit 54, and an entropy coding unit 56. The prediction processing unit 41 further includes a motion estimation unit 42, a motion compensation unit 44, a segmentation unit 45, an intra-frame prediction processing unit 46, and an intra-frame block copying (BC) unit 48. In some embodiments, the video encoder 20 also includes an inverse quantization unit 58, an inverse transform processing unit 60, and an adder 62 for video block reconstruction. A loop filter 63, such as a deblocking filter, can be located between the adder 62 and the DPB 64 to filter block boundaries to remove block artifacts from the reconstructed video. In addition to the deblocking filter, another loop filter (e.g., a sample adaptive offset (SAO) filter, a cross-component sample adaptive offset (CCSAO) filter, and / or an adaptive loop filter (ALF)) can be used to filter the output of the adder 62. It should be noted that, regarding the CCSAO technology, this application is not limited to the embodiments described herein, but can also be applied to situations where an offset is selected for any other component among the luminance, Cb, and Cr chrominance components based on any one of the luminance, Cb, and Cr chrominance components to modify that other component based on the selected offset. Furthermore, it should be noted that the first component mentioned in the text can be any one of the luminance, Cb, and Cr chrominance components, the second component can be any other one of the luminance, Cb, and Cr chrominance components, and the third component can be the remaining component of any one of the luminance, Cb, and Cr chrominance components. In some examples, the in-loop filter can be omitted, and the decoded video block can be directly provided to the DPB 64 by the adder 62. The video encoder 20 can take the form of a fixed or programmable hardware unit, or it can be distributed among one or more of the described fixed or programmable hardware units.

[0055] The video data storage device 40 can store video data encoded by the components of the video encoder 20. For example, it can store data from... Figure 1 The video source 18 shown receives video data from the video data memory 40. The DPB 64 is a buffer that stores reference video data (e.g., reference frames or pictures) used by the video encoder 20 (e.g., in intra-frame or inter-frame predictive coding modes) when encoding the video data. The video data memory 40 and DPB 64 can be formed from any of a variety of memory devices. In various examples, the video data memory 40 may be on-chip along with other components of the video encoder 20, or off-chip relative to those components.

[0056] like Figure 2As shown, after receiving video data, the segmentation unit 45 within the prediction processing unit 41 segments the video data into video blocks. This segmentation may also include segmenting the video frame into strips, tiles (e.g., a collection of video blocks) or other larger coding units (CUs) according to a predefined splitting structure associated with the video data (e.g., a quadtree (QT) structure). A video frame is, or can be considered, a two-dimensional array or matrix of samples with sample values. Samples in the array may also be referred to as pixels or image elements (pel). The number of samples in the horizontal and vertical directions (or axes) of the array or image defines the size and / or resolution of the video frame. For example, a video frame can be divided into multiple video blocks using QT segmentation. A video block is again, or can be considered, a two-dimensional array or matrix of samples with sample values, but its dimension is smaller than that of the video frame. The number of samples in the horizontal and vertical directions (or axes) of the video block defines the size of the video block. By iteratively using, for example, QT segmentation, binary tree (BT) segmentation, or ternary tree (TT) segmentation, or any combination thereof, a video block can be further segmented into one or more block partitions or sub-blocks (which can again form blocks). It should be noted that the term "block" or "video block" as used herein can refer to a portion of a frame or image, particularly a rectangular (square or non-square) portion. Referring, for example, to HEVC and VVC, a block or video block can be or corresponds to a coding tree unit (CTU), CU, prediction unit (PU), or transform unit (TU) and / or can be or corresponds to a corresponding block (e.g., coding tree block (CTB), coding block (CB), prediction block (PB), or transform block (TB)) and / or sub-block.

[0057] The prediction processing unit 41 can select one of several feasible predictive coding modes for the current video block based on error results (e.g., coding rate and distortion level), such as one or more inter-frame predictive coding modes among multiple intra-frame predictive coding modes. The prediction processing unit 41 can provide the resulting intra-frame or inter-frame predictive coding block to adder 50 to generate a residual block, and to adder 62 to reconstruct the coding block for subsequent use as part of a reference frame. The prediction processing unit 41 also provides syntax elements (e.g., motion vectors, intra-frame mode indicators, segmentation information, and other such syntax information) to entropy coding unit 56.

[0058] To select a suitable intra-predictive coding mode for the current video block, the intra-predictive processing unit 46 within the prediction processing unit 41 can perform intra-predictive coding of the current video block in relation to one or more neighboring blocks in the same frame as the current block to be encoded to provide spatial prediction. The motion estimation unit 42 and motion compensation unit 44 within the prediction processing unit 41 perform inter-predictive coding of the current video block in relation to one or more prediction blocks in one or more reference frames to provide temporal prediction. The video encoder 20 can perform multiple coding passes, for example, to select a suitable coding mode for each block of video data.

[0059] In some implementations, motion estimation unit 42 determines an inter-frame prediction mode for the current video frame by generating motion vectors based on a predetermined pattern within the video frame sequence. The motion vectors indicate the displacement of a video block within the current video frame relative to a prediction block within a reference video frame. Motion estimation performed by motion estimation unit 42 is the process of generating motion vectors that estimate the motion of video blocks. For example, the motion vectors may indicate the displacement of a video block within the current video frame or picture relative to a prediction block within a reference frame associated with the current block being encoded in the current frame. The predetermined pattern may designate video frames in the sequence as P-frames or B-frames. Intra-frame BC unit 48 may determine vectors (e.g., block vectors) for intra-frame BC coding in a similar manner to how motion estimation unit 42 determines motion vectors for inter-frame prediction, or the block vectors may be determined using motion estimation unit 42.

[0060] Regarding pixel differences, the predicted block for a video block can be, or can correspond to, a block or reference block of a reference frame considered to closely match the video block to be encoded. Pixel differences can be determined by the sum of absolute differences (SAD), sum of squared differences (SSD), or other difference metrics. In some implementations, the video encoder 20 can compute values ​​for sub-integer pixel positions of the reference frame stored in the DPB 64. For example, the video encoder 20 can interpolate values ​​for quarter-pixel positions, eighth-pixel positions, or other fractional pixel positions of the reference frame. Therefore, the motion estimation unit 42 can perform motion search relative to full-pixel positions and fractional pixel positions and output a motion vector with fractional-pixel accuracy.

[0061] The motion estimation unit 42 calculates the motion vector for a video block in an inter-frame predictive coding frame by comparing the position of the video block with the position of the predicted block in a reference frame selected from either a first reference frame list (list 0) or a second reference frame list (list 1), where each reference frame list identifies one or more reference frames stored in the DPB 64. The motion estimation unit 42 sends the calculated motion vector to the motion compensation unit 44, and then to the entropy coding unit 56.

[0062] Motion compensation performed by motion compensation unit 44 may involve acquiring or generating prediction blocks based on motion vectors determined by motion estimation unit 42. Upon receiving motion vectors for the current video block, motion compensation unit 44 may locate the prediction block pointed to by the motion vector in a reference frame list within a reference frame list, retrieve the prediction block from DPB 64, and forward the prediction block to adder 50. Adder 50 then forms a residual video block of pixel differences by subtracting the pixel values ​​of the prediction block provided by motion compensation unit 44 from the pixel values ​​of the currently encoded video block. The pixel differences forming the residual video block may include luminance component differences or chrominance component differences, or both. Motion compensation unit 44 may also generate syntax elements associated with video blocks of a video frame for use by video decoder 30 when decoding video blocks of a video frame. Syntax elements may include, for example, syntax elements defining motion vectors for identifying prediction blocks, any flags indicating prediction modes, or any other syntax information described herein. It should be noted that motion estimation unit 42 and motion compensation unit 44 may be highly integrated, but are described separately for conceptual purposes.

[0063] In some implementations, the intra-BC unit 48 can generate vectors and acquire prediction blocks in a manner similar to that described above in conjunction with the motion estimation unit 42 and the motion compensation unit 44; however, these prediction blocks are in the same frame as the current block being encoded, and these vectors are referred to as block vectors rather than motion vectors. Specifically, the intra-BC unit 48 can determine the intra-prediction mode to be used for encoding the current block. In some examples, the intra-BC unit 48 can, for example, use various intra-prediction modes to encode the current block during individual encoding passes and test their performance through rate-distortion analysis. Next, the intra-BC unit 48 can select a suitable intra-prediction mode from the various tested intra-prediction modes to use and generate an intra-mode indicator accordingly. For example, the intra-BC unit 48 can use rate-distortion analysis to calculate rate-distortion values ​​for the various tested intra-prediction modes and select the intra-prediction mode with the best rate-distortion characteristics from the tested modes as the suitable intra-prediction mode to use. Rate-distortion analysis generally determines the amount of distortion (or error) between the coded block and the original uncoded block that was encoded to produce the coded block, as well as the bit rate (i.e., the number of bits) used to produce the coded block. Intra-frame BC unit 48 can calculate the ratio based on the distortion and rate for various coded blocks to determine which intra-frame prediction mode exhibits the optimal rate-distortion value for the block.

[0064] In other examples, the intra-frame BC unit 48 may use, in whole or in part, the motion estimation unit 42 and the motion compensation unit 44 to perform such functions for intra-frame BC prediction according to the embodiments described herein. In any case, for intra-frame block copying, in terms of pixel differences, the predicted block may be a block considered to closely match the block to be encoded, the pixel differences may be determined by SAD, SSD, or other difference metrics, and identifying the predicted block may include calculating values ​​for sub-integer pixel positions.

[0065] Regardless of whether the predicted block comes from the same frame predicted intra-frame or from different frames predicted inter-frame, the video encoder 20 can form a residual video block by subtracting the pixel values ​​of the predicted block from the pixel values ​​of the current video block being encoded. The pixel difference forming the residual video block can include both luma component difference and chroma component difference.

[0066] As an alternative to the inter-frame prediction performed by the motion estimation unit 42 and the motion compensation unit 44 as described above, or the intra-block copy prediction performed by the intra-BC unit 48, the intra-prediction processing unit 46 can perform intra-frame prediction on the current video block. Specifically, the intra-prediction processing unit 46 can determine an intra-prediction mode for encoding the current block. To this end, the intra-prediction processing unit 46 can use various intra-prediction modes to encode the current block, for example, during individual encoding passes, and the intra-prediction processing unit 46 (or, in some examples, the mode selection unit) can select a suitable intra-prediction mode from the tested intra-prediction modes for use. The intra-prediction processing unit 46 can provide information indicating the intra-prediction mode selected for the block to the entropy coding unit 56. The entropy coding unit 56 can encode the information indicating the selected intra-prediction mode into the bitstream.

[0067] After prediction processing unit 41 determines the prediction block for the current video block via inter-frame prediction or intra-frame prediction, adder 50 forms a residual video block by subtracting the prediction block from the current video block. The residual video data in the residual block may be included in one or more TUs and provided to transform processing unit 52. Transform processing unit 52 uses a transform (e.g., discrete cosine transform (DCT) or a conceptually similar transform) to transform the residual video data into residual transform coefficients.

[0068] The transform processing unit 52 can send the resulting transform coefficients to the quantization unit 54. The quantization unit 54 quantizes the transform coefficients to further reduce the bit rate. The quantization process can also reduce the bit depth associated with some or all of the coefficients. The degree of quantization can be modified by adjusting the quantization parameters. In some examples, the quantization unit 54 can subsequently perform a scan on the matrix including the quantized transform coefficients. Alternatively, the entropy coding unit 56 can perform the scan.

[0069] After quantization, the entropy coding unit 56 entropy codes the quantized transform coefficients into a video bitstream using, for example, context-adaptive variable-length coding (CAVLC), context-adaptive binary arithmetic coding (CABAC), syntax-based context-adaptive binary arithmetic coding (SBAC), probabilistic interval segmented entropy (PIPE) coding, or another entropy coding method or technique. The encoded bitstream can then be sent to, for example,... Figure 1 The video decoder 30 shown, or archived in, for example Figure 1 The data is stored in storage device 32 for later transmission to or retrieval by video decoder 30. Entropy coding unit 56 can also entropy code the motion vectors and other syntax elements used for the current video frame being encoded.

[0070] The inverse quantization unit 58 and the inverse transform processing unit 60 apply inverse quantization and inverse transform, respectively, to reconstruct residual video blocks in the pixel domain for generating reference blocks to predict other video blocks. As noted above, the motion compensation unit 44 can generate motion-compensated prediction blocks from one or more reference blocks of frames stored in the DPB 64. The motion compensation unit 44 can also apply one or more interpolation filters to the prediction blocks to compute sub-integer pixel values ​​for use in motion estimation.

[0071] Adder 62 adds the reconstructed residual block to the motion-compensated prediction block generated by motion compensation unit 44 to generate a reference block for storage in DPB 64. The reference block can then be used as a prediction block by intra-frame BC unit 48, motion estimation unit 42, and motion compensation unit 44 for inter-frame prediction of another video block in subsequent video frames.

[0072] Figure 3 A block diagram of an exemplary video decoder 30 according to some embodiments of this application is shown. The video decoder 30 includes a video data memory 79, an entropy decoding unit 80, a prediction processing unit 81, an inverse quantization unit 86, an inverse transform processing unit 88, an adder 90, and a DPB 92. The prediction processing unit 81 further includes a motion compensation unit 82, an intra-frame prediction unit 84, and an intra-frame BC unit 85. The video decoder 30 can perform operations in conjunction with the above. Figure 2 The decoding process described for the video encoder 20 is essentially the inverse of the encoding process. For example, the motion compensation unit 82 can generate prediction data based on the motion vectors received from the entropy decoding unit 80, while the intra-frame prediction unit 84 can generate prediction data based on the intra-frame prediction mode indicator received from the entropy decoding unit 80.

[0073] In some examples, units of the video decoder 30 may be assigned tasks to perform embodiments of this application. Furthermore, in some examples, embodiments of this disclosure may be distributed across one or more units of the video decoder 30. For example, the intra-frame BC unit 85 may perform embodiments of this application individually or in combination with other units of the video decoder 30 (e.g., motion compensation unit 82, intra-frame prediction unit 84, and entropy decoding unit 80). In some examples, the video decoder 30 may not include the intra-frame BC unit 85, and the functionality of the intra-frame BC unit 85 may be performed by other components of the prediction processing unit 81 (e.g., motion compensation unit 82).

[0074] Video data memory 79 can store video data, such as encoded video bitstreams, that will be decoded by other components of video decoder 30. The video data stored in video data memory 79 can be obtained, for example, from storage device 32, from a local video source (e.g., a camera), via wired or wireless network communication of video data, or by accessing a physical data storage medium (e.g., a flash drive or hard disk). Video data memory 79 may include an encoded picture buffer (CPB) that stores encoded video data from the encoded video bitstream. The DPB 92 of video decoder 30 stores reference video data for use by video decoder 30 (e.g., in intra-frame or inter-frame predictive coding modes) when decoding video data. Video data memory 79 and DPB 92 can be formed of any memory device from a variety of memory devices, such as dynamic random access memory (DRAM) (including synchronous DRAM (SDRAM)), magnetoresistive RAM (MRAM), resistive RAM (RRAM), or other types of memory devices. For illustrative purposes, video data memory 79 and DPB 92 are shown in... Figure 3 The video data memory 79 and DPB 92 are depicted as two distinct components of the video decoder 30. However, it will be apparent to those skilled in the art that the video data memory 79 and DPB 92 may be provided by the same memory device or separate memory devices. In some examples, the video data memory 79 may be on-chip along with other components of the video decoder 30, or off-chip relative to those components.

[0075] During the decoding process, the video decoder 30 receives an encoded video bitstream representing video blocks of encoded video frames and associated syntax elements. The video decoder 30 may receive syntax elements at the video frame level and / or the video block level. The entropy decoding unit 80 of the video decoder 30 performs entropy decoding on the bitstream to generate quantization coefficients, motion vectors or intra-prediction mode indicators, and other syntax elements. The entropy decoding unit 80 then forwards the motion vectors or intra-prediction mode indicators, and other syntax elements to the prediction processing unit 81.

[0076] When a video frame is encoded as an intra-predictive coded (I) frame or used as an intra-coded prediction block in other types of frames, the intra-prediction unit 84 of the prediction processing unit 81 can generate prediction data for the video block of the current video frame based on the intra-prediction mode transmitted by the signal and reference data from the previous decoded block of the current frame.

[0077] When a video frame is encoded as an inter-frame predictive coded (i.e., B or P) frame, the motion compensation unit 82 of the prediction processing unit 81 generates one or more prediction blocks for the current video frame based on motion vectors and other syntax elements received from the entropy decoding unit 80. Each of the prediction blocks can be generated from a reference frame within a reference frame list. The video decoder 30 can construct the reference frame list, i.e., list 0 and list 1, based on the reference frames stored in the DPB 92 using a default construction technique.

[0078] In some examples, when a video block is encoded according to the intra-frame BC mode described herein, the intra-frame BC unit 85 of the prediction processing unit 81 generates a prediction block for the current video block based on the block vector and other syntax elements received from the entropy decoding unit 80. The prediction block can be located within a reconstructed region of the same image as the current video block, as defined by the video encoder 20.

[0079] Motion compensation unit 82 and / or intra-frame prediction (BC) unit 85 determine prediction information for video blocks in the current video frame by parsing motion vectors and other syntax elements, and then use this prediction information to generate prediction blocks for the current video block being decoded. For example, motion compensation unit 82 uses some of the received syntax elements to determine the prediction mode (e.g., intra-frame prediction or inter-frame prediction) for encoding video blocks in the video frame, the inter-frame prediction frame type (e.g., B or P), construction information for one or more reference frames in the reference frame list for the frame, motion vectors for each inter-frame prediction encoded video block in the frame, the inter-frame prediction state for each inter-frame prediction encoded video block in the frame, and other information for decoding video blocks in the current video frame.

[0080] Similarly, the intra-BC unit 85 can use some of the received syntax elements, such as flags, to determine which video blocks in the frame are predicted using the intra-BC mode, which video blocks in the frame are in the reconstruction region and should be stored in the DPB 92, the block vector for each intra-BC predicted video block in the frame, the intra-BC prediction state for each intra-BC predicted video block in the frame, and other information for decoding video blocks in the current video frame.

[0081] The motion compensation unit 82 can also perform interpolation using interpolation filters, such as those used by the video encoder 20 during encoding of video blocks, to calculate interpolated values ​​for sub-integer pixels of the reference block. In this case, the motion compensation unit 82 can determine the interpolation filters used by the video encoder 20 based on the received syntax elements and use these interpolation filters to generate the prediction block.

[0082] The dequantization unit 86 dequantizes the quantized transform coefficients provided in the bitstream and entropy-decoded by the entropy decoding unit 80 using the same quantization parameters calculated by the video encoder 20 for each video block in the video frame to determine the degree of quantization. The inverse transform processing unit 88 applies an inverse transform (e.g., inverse DCT, inverse integer transform, or a conceptually similar inverse transform process) to the transform coefficients in order to reconstruct the residual block in the pixel domain.

[0083] After the motion compensation unit 82 or the intra-frame BC unit 85 generates a prediction block for the current video block based on vectors and other syntax elements, the adder 90 reconstructs the decoded video block for the current video block by adding the residual block from the inverse transform processing unit 88 to the corresponding prediction block generated by the motion compensation unit 82 and the intra-frame BC unit 85. A loop filter 91 (e.g., a deblocking filter, SAO filter, CCSAO filter, and / or ALF) may be located between the adder 90 and the DPB 92 for further processing of the decoded video block. In some examples, the loop filter 91 may be omitted, and the decoded video block may be directly provided to the DPB 92 by the adder 90. The decoded video block in a given frame is then stored in the DPB 92, which stores reference frames for subsequent motion compensation of the next video block. The DPB 92 or a separate memory device may also store the decoded video for later presentation on a display device (e.g., ...). Figure 1 On the display device 34).

[0084] In a typical video encoding and decoding process, a video sequence usually consists of an ordered set of frames or images. Each frame may include three sample arrays, denoted as SL, SCb, and SCr. SL is a two-dimensional array of luminance samples. SCb is a two-dimensional array of Cb chrominance samples. SCr is a two-dimensional array of Cr chrominance samples. In other instances, a frame may be monochrome and therefore consist of only a two-dimensional array of luminance samples.

[0085] like Figure 4AAs shown, the video encoder 20 (or more specifically, the segmentation unit 45) generates an encoded representation of a frame by first segmenting the frame into a set of CTUs. A video frame may include an integer number of CTUs ordered consecutively from left to right and top to bottom in raster scan order. Each CTU is the largest logical coding unit, and the width and height of the CTU are signaled by the video encoder 20 in a sequence parameter set such that all CTUs in the video sequence have the same size, such as 128×128, 64×64, 32×32, and 16×16. However, it should be noted that this application is not limited to a specific size. Figure 4B As shown, each CTU may include a CTB for the luma sample, two corresponding coding tree blocks for the chroma sample, and syntax elements for encoding the samples of the coding tree block. The syntax elements describe the properties of different types of units in the coded pixel block and how the video sequence can be reconstructed at the video decoder 30, including inter-frame prediction or intra-frame prediction, intra-frame prediction mode, motion vectors, and other parameters. In a monochrome image or an image with three separate color planes, the CTU may include a single coding tree block and syntax elements for encoding the samples of that coding tree block. The coding tree block can be an N×N sample block.

[0086] To achieve better performance, the video encoder 20 can recursively perform tree splitting on the coding tree blocks of the CTU, such as binary tree splitting, ternary tree splitting, quadtree splitting, or combinations thereof, and divide the CTU into smaller CUs. Figure 4C As described, the 64×64 CTU 400 is first divided into four smaller CUs, each with a block size of 32×32. Of these four smaller CUs, CU 410 and CU 420 are each divided into four CUs with a block size of 16×16. The two 16×16 CUs, 430 and CU 440, are further divided into four CUs with a block size of 8×8. Figure 4D Depicting as shown Figure 4C The final result of the CTU 400 partitioning process described in the figure is a quadtree data structure, where each leaf node of the quadtree corresponds to a CU of a corresponding size ranging from 32×32 to 8×8. Similar to... Figure 4B The CTU depicted in the image can include, for example, two corresponding coded blocks (CBs) of luminance and chrominance samples of the same size frame, as well as syntax elements for encoding the samples of the coded blocks. In monochrome images or images with three separate color planes, a CU can include a single coded block and a syntax structure for encoding the samples of the coded block. It should be noted that... Figure 4C and Figure 4DThe quadtree partitioning depicted is for illustrative purposes only, and a CTU can be split into multiple CUs based on quadtree / ternary / binary partitioning to adapt to varying local characteristics. In multi-type tree structures, a CTU is partitioned according to a quadtree structure, and each quadtree leaf CU can be further partitioned according to binary and ternary tree structures. Figure 4E As shown, a coded block with width W and height H has five possible segmentation types: quad segmentation, horizontal binary segmentation, vertical binary segmentation, horizontal triple segmentation, and vertical triple segmentation.

[0087] In some implementations, the video encoder 20 may further segment the coded blocks of the CU into one or more (M×N) PBs. A PB is a rectangular (square or non-square) sample block to which the same prediction (inter-frame or intra-frame) is applied. The PU of the CU may include a PB for luma samples, two corresponding PBs for chroma samples, and syntax elements for predicting the PBs. In a monochrome image or an image with three separate color planes, a PU may include a single PB and a syntax structure for predicting the PBs. The video encoder 20 may generate predicted luma blocks, predicted Cb blocks, and predicted Cr blocks for each PU of the CU, representing the luma PB, Cb PB, and Cr PB.

[0088] Video encoder 20 can generate prediction blocks for a PU using intra-frame prediction or inter-frame prediction. If video encoder 20 uses intra-frame prediction to generate prediction blocks for a PU, then video encoder 20 can generate prediction blocks for a PU based on decoded samples of the frame associated with the PU. If video encoder 20 uses inter-frame prediction to generate prediction blocks for a PU, then video encoder 20 can generate prediction blocks for a PU based on decoded samples of one or more frames other than the frame associated with the PU.

[0089] After the video encoder 20 generates predicted luminance blocks, predicted Cb blocks, and predicted Cr blocks for one or more PUs of the CU, the video encoder 20 can generate luminance residual blocks for the CU by subtracting the predicted luminance blocks of the CU from the original luminance coding blocks of the CU, such that each sample in the luminance residual block of the CU indicates the difference between a luminance sample in one of the predicted luminance blocks of the CU and a corresponding sample in the original luminance coding block of the CU. Similarly, the video encoder 20 can generate Cb residual blocks and Cr residual blocks for the CU, respectively, such that each sample in the Cb residual block of the CU indicates the difference between a Cb sample in one of the predicted Cb blocks of the CU and a corresponding sample in the original Cb coding block of the CU, and each sample in the Cr residual block of the CU indicates the difference between a Cr sample in one of the predicted Cr blocks of the CU and a corresponding sample in the original Cr coding block of the CU.

[0090] In addition, such as Figure 4CAs shown, the video encoder 20 can use quadtree partitioning to decompose the luminance residual block, Cb residual block, and Cr residual block of the CU into one or more luminance transform blocks, Cb transform blocks, and Cr transform blocks, respectively. A transform block is a rectangular (square or non-square) sample block to which the same transform is applied. A TU of the CU can include a transform block of the luminance samples, two corresponding transform blocks of the chrominance samples, and syntax elements for transforming the transform block samples. Therefore, each TU of the CU can be associated with a luminance transform block, a Cb transform block, and a Cr transform block. In some examples, the luminance transform block associated with a TU can be a sub-block of the CU's luminance residual block. A Cb transform block can be a sub-block of the CU's Cb residual block. A Cr transform block can be a sub-block of the CU's Cr residual block. In a monochrome image or an image with three separate color planes, a TU can include a single transform block and syntax structures for transforming the samples of that transform block.

[0091] The video encoder 20 can apply one or more transforms to the luminance transform block of the TU to generate a luminance coefficient block for the TU. The coefficient block can be a two-dimensional array of transform coefficients. The transform coefficients can be scalars. The video encoder 20 can apply one or more transforms to the Cb transform block of the TU to generate a Cb coefficient block for the TU. The video encoder 20 can apply one or more transforms to the Cr transform block of the TU to generate a Cr coefficient block for the TU.

[0092] After generating coefficient blocks (e.g., luminance coefficient blocks, Cb coefficient blocks, or Cr coefficient blocks), video encoder 20 can quantize the coefficient blocks. Quantization typically refers to the process of quantizing transform coefficients to potentially reduce the amount of data used to represent the transform coefficients, thereby providing further compression. After quantizing the coefficient blocks, video encoder 20 can entropy encode the syntax elements indicating the quantized transform coefficients. For example, video encoder 20 can perform CABAC on the syntax elements indicating the quantized transform coefficients. Finally, video encoder 20 can output a bitstream comprising a bit sequence that forms a representation of coded frames and associated data; the bitstream is stored in storage device 32 or transmitted to target device 14.

[0093] After receiving the bitstream generated by the video encoder 20, the video decoder 30 can parse the bitstream to obtain syntax elements. The video decoder 30 can reconstruct frames of video data, at least in part, based on the syntax elements obtained from the bitstream. The process of reconstructing the video data is generally the inverse of the encoding process performed by the video encoder 20. For example, the video decoder 30 can perform an inverse transform on the coefficient block associated with the TU of the current CU to reconstruct the residual block associated with the TU of the current CU. The video decoder 30 also reconstructs the coded blocks of the current CU by adding samples of the predicted blocks of the PU for the current CU to corresponding samples of the transformed blocks of the TU of the current CU. After reconstructing the coded blocks for each CU of the frame, the video decoder 30 can reconstruct the frame.

[0094] As mentioned above, video coding primarily uses two modes (i.e., intra-frame prediction and inter-frame prediction) to achieve video compression. It should be noted that IBC can be considered either intra-frame prediction or a third mode. Between the two modes, inter-frame prediction contributes more to coding efficiency than intra-frame prediction because it uses motion vectors to predict the current video block based on a reference video block.

[0095] However, with continuously improving video data capture technologies and finer video tile sizes used to preserve details in video data, the amount of data required to represent the motion vectors for the current frame has also increased significantly. One way to overcome this challenge benefits from the fact that not only do a set of neighboring CUs in both the spatial and temporal domains have similar video data for prediction purposes, but the motion vectors between these neighboring CUs are also similar. Therefore, the motion information of spatially neighboring CUs and / or temporally co-located CUs can be used as an approximation of the motion information (e.g., motion vectors) of the current CU (also known as the "Motion Vector Prediction" (MVP) of the current CU) by exploring their spatial and temporal correlations.

[0096] Instead of the above combination Figure 2 The method described involves encoding the actual motion vector of the current CU, determined by the motion estimation unit 42, into the video bitstream, and subtracting the predicted motion vector value of the current CU from the actual motion vector of the current CU to produce the motion vector difference (MVD) for the current CU. By doing so, it is not necessary to encode the motion vector determined by the motion estimation unit 42 for each CU of the frame into the video bitstream, and the amount of data used to represent motion information in the video bitstream can be significantly reduced.

[0097] Similar to the process of selecting a prediction block in a reference frame during inter-frame prediction of a coded block, both the video encoder 20 and the video decoder 30 need to employ a set of rules to construct a motion vector candidate list (also known as a "merging list") for the current CU using those potential candidate motion vectors associated with spatially neighboring CUs and / or temporally co-located CUs. Then, a member is selected from the motion vector candidate list as the motion vector prediction value for the current CU. By doing so, it is not necessary to send the motion vector candidate list itself from the video encoder 20 to the video decoder 30, and the index of the selected motion vector prediction value within the motion vector candidate list is sufficient for both the video encoder 20 and the video decoder 30 to encode and decode the current CU using the same motion vector prediction value from the motion vector candidate list.

[0098] Typically, the basic intra prediction scheme used in VVC is almost identical to that in HEVC, with the difference being the further extension, addition, and / or improvement of several prediction tools, such as extended intra prediction using wide-angle intra mode, multi-reference line (MRL) intra prediction, PDPC, intra-segment sub-partition (ISP) prediction, CCLM prediction, MIP, and intra template matching prediction (TMP).

[0099] In ECM, intra-template matching prediction (intra TMP) is used to improve the compression efficiency of intra-frame encoding and decoding. The intra-frame TMP technique and its improvement methods are presented below.

[0100] Intra-frame template matching prediction

[0101] Intra-Template Matching Prediction (Intra TMP) is a special intra-prediction mode that copies the best prediction block from the reconstructed portion of the current frame that best matches the current template. For a predefined search range, the encoder searches for the template most similar to the current template in the reconstructed portion of the current frame and uses the corresponding block as the prediction block. The encoder then signals the use of this mode and performs the same prediction operation on the decoder side.

[0102] By comparing the L-shaped causal neighbors of the current block with Figure 6 The prediction signal is generated by matching another block in a predefined search region, which consists of the following:

[0103] R1: Current CTU

[0104] R2: Top left CTU

[0105] R3: Above CTU

[0106] R4: Left CTU

[0107] The sum of absolute differences (SAD) is used as the cost function.

[0108] Within each region, the decoder searches for the template with the minimum SAD relative to the current template and uses its corresponding block as the prediction block.

[0109] The dimensions of all regions (SearchRange_w, SearchRange_h) are set proportionally to the block size (BlkW, BlkH) to ensure a fixed number of SAD comparisons per pixel. That is:

[0110] SearchRange_w=a*BlkW

[0111] SearchRange_h = a * BlkH

[0112] Here, 'a' is a constant that controls the gain / complexity tradeoff. In practice, 'a' equals 5.

[0113] For CUs with width and height dimensions less than or equal to 64, enable the intra-frame template matching tool. This maximum CU size for intra-frame template matching is configurable.

[0114] When the current CU does not use DIMD, the intra-template matching prediction mode is transmitted at the CU level using a dedicated flag.

[0115] Block vector candidates for intra-frame TMP derivation for IBC

[0116] In this method, the block vector (BV) derived from IntraTMP (Intra-Temporal Matching Prediction) is used for Intra-Block Copy (IBC). The stored IntraTMP BV and IBC BV of neighboring blocks are used as spatial BV candidates in the construction of the IBC candidate list.

[0117] The IntraTMP block vector is stored in the IBC block vector buffer, and the current IBC block can use both the IBC BV and the IntraTMP BV of neighboring blocks as BV candidates in the IBC BV candidate list, such as... Figure 7 As shown.

[0118] Intra-frame TMP using a linear filter model

[0119] In JVET-AC0109, a linear filter model was proposed for the prediction of intra-frame TMP (Intra-Frame TMP-FLM). The proposed 6-tap filter consists of a 5-tap plus shape spatial component and a bias term. The input of the filter's 5-tap spatial component consists of the center (C) sample of the reference block (located at the position corresponding to the sample in the current block to be predicted) and its top / north (N), bottom / south (S), left / west (W), and right / east (E) neighboring samples, as shown below. Figure 8 As shown.

[0120] Deviation term B represents the scalar offset between the input and output and is set to the intermediate brightness value (512 for 10-bit content).

[0121] The filter output is calculated as follows:

[0122] predLumaVal=c0C+c1N+c2S+c3E+c4W+c5B

[0123] The filter coefficients ci are calculated by minimizing the MSE between the reference template and the current template, such as... Figure 9 As shown. The area indicated by the dots needs to be expanded to support the "sidesampling points" of the plus shape space filter, and the unavailable areas need to be filled.

[0124] Minimizing the MSE is performed by computing the autocorrelation matrix of the reference template input and the current template output. The autocorrelation matrix is ​​then decomposed using LDL, and the final filter coefficients are computed using back-substitution.

[0125] The use of intra-frame TMP-FLM mode is signaled using an encoded CU-level flag. Specifically, intra-frame TMP-FLM is considered a sub-mode of intra-frame TMP. That is, the intra-frame TMP-FLM flag is only signaled when the intra-frame TMP flag is true.

[0126] Combination of intra-frame and intra-frame TMP

[0127] At the JVET-AC conference, several methods were proposed to fuse intra-frame prediction and intra-frame TMP prediction. For example, JVET-AC0097 proposed using geometric partitioning to combine intra-frame prediction with intra-frame TMP. JVET-AC0201 proposed a spatial CIIP mode, in which intra-frame prediction and intra-frame TMP can be fused in a CIIP manner.

[0128] CCLM Forecast

[0129] To reduce cross-component redundancy, the CCLM prediction mode is used in VVC, where the chromaticity samples of the CU are based on the reconstructed luminance samples of the CU.L (i,j) is predicted using the following linear model:

[0130] pred C (i,j)=α·rec L ′(i,j)+β (1)

[0131] Among them, pred C (i,j) represents the predicted chromaticity sample points in the CU, rec L ′(i,j) represents the reconstructed brightness sample rec. L (i,j) represents the downsampled reconstructed luminance sample of the CU obtained after downsampling, where α and β are linear model coefficients obtained from at most four neighboring chroma samples and their corresponding downsampled luminance samples. Assuming the current chroma block size is W×H, W' and H' can be obtained as follows:

[0132] –When applying LM mode, W' = W, H' = H;

[0133] – When applying LM_A mode, W' = W + H;

[0134] – When applying the LM_L mode, H' = H + W.

[0135] In the LM mode, the upper and left samples of the CU are used together to calculate the linear model coefficients; in the LM_A mode, only the upper samples of the CU are used to calculate the linear model coefficients; and in the LM_L mode, only the left samples of the CU are used to calculate the linear model coefficients.

[0136] If the position of the upper sample point of the chroma block is represented as S[0,-1]…S[W'-1,-1] and the position of the left sample point of the chroma block is represented as S[-1,0]…S[-1,H'-1], then the positions of the four adjacent chroma samples are selected as follows:

[0137] –When applying LM mode and both the top and left sample points are available, select S[W' / 4,-1], S[3*W' / 4,-1], S[-1,H' / 4] and S[-1,3*H' / 4] as the positions of the four adjacent chromaticity sample points;

[0138] – When applying LM_A mode and the top sample is available, or when only the top sample is available, select S[W' / 8,-1], S[3*W' / 8,-1], S[5*W' / 8,-1] and S[7*W' / 8,-1] as the positions of the four adjacent chromaticity sample points;

[0139] – When applying LM_L mode and the left sample is available, or when only the left sample is available, select S[-1,H' / 8], S[-1,3*H' / 8], S[-1,5*H' / 8] and S[-1,7*H' / 8] as the positions of the four neighboring chromaticity samples.

[0140] The downsampling operation obtains four neighboring brightness samples corresponding to the selected position, and these four neighboring brightness samples are compared four times to find the two larger values: x 0 A and x 1 A And two smaller values: x 0 B and x 1 B The chromaticity sample values ​​corresponding to these two larger and two smaller values ​​are represented as y. 0 A ,y 1 A ,y 0 B and y 1 B Then obtain X as follows: a X b Y a and Y b :

[0141] X a =(x 0 A +x 1 A +1)>>1;

[0142] X b =(x 0 B + x 1 B +1)>>1; (2)

[0143] Y a =(y 0 A +y 1 A +1)>>1;

[0144] Y b =(y 0 B +y 1 B +1)>>1.

[0145] Finally, the linear model coefficients α and β are obtained according to the following formula.

[0146]

[0147] β=Y b -α·X b

[0148] Figure 10 The diagram shows the positions of the left and top sample points of the CU involved in the CCLM mode, including the positions of the left and top sample points of the N×N chromaticity block 501 in the CU, and the positions of the left and top sample points of the 2N×2N luminance block 503 in the CU.

[0149] The above parameter calculations are performed as part of the decoding process, therefore no syntax elements are used to transmit the values ​​of α and β from the video encoder 20 to the video decoder 30.

[0150] Convolutional cross-component intra-prediction model

[0151] In this method, a convolutional cross-component model (CCCM) is applied to predict chroma samples from reconstructed luminance samples, similar in spirit to the current CCLM model. As with CCLM, when using chroma subsampling, the reconstructed luminance samples are downsampled to match a lower-resolution chroma grid. Similar to CCLM, top reference samples, left reference samples, or both top and left reference samples are used as templates for model derivation.

[0152] Furthermore, similar to CCLM, a single-model or multi-model variant of CCCM can be selected. The multi-model variant uses two models: one model is derived for samples above the average luminance reference value, and the other model is derived for the remaining samples (following the spirit of CCLM design). For PUs with at least 128 reference samples available, the multi-model CCCM mode can be selected.

[0153] Convolutional Filter

[0154] The convolutional 7-tap filter consists of a 5-tap plus shape spatial component, a nonlinear term, and a bias term. The input to the filter's 5-tap spatial component comprises the center (C) luminance sample co-located with the chrominance sample to be predicted, and its upper / north (N), lower / south (S), left / west (W), and right / east (E) neighboring samples, as shown below. Figure 8 As shown.

[0155] The nonlinear term P is represented as the second power of the center brightness sample C, scaled to the range of sample values ​​for the content:

[0156] P = (C*C + midVal) >> Bit depth

[0157] That is, for a 10-digit content, the calculation method is as follows:

[0158] P = (C * C + 512) >> 10

[0159] The deviation term B represents the scalar offset between the input and output (similar to the offset term in CCLM) and is set to an intermediate chroma value (512 for 10-bit content).

[0160] The filter output is calculated as the convolution between the filter coefficients ci and the input values, and is truncated to the effective chromaticity sample range:

[0161] predChromaVal=c0C+c1N+c2S+c3E+c4W+c5P+c6B

[0162] Calculation of filter coefficients

[0163] The filter coefficients ci are calculated by minimizing the MSE between the predicted chromaticity samples and the reconstructed chromaticity samples in the reference region. Figure 11 The reference region is shown, consisting of six rows of chroma samples above and to the left of the PU. The reference region extends one PU width to the right of the PU boundary and one PU height below the PU boundary. The region is adjusted to include only available samples. The region indicated by the dots needs to be expanded to support the "side samples" of the plus shape space filter, and unavailable areas need to be filled.

[0164] MSE minimization is performed by calculating the autocorrelation matrix of the luminance input and the cross-correlation vector between the luminance input and chrominance output. The autocorrelation matrix is ​​decomposed using LDL, and the final filter coefficients are calculated using back-substitution. This process largely follows the calculation of ALF filter coefficients in ECM, but LDL decomposition is chosen instead of Cholesky decomposition to avoid the use of square root operations.

[0165] The autocorrelation matrix is ​​calculated using reconstructed values ​​from luma and chroma samples. These samples are full-range (e.g., between 0 and 1023 for 10-bit content), resulting in relatively large values ​​in the autocorrelation matrix. This requires high-bit-depth operations during model parameter calculation. A proposed approach is to remove a fixed offset from the luma and chroma samples in each PU for each model. This reduces the size of the values ​​used in model creation and allows for a reduction in the precision required for fixed-point arithmetic operations. Therefore, 16-bit decimal precision is proposed instead of the 22-bit precision of the original CCCM implementation.

[0166] For simplicity, the reference sample values ​​outside the top-left corner of the PU are used as offsets (offsetLuma, offsetCb, and offsetCr). These fixed values ​​are subtracted from the sample values ​​used in model creation and final prediction (i.e., the luminance and chrominance in the reference region and the luminance in the current PU), as follows:

[0167] ·C'=C–offsetLuma

[0168] ·N'=N–offsetLuma

[0169] ·S'=S–offsetLuma

[0170] ·E'=E–offsetLuma

[0171] ·W'=W–offsetLuma

[0172] P' = nonLinear(C')

[0173] B = midValue = 1 << (bit depth - 1)

[0174] Furthermore, the following equations are used to predict chromaticity values, where offsetChroma is equal to offsetCr and offsetCb for the Cr and Cb components, respectively:

[0175] predChromaVal=c0C'+c1N'+c2S'+c3E'+c4W'+c5P'+c6B+offsetChroma

[0176] To avoid any additional sample-level computations, the luminance offset is removed during luminance reference sample interpolation. For example, this can be achieved by replacing the rounding term used in luminance reference sample interpolation with an updated offset that includes both the rounding term and offsetLuma. The chrominance offset can also be removed by directly subtracting it from the reference chrominance sample. Alternatively, the effect of the chrominance offset can be removed from the cross-component vectors, yielding the same result. To add the chrominance offset back to the output of the convolution prediction operation, it is added to the bias term of the convolutional model.

[0177] The calculation of CCCM model parameters requires division operations. Division is not always considered implementation-friendly. Division is replaced with multiplication (via scaling factors) and shift operations, where the scaling factor and the number of shifts are calculated based on the denominator, similar to the method used when calculating CCLM parameters.

[0178] Gradient linear model

[0179] For the YUV 4:2:0 color format, the Gradient Linear Model (GLM) method can be used to predict chromaticity samples from the luminance sample gradient. Two modes are supported: two-parameter GLM mode and three-parameter GLM mode.

[0180] Compared to CCLM, two-parameter GLM does not utilize downsampled luminance values, but instead uses luminance sample gradients to derive a linear model. Specifically, when applying two-parameter GLM, the input to the CCLM process (i.e., the downsampled luminance sample L) is replaced with the luminance sample gradient G. Other parts of CCLM (e.g., parameter derivation, linear transformation of predicted samples) remain unchanged.

[0181] C = α·G + β

[0182] In a three-parameter GLM, different parameters can be used to predict chromaticity samples based on both the luminance sample gradient and the downsampled luminance value. The model parameters of the three-parameter GLM are derived from 6 rows and 6 columns of adjacent samples using an MSE minimization method based on LDL decomposition, as used in CCCM.

[0183] C = α0·G + α1·L + α2·β

[0184] For signal transmission, when CCLM mode is enabled for the current CU, a flag is transmitted to indicate whether GLM is enabled for both Cb and Cr components; if GLM is enabled, another flag is transmitted to indicate which of the two GLM modes is selected, and a syntax element is further transmitted to select one of the four gradient filters for gradient calculation. Figure 12 The image shows four gradient filters for the GLM.

[0185] Bit stream signal transmission

[0186] The usage of this mode is signaled via the PU-level flags of the CABAC codec. This is supported by including a new CABAC context. In terms of signaling, CCCM is treated as a sub-mode of CCLM. That is, the CCCM flag is signaled only when the intra-prediction mode is LM_CHROMA.

[0187] While existing intra-frame TMP schemes offer significant improvements to intra-frame encoding and decoding in ECM, their design can still be further improved. For example, this disclosure recognizes that current intra-frame TMP designs have the following drawbacks:

[0188] First, the current intra-frame TMP only utilizes integer pixel-based template search, ignoring fractional interpolation; that is, it only uses integer pixel template matching blocks as prediction blocks. It should be understood that fractional interpolation can improve the prediction accuracy of inter-frame prediction, therefore ignoring fractional interpolation makes the current intra-frame TMP inefficient.

[0189] Second, current intra-frame TMPs utilize only one assumption: selecting the best matching block that minimizes the matching cost as the final prediction. However, a single assumption can be susceptible to noise, leading to inaccurate predictions. Furthermore, it has been demonstrated that introducing multiple assumptions in inter-frame prediction can improve prediction accuracy. Based on the above analysis, proposing a multi-assumption-based intra-frame TMP is reasonable.

[0190] Third, in JVET-AC0109, a linear filter model is used to further filter the intra-frame TMP block. More specifically, the intra-frame TMP block is the reference block with the minimum template matching cost within the search range. However, as revealed above, intra-frame TMP with a single hypothesis may not be efficient enough. Therefore, a combination of multiple hypotheses with the linear filter model is proposed.

[0191] Fourth, in the current ECM, intra-frame TMP is only enabled for the luma component, without considering the chroma component. Therefore, it is proposed to enable intra-frame TMP for the chroma component.

[0192] Fifth, in current methods that combine intra-frame and intra-frame TMPs, the fused predictions are directly used as the final predictions without further refinement.

[0193] Sixth, in the current intra-frame TMP, prediction is based on prediction blocks, and the granularity of the prediction blocks may be too coarse, resulting in low efficiency.

[0194] This disclosure proposes a method to further improve the compression efficiency of intra-frame TMPs in ECM. The main features of the method proposed in this disclosure are summarized below.

[0195] First, to improve the prediction accuracy of intra-frame TMP, fractional pixel template matching is proposed. Integer template matching blocks can be further refined by using fractional interpolation.

[0196] Second, to improve the accuracy of intra-frame TMP, a multi-hypothesis-based intra-frame TMP scheme is proposed. In the proposed scheme, multiple template matching blocks are searched and weighted to generate the final prediction block. Based on the derivation of the weighting factors, the proposed multi-hypothesis intra-frame TMP can be divided into two categories: fixed multi-hypothesis intra-frame TMP and adaptive intra-frame TMP. Third, to further improve the prediction accuracy of intra-frame TMP using a linear filter model, multiple hypotheses are introduced; that is, intra-frame TMP is performed using multiple hypotheses, and then a linear filter model is applied.

[0197] Fourth, in order to further improve the prediction of chroma components, it is proposed to enable intra-frame TMP for chroma components.

[0198] Fifth, in order to utilize the cross-component correlation between the luminance and chrominance components, it is proposed to combine cross-component prediction with intra-frame TMP of the chrominance component.

[0199] Sixth, in order to further improve the prediction of combined intra-frame and intra-frame TMP, a linear filter model is used to refine the fused prediction.

[0200] Seventh, in order to improve the prediction granularity, intra-frame TMP based on sub-blocks is utilized.

[0201] Fractional intra-frame TMP

[0202] In the current intra-frame TMP mode, an L-shaped template is used to search for prediction blocks from the reconstructed region in the current frame. During template matching, the best prediction block that results in the minimum matching cost is identified and used as the final prediction block. In the current example of intra-frame TMP, template matching is based on integer pixels. To further improve the encoding and decoding performance of the intra-frame TMP mode, this disclosure provides an example of using fractional pixel template matching for the intra-frame TMP mode. The interpolation accuracy of fractional pixel template matching can be, but is not limited to, half a pixel, quarter a pixel, etc.

[0203] In one or another embodiment, an index value can be signaled in the bitstream to indicate the interpolation accuracy used. The interpolation accuracy index can be signaled at the PPS, SPS, or stripe header.

[0204] In one or another embodiment, the fractional intra-frame TMP can be divided into two steps. In the first step, a template matching process in the current ECM is performed to identify the best integer prediction template. In the second step, fractional templates around the integer template are interpolated, and fractional pixel template matching is performed. Figure 13 An example of half-pixel template matching is provided, where A i,j Represents integer pixels, and b i,j h i,j and j i,j This represents a half-pixel. After identifying the best integer prediction template, interpolation is then performed on the eight half-pixel prediction templates. The distances between the current template and the best integer prediction template, as well as the eight half-pixel prediction templates, are calculated and compared. The prediction template that results in the minimum template matching cost is identified and used as the best prediction template. The corresponding prediction block of the best prediction template is used as the final prediction for the current block.

[0205] Multiple Hypothesis Intra-Frame TMP

[0206] In current ECM, only the best template matching block resulting in the minimum template matching cost is used for prediction. In some examples, optional template matching blocks are selected based on ascending order of template matching cost, and the template resulting in a lower template matching cost than other candidates is selected as the best template matching block / CU. To further improve the prediction accuracy of intra-frame TMP, this disclosure provides a multi-hypothesis prediction method. In multi-hypothesis intra-frame TMP, more than one prediction block candidate is used and weighted to generate the final prediction for the current block. It is assumed that N prediction block candidates are used. In some examples, block candidates are searched within a predefined search range, as in current intra-frame TMP.

[0207] Predicted block candidate derivation

[0208] In one and another embodiment, candidate prediction blocks are searched and selected based on a criterion of minimizing template matching cost; that is, the top N candidates with the minimum template matching cost are selected. Template matching cost may not be limited to SAD (Sum of Absolute Differences) and SSE (Sum of Squared Errors).

[0209] Fixed Multiple Hypothesis Intra-Frame TMP

[0210] In this embodiment, the weighting factors used to generate the final prediction block are predefined and fixed on both the encoder and decoder sides. As an example, equal weighting factors can be used, i.e., all candidate blocks have a weighting factor of 1 / N.

[0211] Adaptive Multihypothesis Intra-Frame TMP

[0212] To adapt to the different characteristics of video content, an example of an adaptive multi-hypothesis intra-frame TMP method is also provided.

[0213] Method 1

[0214] In one and another embodiment, the weighting factor can be derived based on the template matching cost. The template matching costs of the N candidates are represented as C1, C2, ..., C... N The weighting factor is calculated according to the following formula.

[0215]

[0216] It should be noted that SAD and SSE can be used (but are not limited to) to measure the cost of template matching.

[0217] Method 2

[0218] In yet another embodiment, the weighting factors can be derived on the encoder side and then transmitted as signals to the decoder in the bitstream. The N prediction block candidates are represented as P1, P2, ..., P... NAnd let the current block be represented as X, then the weighting factor can be solved using the following formula:

[0219]

[0220] Equation (5) can be solved using the Wiener-Hopf equation as an ALF. The derived filter coefficients are then quantized to integer type and transmitted as signals at the block level.

[0221] Method 3

[0222] In yet another embodiment, weighting factors are derived based on templates, and these derived weighting factors are applied to prediction block candidates to generate the final prediction block. The templates for the prediction candidates are denoted as T1, T2, ..., T. N And let the template of the current block be T. Then the weighting factor can be derived using the following formula:

[0223]

[0224] Equation (6) can be solved using the Wiener-Hopf equations. Then, the final predicted block can be calculated as follows: Among them, P i This represents the i-th prediction block candidate.

[0225] Method 4

[0226] Intra-frame TMP mode utilizes nonlocal correlation to improve prediction accuracy, where similar blocks are searched and used to generate the final prediction block. In this embodiment, an example combining nonlocal mean filtering with multi-hypothesis intra-frame TMP is provided, described below. In the first step, as performed in intra-frame TMP of ECM, N prediction block candidates are searched and identified. In the second step, weighting factors are calculated according to the following formula.

[0227]

[0228] Among them, D i The distance between the template of the i-th prediction block candidate and the template of the current block is used to measure the distance, h is used as the weighting strength, and Z[i] is the normalization constant:

[0229]

[0230] In order to calculate the weighting factor in equation (7), the weighting strength or weighting intensity should first be determined. Several methods are provided in this disclosure to determine the weighting intensity.

[0231] In the first approach, a candidate list of weighted strength values, consisting of some typical weighted strength values, is defined and fixed on both the encoder and decoder sides. On the encoder side, rate-distortion optimization is used to examine the weighted strength values, identify the optimal weighted strength value, and transmit it as a signal to the decoder side in the bitstream.

[0232] In the second method, the templates of the predicted block candidates and the template of the current block are used to estimate the weighted intensity value. The templates of the predicted candidates are denoted as T1, T2, ..., T. N And let T be the template of the current block. Then, the weighted strength value can be solved using the following formula:

[0233]

[0234] In the third method, the weighted intensity value can be estimated using the QP value and the variance of the template of the current block. That is, the relationship between the weighted intensity value, the QP value and the template variance can be fitted offline.

[0235] Method 5

[0236] To better utilize the nonlocal correlations in intra-frame TMPs, this embodiment employs singular value decomposition (SVD) to generate the final prediction block from the prediction block candidates. The width and height of the current block are represented as W and H, respectively, and the area of ​​the current block is represented as d = H × W.

[0237] Step 1. Search and identify K candidate y-blocks for prediction i This is done, as in the intra-frame TMP of ECM.

[0238] Step 2. The K predicted block candidates of the current block y are constructed into a block group G and arranged as a matrix:

[0239]

[0240] Among them, Y G It is a matrix of size d×K formed by arranging each candidate in group G as a column vector.

[0241] Step 3. For matrix Y G Perform SVD decomposition.

[0242]

[0243] Step 4. For the singular value matrix Λ G Apply soft threshold operation.

[0244]

[0245] Among them, softTh() shrinks Λ using a threshold τ. G A function of the diagonal elements. For ΛG The k-th diagonal element in the array is obtained through the nonlinear function D. τ(k) Shrink it at level τ(k):

[0246] D τ(k) :λ k,τ(k) =max(|λ k |-τ(k),0) (13)

[0247] Λ G,τ It is formed by the contracted singular value λ located on the diagonal. k,τ(k) The matrix formed by these elements.

[0248] Step 5. Perform inverse SVD to obtain the filtered block group.

[0249]

[0250] One of the key steps is determining the threshold for each diagonal element in step 4. In this disclosure, the threshold is calculated as follows. For each group of image patches, the threshold is estimated using the following formula:

[0251]

[0252] Where, σ n,G It is the standard deviation of the noise, and σ x,G,k Let be the standard deviation of the original block in the k-th dimension of the SVD space of group G. The deviation of the original block in the SVD space is estimated according to the following formula.

[0253]

[0254] in, yes The kth singular value. When σ x,G,k When the value is zero, the soft thresholding operation is skipped. Additionally, a power function parameterized with α and β is used to estimate the noise bias using the bias of the prediction block.

[0255] σ n =α×σ y β (17)

[0256] Where σ y The calculation is as follows:

[0257]

[0258] Here, y k (i) represents the candidate vector of the predicted block y k The i-th pixel.

[0259] Predicted block candidate derivation

[0260] The first step of multi-hypothesis intra-frame TMP is to derive N reference block candidates, which are used to generate the final prediction block. Several methods are provided below to derive the reference block candidates for multi-hypothesis intra-frame TMP.

[0261] In the first embodiment, candidate predicted blocks are searched and selected based on a criterion of minimizing template matching cost; that is, the top N candidates that produce the minimum template matching cost are selected. The matching cost can be the sum of absolute differences (SAD), the sum of squared differences (SSD), or any other means used to measure the similarity between two blocks.

[0262] In the second embodiment, the derivation is first expressed as follows: The intermediate prediction blocks are then used to derive reference block candidates for multi-hypothesis intra-TMP. Several methods for deriving intermediate prediction blocks are proposed in this disclosure.

[0263] a) In the first method, N reference block candidates are first derived according to the criterion of minimizing template matching cost described in the first embodiment. Then, these N reference block candidates are used to generate intermediate prediction blocks. The method used to generate intermediate prediction blocks can be any method, including but not limited to the fixed multi-hypothesis intra-frame TMP and adaptive multi-hypothesis intra-frame TMP introduced in the previous section.

[0264] b) In the second method, the intra-frame TMP prediction value (i.e., the reference block that produces the minimum template matching cost) is directly used as the intermediate prediction block.

[0265] c) In the third method, intermediate prediction blocks are determined on the encoder side by minimizing rate distortion costs, and the block vector (BV) of the intermediate prediction blocks is transmitted to the decoder as a signal in the bitstream.

[0266] After identifying intermediate prediction blocks, these blocks are used to search for reference block candidates and perform multi-hypothesis intra-frame TMP. Several methods are proposed to identify reference block candidates and perform multi-hypothesis intra-frame TMP.

[0267] a) In the first method, N reference block candidates are searched directly using intermediate predicted blocks by minimizing the distance between the reference block and the intermediate predicted blocks. The top N reference blocks that produce the minimum distance are selected as reference block candidates. This distance can be the sum of absolute differences (SAD), the sum of squared differences (SSD), or any other means used to measure the similarity between two blocks. After identifying the reference block candidates, any method can be used to generate the final predicted block, including but not limited to the fixed multi-hypothesis intra-TMP method and the adaptive multi-hypothesis intra-TMP method described in the previous sections.

[0268] (b) In the second method, the intermediate region (intermediate prediction block along with neighboring templates) is directly used to search for N reference block candidates by minimizing the distance between the reference region and the intermediate region. The reference region includes the reference block and its corresponding neighboring templates. The top N reference blocks that generate the minimum distance are selected as reference block candidates. This distance can be the sum of absolute differences (SAD), the sum of squared differences (SSD), or any other means used to measure the similarity between two blocks. After identifying the reference block candidates, any method can be used to generate the final prediction block, including but not limited to the fixed multi-hypothesis intra-TMP method and the adaptive multi-hypothesis intra-TMP method described in the previous sections.

[0269] c) In the third method, the intermediate prediction block is first divided into several sub-blocks, and then a multi-hypothesis intra-frame TMP is performed based on the sub-blocks, such as... Figure 17 As shown. In this disclosure, sub-blocks in intermediate prediction blocks are referred to as intermediate sub-blocks. For each intermediate sub-block, N reference sub-block candidates are searched by minimizing the distance between the reference sub-block and the intermediate sub-block. The top N reference sub-blocks that produce the minimum distance are selected as reference sub-block candidates. This distance can be the sum of absolute differences (SAD), the sum of squared differences (SSD), or any other means used to measure the similarity between two blocks. After identifying the reference sub-block candidates, any method can be used to generate the final predicted sub-block, including but not limited to the fixed multi-hypothesis intra-TMP method and the adaptive multi-hypothesis intra-TMP method described in the previous sections.

[0270] d) In the fourth method, multi-hypothesis intra-frame TMP is adaptively performed for each pixel in the intermediate prediction block, i.e., pixel-level multi-hypothesis intra-frame TMP.

[0271] The following example illustrates the pixel-level multi-hypothesis intra-frame TMP method.

[0272] In the first example, intermediate prediction blocks or intermediate regions (intermediate prediction blocks together with neighboring templates) are used to identify N candidate reference blocks. The intermediate prediction blocks and the N reference blocks are weighted to generate the final prediction block. A weighting factor is calculated for each pixel. Let Rk be the pixel at (x,y) of the k-th reference block and the intermediate prediction block. k [x,y] and The weighting factor for the pixel corresponding to the i-th reference block is calculated as follows:

[0273]

[0274] Note that here, ω0 is the weighting factor for the intermediate prediction blocks. i This is the distance between the current pixel and its corresponding pixel in the reference block.

[0275] Another method for calculating the weighting factor is shown below.

[0276] and

[0277] Here, h is used as the weighting degree, which can be a fixed value or adaptively determined for each block / pixel.

[0278] In the second example, when predicting the pixel at position (x,y), the sub-block centered at (x,y) is used to find the reference sub-block for the current pixel by minimizing the distance between the current sub-block and the reference sub-block, such as... Figure 18 As shown. The distance between the current sub-block and the reference sub-block is used to calculate the weighting factor of the reference sub-block. In this example, the two methods for calculating the weighting factor from the first example can be used.

[0279] In the third example, when predicting the pixel at position (x,y), the sub-block centered at (x,y) is used to find the reference sub-block for the current pixel by minimizing the distance between the current sub-block and the reference sub-block, such as... Figure 18 As shown. The distance between the current pixel at the center of the current sub-block and the reference pixel at the center of the reference sub-block is used to calculate the weighting factor of the reference sub-block. In this example, the two methods for calculating the weighting factor from the first example can be used. In one or more examples, the size of the reference sub-block or the current sub-block is (2n+1)*(2n+1), where n is a non-negative integer, thus well defining the pixel at the center of the reference sub-block or the pixel at the center of the current sub-block.

[0280] In the fourth example, when predicting the pixel at position (x,y), the sub-block centered at (x,y) is used to find the reference sub-block for the current pixel by minimizing the distance between the current sub-block and the reference sub-block, such as... Figure 18 As shown. The current sub-block and the reference sub-block are used to generate the prediction sub-block using the Singular Value Decomposition (SVD) method introduced in the previous section. The pixel at the center of the prediction sub-block is used as the final prediction pixel at position (x,y).

[0281] Multiple Hypothesis Intra-Frame TMP Signal Transmission

[0282] In this disclosure, the provided multi-hypothesis intra-TMP can be used as an alternative to the current intra-TMP mode, or the encoder can adaptively select the intra-TMP mode or the multi-hypothesis intra-TMP mode.

[0283] In one and another embodiment, the provided multi-hypothesis intra-TMP is used as an alternative to the current intra-TMP mode, i.e., multiple hypotheses are always used for prediction.

[0284] In yet another embodiment, one of the multi-hypothesis intra-TMP methods described above is used in conjunction with the current intra-TMP mode in the ECM. A signal transmission flag is used in the bitstream to indicate whether the multi-hypothesis intra-TMP mode is applied to the CU.

[0285] In yet another embodiment, more than one of the multi-hypothesis intra-TMP methods described above is used in conjunction with the current intra-TMP mode in the ECM. First, a signal transmission flag is used in the bitstream to indicate whether a multi-hypothesis intra-TMP mode is applied. Then, a signal transmission index is used to indicate which of the multi-hypothesis intra-TMP methods is applied to the CU.

[0286] Multi-hypothesis intra-frame TMP using a linear filter model

[0287] Current intra-frame TMP using a linear filter model can be divided into two steps. In the first step, a reference block is obtained through template matching. In the second step, the reference block is filtered using a linear filter to generate the final prediction block, where the filter coefficients are derived using a template. The current method uses only one reference block, i.e., only one assumption, which is inefficient.

[0288] To improve the prediction quality of intra-frame TMP using a linear filter model, multiple hypotheses are introduced in the first step. Instead of obtaining only one reference block, multiple reference blocks are obtained and used to generate a fused block. It should be noted that this disclosure does not limit the method of generating the fused block. The fused block is then used as input for the second step.

[0289] The derivation of linear filter coefficients also employs the same method as for the reference blocks, fusing the templates of these reference blocks and using the fused template and the template of the current block to derive the filter coefficients.

[0290] Intra-frame TMP of chroma components

[0291] In the current ECM, intra-frame TMP is only enabled for the luma component, not the chroma component. To take advantage of intra-frame TMP for the chroma component, a method to enable intra-frame TMP for the chroma component is proposed.

[0292] In one and another embodiment, for inter-frame or intra-frame stripes with dual-tree disabled, the luma and chroma components share the same segmentation tree, and template matching is performed using only the luma component's template to identify the block vector of the current block. The reference block for the chroma component is the co-located block of the luma component's reference block. In this embodiment, the luma-coded block and the chroma-coded block share the same intra-frame TMP flag to indicate the use of intra-frame TMP mode.

[0293] In another embodiment, for inter-frame or intra-frame stripes with dual-trees disabled, the luma and chroma components share the same segmentation tree. Template matching is performed using templates from the luma component and the two chroma components to identify the block vector of the current block. The total template matching cost is a weighted sum of the template matching costs of the luma and chroma components. In this embodiment, the luma-coded block and the chroma-coded block share the same intra-frame TMP flag to indicate the use of intra-frame TMP mode.

[0294] cost total =cost Y +ω×(cost Cb +cost Cr )

[0295] In another embodiment, for inter-frame or intra-frame stripes with dual-trees disabled, the luma and chroma components share the same segmentation tree, and the template matching process for the luma-coded block and the two chroma-coded blocks is performed separately. The two chroma-coded blocks share the same template matching process, i.e., the template matching cost is calculated by adding the template matching costs of the Cb and Cr components. In this embodiment, the luma-coded block and the chroma-coded block share the same intra-frame TMP flag to indicate the use of intra-frame TMP mode.

[0296] cost chroma =cost Cb +cost Cr

[0297] In yet another embodiment, for inter-frame or intra-frame stripes with dual-tree disabled, the luma and chroma components share the same segmentation tree, and the template matching process for the Y, U, and V coded blocks is performed separately. In this embodiment, the luma and chroma coded blocks share the same intra-frame TMP flag to indicate the use of intra-frame TMP mode.

[0298] In yet another embodiment, for intra-strips with dual-tree enabled, template matching searches are performed separately for the U and V components. A flag is signaled to indicate whether intra-TMP is enabled for the chroma component.

[0299] In yet another embodiment, for intra-strips with dual-tree enabled, template matching searches are performed separately for the U and V components. A flag is signaled to indicate whether intra-TMP is enabled for the U component, and another flag is signaled to indicate whether intra-TMP is enabled for the V component.

[0300] In another embodiment, for intra-stripes with dual-tree enabled, a joint template matching search is performed on the U and V components; that is, the total template matching cost is the sum of the template matching costs of the U and V components. A flag is signaled to indicate whether intra-TMP is enabled for the chroma component.

[0301] Combination of cross-component prediction and chroma component intra-frame TMP

[0302] In current ECM (Enhanced Component Mapping), CCLM (Corrected Component Mapping) and CCCM (Corrected Component Mapping) are used to predict chroma blocks using co-located reconstructed luma blocks. Intra-frame TMP (Transient Multi-Phase Mapping) utilizes non-local correlation. To simultaneously utilize both cross-component and non-local correlation, this paper proposes combining cross-component prediction with intra-frame TMP for the chroma component. The current chroma block is denoted as B, the template for the current chroma block as T, and the co-located luma block as Bo. Y And the template representation of the same brightness block is T. Y .

[0303] In an example method that combines cross-component prediction and intra-frame TMP, the proposed method can be divided into several steps as described below.

[0304] In the first step, intra-frame template matching is performed to generate a reference block and a reference template, denoted as P. intraTMP and T intraTMP Here, the reference templates include the reference chromaticity template and the reference luminance template.

[0305] In the second step, the template T of the current chroma block and the reference chroma template T obtained in the first step are used. intraTMP And the template T of the same brightness block Y To derive the parameters of CCLM or CCCM, express the CCLM or CCCM model as F(·) and solve the following equations to obtain the parameters of CCLM or CCCM.

[0306] T = T intraTMP +F(T Y )

[0307] In the third step, the parameters obtained in the second step are applied to the same-position luminance block to generate the final prediction B for the current chrominance block. pred As shown below.

[0308] B pred =P intraTMP +F(B Y )

[0309] Combination of intra-frame and intra-frame TMP using a linear filter model

[0310] In the current combination of intra-frame and intra-frame TMP, the intra-frame prediction block and the intra-frame TMP block are weighted to generate the final prediction block without further refinement. This disclosure proposes using a linear filter model to refine the weighted prediction of intra-frame and intra-frame TMP. The proposed method can be performed according to the following steps.

[0311] In the first step, weighted prediction is generated by weighting the intra-prediction block and the intra-TMP block.

[0312] In the second step, the coefficients of the linear filter are derived from the templates (including the template for the current block, the intra-prediction template, and the reference template). An intra-prediction template is generated by performing intra-prediction using the current intra-mode. The intra-prediction template is then fused with the reference template to generate a fused template. Using the fused template and the current template, the filter coefficients are derived by solving least-squares equations.

[0313] In the third step, the filter coefficients obtained in the second step are applied to the weighted prediction to generate the final prediction.

[0314] In this disclosure, several embodiments are proposed to apply the proposed combination of intra-frame and intra-frame TMP utilizing a linear filter model.

[0315] In one embodiment, the proposed method is always applied to weighted prediction of intra-frame and intra-frame TMP. No changes to syntax elements are required.

[0316] In another embodiment, the use of the proposed method is signaled via a flag. If the flag is 1, a linear filter model is applied; otherwise, it is not. On the encoder side, an RDO (Remote Decision Object) is performed to determine whether to enable the linear filter model for that block.

[0317] In one or another embodiment, the proposed combination of intra-frame and intra-frame TMP utilizing a linear filter model can be applied simultaneously to the luma and chroma components.

[0318] Combination of IBC and intra-frame TMP

[0319] This disclosure provides a method for combining IBC and intra TMP to further improve the encoding / decoding performance of intra codecs. In one embodiment, the intra TMP is first performed to generate a first prediction value for the current codec block. Then, a second prediction value for the current codec block is identified using a block vector (BV). The first and second prediction values ​​are weighted to generate a final prediction block for the codec block.

[0320] Sub-block Intra-frame TMP

[0321] This disclosure provides a method for improving the encoding and decoding efficiency of intra-frame TMP by utilizing sub-block-based intra-frame TMP.

[0322] In one embodiment, the current prediction block is divided into several regions. For the k-th region R... k Identify the corresponding template T k It is a subset of the intra-frame TMP template. Template T kUsed for template matching to find R k The optimal prediction region is shown. Several examples are provided to illustrate the proposed method in detail.

[0323] In one example, the current prediction block is divided into several sub-blocks, and intra-frame template matching is performed for each sub-block, as shown in Figure 14. More specifically, the template is also divided into several segments. For the top / left template, it is divided into segments with a width / height equal to the size of the sub-block. For each sub-block, template matching is performed using the top and left template segments closest to the sub-block.

[0324] For example, the current prediction block is divided into four parts: the upper left region, the upper right region, the lower left region, and the lower right region, such as... Figure 15 As shown. In template matching, different templates are used for these four regions. For the top-left and bottom-right regions, both the left and top templates are used for template matching. For the bottom-left region, only the left template is used for template matching. And for the top-right region, only the top template is used for template matching.

[0325] In one embodiment, if the prediction block is encoded and decoded using intra-frame TMP mode, then sub-block intra-frame TMP is always applied.

[0326] In yet another embodiment, if intra-frame TMP is used for a prediction block, an additional flag is signaled to indicate whether sub-block intra-frame TMP is utilized.

[0327] Two-stage combination of intra-frame and intra-frame TMP

[0328] In the first example, a subset of the predicted values ​​for the current prediction block is first determined using intra-frame TMP; this process is referred to as the first prediction process, and this subset is referred to as the first subset. Next, additional predicted values ​​for the current prediction block are generated using regular intra-frame prediction modes (including planar and angular modes defined in HEVC or VVC); this process is referred to as the second prediction process, and these additional predicted values ​​are referred to as the second subset.

[0329] In one example, the first subset of predicted values ​​generated by the first prediction process could be the lower right predicted values ​​as shown in Figure 16. The second subset contains the other predicted samples as shown in Figure 16.

[0330] In one embodiment, the final predicted samples in the second subset can be derived using a planar pattern during the second prediction process.

[0331] p[x][y]=(w0[x][y]*p[x][-1]+w1[x][y]*p[x][height-1]+w2[x][y]*p[-1][y]+w3[x][y]*p[width-1][y]) / N

[0332] In some examples, neighboring samples are represented as p[x][y], where x = -1, y = -1…height and x = 0…width, y = -1. The width and height are the width and height of the current prediction block. w0, w1, w2, and w3 are predefined weighting factors. N is the normalization factor, which is usually the sum of all weighting factors. N = w0[x][y] + w1[x][y] + w2[x][y] + w3[x][y]. In a specific example, the weighting factors are predefined as follows:

[0333] w0[x][y] = (height - y), w1[x][y] = y, w2[x][y] = (width - x), w3[x][y] = x

[0334] In another embodiment, the final predicted samples in the second subset can be obtained using the intra-frame mode of the angle during the second prediction process. First, the predicted samples in the first subset are obtained using the intra-frame TMP. Second, the predicted samples in the second subset are obtained by interpolating the neighboring reconstructed samples and the predicted samples in the first subset.

[0335] p[x][y]=w1*r1+w2*r2

[0336] Wherein, w0 and w1 are calculated as follows:

[0337] and

[0338] In the second example, the planar mode is always used as the prediction mode for the second prediction process. If the current coding block is encoded and decoded in intra-frame mode, the signal transmission syntax element indicates the use of the provided method.

[0339] In the third example, both planar mode and angular mode can be selected as the prediction mode for the second prediction process. If the current coding block is encoded and decoded using intra-frame mode, a signal transmission syntax element is used to indicate the use of the proposed method. Furthermore, another signal transmission syntax element is used to indicate the intra-frame prediction mode for the second prediction process.

[0340] In some examples, the above methods can be applied independently or in combination.

[0341] Figure 5A computing environment (or computing device) 1610 coupled to a user interface 1650 is illustrated. In some embodiments, the computing device 1610 may perform any of the various methods or processes (e.g., decoding / encoding methods or processes) described above according to various examples of this disclosure. The computing environment 1610 may be part of a data processing server. The computing environment 1610 includes a processor 1620, a memory 1630, and an input / output (I / O) interface 1640.

[0342] Processor 1620 typically controls the overall operation of computing environment 1610, such as operations associated with display, data acquisition, data communication, and image processing. Processor 1620 may include one or more processors for executing instructions to perform all or some of the steps in the methods described above. Furthermore, processor 1620 may include one or more modules that facilitate interaction between processor 1620 and other components. The processor may be a central processing unit (CPU), microprocessor, microcontroller, graphics processing unit (GPU), etc.

[0343] Memory 1630 is configured to store various types of data to support the operation of computing environment 1610. Memory 1630 may include predefined software 1632. Examples of such data include instructions for any application or method operating on computing environment 1610, video datasets, image data, etc. Memory 1630 can be implemented using any type of volatile or non-volatile memory device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0344] I / O interface 1640 provides an interface between processor 1620 and peripheral interface modules (such as keyboards, click wheels, buttons, etc.). Buttons may include, but are not limited to, home buttons, start scan buttons, and stop scan buttons. I / O interface 1640 can be coupled to encoders and decoders.

[0345] Figure 19 A flowchart of a video decoding method according to an example of this disclosure is shown. Specifically, Figure 19 A method for deriving reference block candidates is shown in a multi-hypothesis intra-frame TMP.

[0346] In step 1910, in response to determining to use multi-hypothesis intra-template matching prediction (TMP), processor 1620 may obtain one or more reference block candidates for the current block on the decoder side.

[0347] In step 1920, the processor 1620 may obtain a final prediction for the current block based on one or more reference block candidates.

[0348] In some examples, in step 1910, processor 1620 may perform the following operations on the decoder side: select one or more prediction block candidates based on minimizing template matching costs, wherein the template matching costs of the one or more prediction block candidates are less than the template matching costs of other prediction block candidates; and set the one or more reference block candidates as the one or more prediction block candidates.

[0349] In some examples, the template matching cost includes the sum of absolute differences (SAD), the sum of squared differences (SSD), or other values ​​that measure the similarity between the two blocks. For example, the template matching cost can be the total template matching cost, which can be obtained as follows:

[0350] cost 总 = Cost Y +ω×(cost) Cb +cost Cr )

[0351] Here, ω can be a predefined empirical value.

[0352] In some examples, processor 1620 may further obtain intermediate prediction blocks on the decoder side. In some examples, in step 1910, processor 1620 may derive the one or more reference block candidates based on the intermediate prediction blocks. For example, the intermediate prediction blocks may be generated using existing multi-hypothesis intra-TMP methods.

[0353] In one or more examples, to obtain the intermediate prediction block, processor 1620 may search for one or more intermediate reference block candidates on the decoder side, wherein the template matching cost of the one or more intermediate reference block candidates is less than the template matching cost of other block candidates; and generate the intermediate prediction block based on the one or more intermediate reference block candidates. Here, existing methods can be used to obtain the intermediate reference block. In some examples, the intermediate prediction block may be generated using a fixed multi-hypothesis intra-TMP or an adaptive multi-hypothesis intra-TMP.

[0354] In one or more examples, in order to obtain the intermediate prediction block, the processor 1620 may set the intermediate prediction block as a reference block on the decoder side, wherein the template matching cost of the reference block is less than the template matching cost of other reference blocks.

[0355] In one or more examples, to obtain the intermediate prediction block, the processor 1620 may receive, on the decoder side, a block vector (BV) corresponding to the intermediate prediction block, transmitted in the bitstream, wherein the rate-distortion cost of the intermediate prediction block is less than the rate-distortion cost of other prediction blocks. In some examples, the BV is configured to indicate the intermediate prediction block.

[0356] In one or more examples, in order to derive the one or more reference block candidates based on the intermediate prediction block, the processor 1620 may obtain one or more reference blocks on the decoder side, wherein the distance between the one or more reference blocks and the intermediate prediction block is less than the distance between other reference blocks and the intermediate prediction block; and set the one or more reference block candidates as the one or more reference blocks.

[0357] In one or more examples, in order to derive the one or more reference block candidates based on the intermediate prediction block, the processor 1620 may obtain an intermediate region on the decoder side, the intermediate region including the intermediate prediction block and a neighboring template corresponding to the intermediate prediction block; obtain one or more reference regions corresponding to the one or more reference blocks, wherein each of the one or more reference regions includes a reference block and a neighboring template corresponding to the reference block; obtain one or more reference blocks, wherein the distance between the one or more reference regions containing the one or more reference blocks and the intermediate region is less than the distance between other reference regions and the intermediate region; and set the one or more reference block candidates as the one or more reference blocks.

[0358] In one or more examples, processor 1620 may further divide the intermediate prediction block into multiple intermediate sub-blocks on the decoder side; further, for each of the multiple intermediate sub-blocks, processor 1620 may obtain one or more reference sub-blocks on the decoder side, wherein the distance between the one or more reference sub-blocks and the intermediate sub-block among the multiple intermediate sub-blocks is less than the distance between other reference sub-blocks and the intermediate sub-block among the multiple intermediate sub-blocks; and select the one or more reference sub-blocks as one or more reference sub-block candidates. In one or more examples, processor 1620 may use the one or more reference sub-block candidates to obtain the final prediction for the current sub-block.

[0359] In one or more examples, the distance includes the sum of absolute differences (SAD), the sum of squared differences (SSD), or other values ​​that measure the similarity between two blocks. In one or more examples, in step 1920, the processor 1620 may obtain a final prediction for the current block on the decoder side based on the one or more reference block candidates and using a fixed multi-hypothesis intra-frame TMP or an adaptive multi-hypothesis intra-frame TMP.

[0360] In some examples, the intermediate prediction block comprises one or more pixels, and the multi-hypothesis intra-TMP comprises a pixel-level multi-hypothesis intra-TMP. In one or more examples, the processor 1620 may further determine one or more weighting factors on the decoder side, and in step 1920, the processor 1620 may obtain a final prediction for the current block based on the one or more reference block candidates, the multi-hypothesis intra-TMP, and the one or more weighting factors.

[0361] In one or more examples, the one or more weighting factors correspond to one or more reference pixels in one or more reference blocks, and in order to determine the one or more weighting factors, the processor 1620 may obtain one or more distances on the decoder side, wherein each of the one or more distances is the distance between the current pixel and a co-occurring pixel in one of the one or more reference blocks; and calculate the one or more weighting factors based on the one or more distances.

[0362] In one or more examples, the one or more weighting factors correspond to one or more reference pixels in one or more reference sub-blocks, and in order to determine the one or more weighting factors, the processor 1620 may obtain the current sub-block on the decoder side, wherein the current sub-block is centered on the current pixel; obtain the one or more reference sub-blocks, wherein the distance between the one or more reference sub-blocks and the current sub-block is less than the distance between other reference sub-blocks and the current sub-block; and calculate the one or more weighting factors based on these distances between the one or more reference sub-blocks and the current sub-block.

[0363] In one or more examples, the one or more weighting factors correspond to one or more reference pixels in one or more reference sub-blocks, and in order to determine the one or more weighting factors, the processor 1620 may obtain the current sub-block from the decoder, wherein the current sub-block is centered on the current pixel; obtain the one or more reference sub-blocks, wherein the distance between the center pixel of the one or more reference sub-blocks and the current pixel is less than the distance between the center pixel of other reference sub-blocks and the current pixel; and calculate the one or more weighting factors based on these distances between the center pixel of the one or more reference sub-blocks and the current pixel.

[0364] In some examples, in order to compute the one or more weighting factors, the processor 1620 may further compute the one or more weighting factors on the decoder side based on the one or more distances and the degree of weighting, wherein the degree of weighting is one of a fixed value or an adaptively determined value.

[0365] In some examples, processor 1620 may further obtain a current sub-block centered on the current pixel on the decoder side; and obtain one or more reference sub-blocks, wherein the distance between the one or more reference sub-blocks and the current sub-block is less than the distance between other reference sub-blocks and the current sub-block. In one or more examples, in step 1920, processor 1620 may generate a prediction sub-block on the decoder side using singular value decomposition (SVD) based on the current sub-block and the one or more reference sub-blocks; and set the pixel at the center of the prediction sub-block as the final prediction of the current pixel corresponding to the current sub-block.

[0366] Figure 20 It shows the same as Figure 19 The flowchart shows the video encoding method corresponding to the video decoding method shown.

[0367] In step 2010, in response to determining to use multi-hypothesis intra-template matching prediction (TMP), processor 1620 may obtain one or more reference block candidates for the current block on the encoder side.

[0368] In step 2020, the processor 1620 may obtain a final prediction for the current block based on one or more reference block candidates.

[0369] In step 2030, the processor 1620 may generate a bitstream based on the final prediction.

[0370] In some examples, in step 2010, processor 1620 may select one or more prediction block candidates on the encoder side based on minimizing template matching cost, wherein the template matching cost of the one or more prediction block candidates is less than the template matching cost of other prediction block candidates; and set the one or more reference block candidates as the one or more prediction block candidates.

[0371] In some examples, the template matching cost includes the sum of absolute differences (SAD), the sum of squared differences (SSD), or other values ​​that measure the similarity between the two blocks. For example, the template matching cost can be the total template matching cost, which can be obtained as follows:

[0372] cost 总 = Cost Y +ω×(cost) Cb +cost Cr )

[0373] Here, ω can be a predefined empirical value.

[0374] In some examples, processor 1620 may further obtain intermediate prediction blocks on the encoder side. In some examples, in step 2010, processor 1620 may derive one or more reference block candidates based on the intermediate prediction blocks.

[0375] In one or more examples, to obtain the intermediate prediction block, processor 1620 may search for one or more intermediate reference block candidates on the encoder side, wherein the template matching cost of the one or more intermediate reference block candidates is less than the template matching cost of other block candidates; and generate the intermediate prediction block based on the one or more intermediate reference block candidates. In some examples, the intermediate prediction block may be generated using a fixed multi-hypothesis intra-TMP or an adaptive multi-hypothesis intra-TMP.

[0376] In one or more examples, in order to obtain the intermediate prediction block, the processor 1620 may set the intermediate prediction block as a reference block on the encoder side, wherein the template matching cost of the reference block is less than the template matching cost of other reference blocks.

[0377] In one or more examples, to obtain the intermediate prediction block, the processor 1620 may signal a block vector (BV) corresponding to the intermediate prediction block in the bitstream on the encoder side, wherein the rate-distortion cost of the intermediate prediction block is less than the rate-distortion cost of other prediction blocks. In some examples, the BV is configured to indicate the intermediate prediction block.

[0378] In one or more examples, in order to derive the one or more reference block candidates based on the intermediate prediction block, the processor 1620 may obtain one or more reference blocks on the encoder side, wherein the distance between the one or more reference blocks and the intermediate prediction block is less than the distance between other reference blocks and the intermediate prediction block; and set the one or more reference block candidates as the one or more reference blocks.

[0379] In one or more examples, in order to derive the one or more reference block candidates based on the intermediate prediction block, the processor 1620 may obtain an intermediate region on the encoder side, the intermediate region including the intermediate prediction block and a neighboring template corresponding to the intermediate prediction block; obtain one or more reference regions corresponding to the one or more reference blocks, wherein each of the one or more reference regions includes a reference block and a neighboring template corresponding to the reference block; obtain one or more reference blocks, wherein the distance between the one or more reference regions containing the one or more reference blocks and the intermediate region is less than the distance between other reference regions and the intermediate region; and set the one or more reference block candidates as the one or more reference blocks.

[0380] In one or more examples, processor 1620 may further divide the intermediate prediction block into multiple intermediate sub-blocks on the encoder side; further, for each of the multiple intermediate sub-blocks, processor 1620 may obtain one or more reference sub-blocks on the encoder side, wherein the distance between the one or more reference sub-blocks and the intermediate sub-blocks in the multiple intermediate sub-blocks is less than the distance between other reference sub-blocks and the intermediate sub-blocks in the multiple intermediate sub-blocks; and select the one or more reference sub-blocks as one or more reference sub-block candidates. In one or more examples, processor 1620 may use the one or more reference sub-block candidates to obtain the final prediction for the current sub-block.

[0381] In one or more examples, the distance includes the sum of absolute differences (SAD), the sum of squared differences (SSD), or the sum of other values ​​that measure the similarity between two blocks. In one or more examples, in step 2020, the processor 1620 may obtain a final prediction for the current block on the encoder side based on the one or more reference block candidates and using a fixed multi-hypothesis intra-frame TMP or an adaptive multi-hypothesis intra-frame TMP.

[0382] In some examples, the intermediate prediction block comprises one or more pixels, and the multi-hypothesis intra-frame TMP comprises a pixel-level multi-hypothesis intra-frame TMP. In one or more examples, the processor 1620 may further determine one or more weighting factors on the encoder side, and in step 2020, the processor 1620 may obtain a final prediction for the current block based on the one or more reference block candidates, the multi-hypothesis intra-frame TMP, and the one or more weighting factors.

[0383] In one or more examples, the one or more weighting factors correspond to one or more reference pixels in one or more reference blocks, and in order to determine the one or more weighting factors, the processor 1620 may obtain one or more distances on the encoder side, wherein each of the one or more distances is the distance between the current pixel and a co-located pixel in one of the one or more reference blocks; and calculate the one or more weighting factors based on the one or more distances.

[0384] In one or more examples, the one or more weighting factors correspond to one or more reference pixels in one or more reference sub-blocks, and in order to determine the one or more weighting factors, the processor 1620 may obtain the current sub-block on the encoder side, wherein the current sub-block is centered on the current pixel; obtain the one or more reference sub-blocks, wherein the distance between the one or more reference sub-blocks and the current sub-block is less than the distance between other reference sub-blocks and the current sub-block; and calculate the one or more weighting factors based on these distances between the one or more reference sub-blocks and the current sub-block.

[0385] In one or more examples, the one or more weighting factors correspond to one or more reference pixels in one or more reference sub-blocks, and in order to determine the one or more weighting factors, the processor 1620 may obtain, on the encoder side, a current sub-block centered on the current pixel; obtain the one or more reference sub-blocks, wherein the distance between the center pixel of the one or more reference sub-blocks and the current pixel is less than the distance between the center pixel of other reference sub-blocks and the current pixel; and calculate the one or more weighting factors based on these distances between the center pixel of the one or more reference sub-blocks and the current pixel.

[0386] In some examples, in order to calculate the one or more weighting factors, the processor 1620 may further calculate the one or more weighting factors on the encoder side based on the one or more distances and the weighting degree, wherein the weighting degree is one of a fixed value or an adaptively determined value.

[0387] In some examples, processor 1620 may further obtain a current sub-block on the encoder side, wherein the current sub-block is centered on the current pixel; and obtain one or more reference sub-blocks, wherein the distance between the one or more reference sub-blocks and the current sub-block is less than the distance between other reference sub-blocks and the current sub-block. In one or more examples, in step 2020, processor 1620 may generate a prediction sub-block on the encoder side based on the current sub-block and the one or more reference sub-blocks using singular value decomposition (SVD); and set the pixel at the center of the prediction sub-block as the final prediction of the current pixel corresponding to the current sub-block.

[0388] In some examples, an apparatus for video encoding and decoding is provided. The apparatus includes a processor 1620 and a memory 1640, the memory being configured to store instructions executable by the processor; wherein the processor, when executing the instructions, is configured to perform actions such as... Figures 19-20 Any of the methods shown.

[0389] In an embodiment, a non-transitory computer-readable storage medium is also provided, including, for example, a plurality of programs in a memory 1630 and / or a bitstream generated by the above-described encoding method or a bitstream to be decoded by the above-described decoding method. The plurality of programs can be executed by a processor 1620 in a computing environment 1610 to perform the above-described methods. In one example, the plurality of programs can be executed by a processor 1620 in a computing environment 1610 to (e.g., from...) Figure 2The video encoder 20 in the computing environment 1610 receives a bitstream or data stream including encoded video information (e.g., video blocks representing encoded video frames, and / or one or more associated syntax elements, etc.), and can also be executed by the processor 1620 in the computing environment 1610 to perform the above-described decoding method based on the received bitstream or data stream. In another example, multiple programs can be executed by the processor 1620 in the computing environment 1610 to perform the above-described encoding method to encode video information (e.g., video blocks representing video frames, and / or one or more associated syntax elements, etc.) into a bitstream or data stream, and can also be executed by the processor 1620 in the computing environment 1610 to (e.g., to...) Figure 3 The video decoder 30 in the medium transmits the bitstream or data stream. Alternatively, a non-transitory computer-readable storage medium may store data generated by an encoder (e.g., Figure 2 The video encoder 20 in the video is generated using, for example, the encoding method described above, for use by the decoder (e.g., Figure 3 The video decoder 30 in the video decoder uses a bitstream or data stream that includes encoded video information (e.g., video blocks representing encoded video frames, and / or one or more associated syntax elements, etc.) when decoding video data. Non-transitory computer-readable storage media may be, for example, ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, etc.

[0390] In one embodiment, a bitstream generated by the above encoding method or a bitstream to be decoded by the above decoding method is provided. In another embodiment, a bitstream comprising encoded video information generated by the above encoding method or encoded video information to be decoded by the above decoding method is provided.

[0391] In one embodiment, a computing device is also provided, comprising: one or more processors (e.g., processor 1620); and a non-transitory computer-readable storage medium or memory 1630 therein storing a plurality of programs executable by the one or more processors, wherein the one or more processors are configured to perform the methods described above when executing the plurality of programs.

[0392] In one embodiment, a computer program product having instructions for storing or transmitting a bitstream, the bitstream including encoded video information generated by the encoding method described above or encoded video information to be decoded by the decoding method described above, is also provided. In another embodiment, a computer program product including, for example, multiple programs in a memory 1630, which can be executed by a processor 1620 in a computing environment 1610 to perform the methods described above, is also provided. For example, the computer program product may include a non-transitory computer-readable storage medium.

[0393] In an embodiment, the computing environment 1610 may be implemented by one or more ASICs, DSPs, digital signal processing devices (DSPDs), programmable logic devices (PLDs), FPGAs, GPUs, controllers, microcontrollers, microprocessors, or other electronic components for performing the methods described above.

[0394] In one embodiment, a method for storing a bitstream is also provided, comprising: storing the bitstream on a digital storage medium, wherein the bitstream includes encoded video information generated by the above-described encoding method or encoded video information to be decoded by the above-described decoding method.

[0395] In one embodiment, a method for transmitting a bitstream generated by the encoder described above is also provided. In another embodiment, a method for receiving a bitstream to be decoded by the decoder described above is also provided.

[0396] The description in this disclosure is presented for illustrative purposes and is not intended to be exhaustive or limited to this disclosure. Many modifications, variations, and alternative embodiments will be apparent to those skilled in the art from the teachings presented in the foregoing description and the associated drawings.

[0397] Unless otherwise specified, the order of steps in the method according to this disclosure is intended to be illustrative only, and the steps of the method according to this disclosure are not limited to the specific order described above, but may be changed according to actual circumstances. Furthermore, at least one step in the method according to this disclosure may be adjusted, combined, or omitted as needed.

[0398] The examples were chosen and described to explain the principles of this disclosure and to enable others skilled in the art to understand the various embodiments of this disclosure, and preferably to utilize the basic principles and various embodiments with various modifications suitable for the intended particular purpose. Therefore, it should be understood that the scope of this disclosure is not limited to the specific examples of the disclosed embodiments, and that modifications and other embodiments are intended to be included within the scope of this disclosure.

Claims

1. A video decoding method, comprising: In response to determining the use of multi-hypothesis intra-template matching prediction (TMP), the decoder obtains one or more reference block candidates for the current block; as well as The decoder obtains a final prediction for the current block based on the one or more reference block candidates.

2. The method as described in claim 1, wherein, Obtaining the one or more reference block candidates by the decoder includes: The decoder selects one or more prediction block candidates based on minimizing the template matching cost, wherein the template matching cost of the one or more prediction block candidates is less than the template matching cost of other prediction block candidates; and The decoder sets the one or more reference block candidates as the one or more prediction block candidates.

3. The method as described in claim 2, wherein, The template matching cost includes the sum of absolute differences (SAD), the sum of squared differences (SSD), or other values ​​that measure the similarity between two blocks.

4. The method of claim 1, further comprising: The intermediate prediction block is obtained by the decoder; and The one or more reference block candidates obtained by the decoder include: The decoder derives one or more reference block candidates based on the intermediate prediction blocks.

5. The method of claim 4, wherein, The intermediate prediction blocks obtained by the decoder include: The decoder searches for one or more intermediate reference block candidates, wherein the template matching cost of the one or more intermediate reference block candidates is less than the template matching cost of other block candidates; and The decoder generates the intermediate prediction block based on the one or more intermediate reference block candidates.

6. The method of claim 5, wherein, The intermediate prediction blocks are generated using either a fixed multi-hypothesis intra-frame TMP or an adaptive multi-hypothesis intra-frame TMP.

7. The method of claim 4, wherein, The intermediate prediction blocks obtained by the decoder include: The decoder sets the intermediate prediction block as a reference block, wherein the template matching cost of the reference block is less than the template matching cost of other reference blocks.

8. The method of claim 4, wherein, The intermediate prediction blocks obtained by the decoder include: The decoder receives the block vector (BV) corresponding to the intermediate prediction block, which is transmitted as a signal in the bitstream; Wherein, the rate-distortion cost of the intermediate prediction block is less than the rate-distortion cost of other prediction blocks; and The BV is configured to indicate the intermediate prediction block.

9. The method of claim 4, wherein, The decoder derives one or more reference block candidates based on the intermediate prediction blocks, including: The decoder obtains one or more reference blocks, wherein the distance between the one or more reference blocks and the intermediate prediction block is less than the distance between other reference blocks and the intermediate prediction block; and The decoder sets the one or more reference block candidates as the one or more reference blocks.

10. The method of claim 4, wherein, The decoder derives one or more reference block candidates based on the intermediate prediction blocks, including: The decoder obtains an intermediate region, which includes the intermediate prediction block and the neighboring templates corresponding to the intermediate prediction block; The decoder obtains one or more reference regions corresponding to one or more reference blocks, wherein each of the one or more reference regions includes a reference block and a neighboring template corresponding to the reference block; The decoder obtains one or more reference blocks, wherein the distance between the one or more reference regions containing the one or more reference blocks and the intermediate region is less than the distance between other reference regions and the intermediate region; and The decoder sets the one or more reference block candidates as the one or more reference blocks.

11. The method of claim 4, further comprising: The decoder divides the intermediate prediction block into multiple intermediate sub-blocks; For each of the plurality of intermediate sub-blocks, the decoder obtains one or more reference sub-blocks, wherein the distance between the one or more reference sub-blocks and the intermediate sub-block among the plurality of intermediate sub-blocks is less than the distance between other reference sub-blocks and the intermediate sub-block among the plurality of intermediate sub-blocks; and The decoder selects one or more reference sub-blocks as one or more reference sub-block candidates; and The decoder uses one or more reference sub-block candidates to obtain the final prediction for the current sub-block.

12. The method of claim 9, wherein, Distance can be the sum of absolute differences (SAD), the sum of squared differences (SSD), or other values ​​that measure the similarity between two blocks.

13. The method of claim 9, wherein, The final prediction for the current block obtained by the decoder includes: The decoder obtains the final prediction for the current block based on the one or more reference block candidates and using either a fixed multi-hypothesis intra-frame TMP or an adaptive multi-hypothesis intra-frame TMP.

14. The method of claim 4, wherein, The intermediate prediction block includes one or more pixels, and the multi-hypothesis intra-frame TMP includes pixel-level multi-hypothesis intra-frame TMP.

15. The method of claim 14, further comprising: One or more weighting factors are determined by the decoder; and The final prediction of the current block obtained by the decoder includes: The decoder obtains the final prediction for the current block based on the one or more reference block candidates, the multiple hypothesis intra-frame TMP, and the one or more weighting factors.

16. The method of claim 15, wherein, The one or more weighting factors correspond to one or more reference pixels in one or more reference blocks; And wherein, the decoder determines the one or more weighting factors including: The decoder obtains one or more distances, wherein each of the one or more distances is the distance between the current pixel and a co-located pixel in one of the one or more reference blocks; and The decoder calculates the one or more weighting factors based on the one or more distances.

17. The method of claim 15, wherein, The one or more weighting factors correspond to one or more reference pixels in one or more reference sub-blocks; And wherein, the decoder determines the one or more weighting factors including: The current sub-block is obtained by the decoder, wherein the current sub-block is centered on the current pixel; The decoder obtains the one or more reference sub-blocks, wherein the distance between the one or more reference sub-blocks and the current sub-block is less than the distance between other reference sub-blocks and the current sub-block; and The decoder calculates the one or more weighting factors based on the distance between the one or more reference sub-blocks and the current sub-block.

18. The method of claim 15, wherein, The one or more weighting factors correspond to one or more reference pixels in one or more reference sub-blocks; And wherein, the decoder determines the one or more weighting factors including: The current sub-block is obtained by the decoder, wherein the current sub-block is centered on the current pixel; The decoder obtains the one or more reference sub-blocks, wherein the distance between the center pixel of the one or more reference sub-blocks and the current pixel is less than the distance between the center pixel of other reference sub-blocks and the current pixel; and The decoder calculates the one or more weighting factors based on the distance between the center pixel of the one or more reference sub-blocks and the current pixel.

19. The method of claim 16, wherein, The calculation of the one or more weighting factors by the decoder further includes: The decoder calculates the one or more weighting factors based on the one or more distances and weighting degrees; The weighting degree is either a fixed value or an adaptive decision value.

20. The method of claim 14, further comprising: The decoder obtains the current sub-block, wherein the current sub-block is centered on the current pixel; and The decoder obtains one or more reference sub-blocks, wherein the distance between the one or more reference sub-blocks and the current sub-block is less than the distance between other reference sub-blocks and the current sub-block; Furthermore, the final prediction of the current block obtained by the decoder includes: The decoder generates a predicted sub-block using singular value decomposition (SVD) based on the current sub-block and the one or more reference sub-blocks; and The decoder sets the pixel at the center of the predicted sub-block as the final prediction for the current pixel corresponding to the current sub-block.

21. A video coding method, comprising: In response to determining the use of multi-hypothesis intra-template matching prediction (TMP), the encoder obtains one or more reference block candidates for the current block; The encoder obtains the final prediction for the current block based on the one or more reference block candidates; as well as The encoder generates a bitstream based on the final prediction.

22. The method of claim 21, wherein, Obtaining the one or more reference block candidates by the encoder includes: The encoder selects one or more prediction block candidates based on minimizing the template matching cost, wherein the template matching cost of the one or more prediction block candidates is less than the template matching cost of other prediction block candidates; and The encoder sets the one or more reference block candidates as the one or more prediction block candidates.

23. The method of claim 22, wherein, The template cost includes the sum of absolute differences (SAD), the sum of squared differences (SSD), or other values ​​that measure the similarity between two blocks.

24. The method of claim 21, further comprising: The encoder obtains intermediate prediction blocks; and The encoder obtains one or more reference block candidates, including: The encoder derives one or more reference block candidates based on the intermediate prediction blocks.

25. The method of claim 24, wherein, The intermediate prediction block obtained by the encoder includes: The encoder searches for one or more intermediate reference block candidates, wherein the template matching cost of the one or more intermediate reference block candidates is less than the template matching cost of other block candidates; and The encoder generates the intermediate prediction block based on the one or more intermediate reference block candidates.

26. The method of claim 25, wherein, The intermediate prediction blocks are generated using either a fixed multi-hypothesis intra-frame TMP or an adaptive multi-hypothesis intra-frame TMP.

27. The method of claim 24, wherein, The intermediate prediction block obtained by the encoder includes: The encoder sets the intermediate prediction block as a reference block, wherein the template matching cost of the reference block is less than the template matching cost of other reference blocks.

28. The method of claim 24, wherein, The intermediate prediction block obtained by the encoder includes: The encoder transmits the block vector (BV) corresponding to the intermediate prediction block in the bit stream using signals; Wherein, the rate-distortion cost of the intermediate prediction block is less than the rate-distortion cost of other prediction blocks; and The BV is configured to indicate the intermediate prediction block.

29. The method of claim 24, wherein, The encoder derives one or more reference block candidates based on the intermediate prediction blocks, including: The encoder obtains one or more reference blocks, wherein the distance between the one or more reference blocks and the intermediate prediction block is less than the distance between other reference blocks and the intermediate prediction block; and The encoder sets the one or more reference block candidates as the one or more reference blocks.

30. The method of claim 24, wherein, The encoder derives one or more reference block candidates based on the intermediate prediction blocks, including: The encoder obtains an intermediate region, which includes the intermediate prediction block and the neighboring templates corresponding to the intermediate prediction block; The encoder obtains one or more reference regions corresponding to one or more reference blocks, wherein each of the one or more reference regions includes a reference block and a neighboring template corresponding to the reference block; The encoder obtains one or more reference blocks, wherein the distance between the one or more reference regions containing the one or more reference blocks and the intermediate region is less than the distance between other reference regions and the intermediate region; and The encoder sets the one or more reference block candidates as the one or more reference blocks.

31. The method of claim 24, further comprising: The encoder divides the intermediate prediction block into multiple intermediate sub-blocks; For each of the plurality of intermediate sub-blocks, the encoder obtains one or more reference sub-blocks, wherein the distance between the one or more reference sub-blocks and the intermediate sub-blocks among the plurality of intermediate sub-blocks is less than the distance between other reference sub-blocks and the intermediate sub-blocks among the plurality of intermediate sub-blocks; The encoder selects one or more reference sub-blocks as one or more reference sub-block candidates; and The decoder uses one or more reference sub-block candidates to obtain the final prediction for the current sub-block.

32. The method of claim 29, wherein, Distance can be the sum of absolute differences (SAD), the sum of squared differences (SSD), or other values ​​that measure the similarity between two blocks.

33. The method of claim 29, wherein, The encoder obtains the final prediction for the current block by: The encoder obtains the final prediction for the current block based on the one or more reference block candidates and using either a fixed multi-hypothesis intra-frame TMP or an adaptive multi-hypothesis intra-frame TMP.

34. The method of claim 24, wherein, The intermediate prediction block includes one or more pixels, and the multi-hypothesis intra-frame TMP includes pixel-level multi-hypothesis intra-frame TMP.

35. The method of claim 34, further comprising: One or more weighting factors are determined by the encoder; and The final prediction of the current block obtained by the encoder includes: The encoder obtains the final prediction for the current block based on the one or more reference block candidates, the multi-hypothesis intra-frame TMP, and the one or more weighting factors.

36. The method of claim 35, wherein, The one or more weighting factors correspond to one or more reference pixels in one or more reference blocks; And wherein, the encoder determines the one or more weighting factors including: One or more distances are obtained by the encoder, wherein each of the one or more distances is the distance between the current pixel and a co-position pixel in one of the one or more reference blocks; and The encoder calculates the one or more weighting factors based on the one or more distances.

37. The method of claim 35, wherein, The one or more weighting factors correspond to one or more reference pixels in one or more reference sub-blocks; And wherein, the encoder determines the one or more weighting factors including: The encoder obtains the current sub-block, wherein the current sub-block is centered on the current pixel; The encoder obtains the one or more reference sub-blocks, wherein the distance between the one or more reference sub-blocks and the current sub-block is less than the distance between other reference sub-blocks and the current sub-block; and The encoder calculates the one or more weighting factors based on the distance between the one or more reference sub-blocks and the current sub-block.

38. The method of claim 35, wherein, The one or more weighting factors correspond to one or more reference pixels in one or more reference sub-blocks; And wherein, the encoder determines the one or more weighting factors including: The encoder obtains the current sub-block, wherein the current sub-block is centered on the current pixel; The encoder obtains the one or more reference sub-blocks, wherein the distance between the center pixel of the one or more reference sub-blocks and the current pixel is less than the distance between the center pixel of other reference sub-blocks and the current pixel; and The encoder calculates the one or more weighting factors based on the distance between the center pixel of the one or more reference sub-blocks and the current pixel.

39. The method of claim 36, wherein, The calculation of the one or more weighting factors by the encoder further includes: The encoder calculates the one or more weighting factors based on the one or more distances and weighting degrees; The weighting degree is either a fixed value or an adaptive decision value.

40. The method of claim 34, further comprising: The encoder obtains the current sub-block, wherein the current sub-block is centered on the current pixel; and One or more reference sub-blocks are obtained by the encoder, wherein the distance between the one or more reference sub-blocks and the current sub-block is less than the distance between other reference sub-blocks and the current sub-block; Furthermore, the encoder obtaining the final prediction for the current block includes: The encoder generates a predicted sub-block using singular value decomposition (SVD) based on the current sub-block and the one or more reference sub-blocks; and The encoder sets the pixel at the center of the predicted sub-block as the final prediction for the current pixel corresponding to the current sub-block.

41. A video decoding apparatus, comprising: One or more processors; as well as A memory coupled to the one or more processors, the memory being configured to store instructions executable by the one or more processors. The one or more processors are configured to perform the method as described in any one of claims 1-20 when executing the instructions.

42. A video encoding apparatus, comprising: One or more processors; as well as A memory coupled to the one or more processors, the memory being configured to store instructions executable by the one or more processors. The one or more processors are configured to perform the method as described in any one of claims 21-40 when executing the instructions.

43. A non-transitory computer-readable storage medium for storing computer-executable instructions, which, when executed by one or more computer processors, cause the one or more computer processors to perform the method as described in any one of claims 1-20.

44. A non-transitory computer-readable storage medium for storing computer-executable instructions, which, when executed by one or more computer processors, cause the one or more computer processors to perform the method as described in any one of claims 21-40.

45. A non-transitory computer-readable storage medium for storing a bit stream to be decoded by the method of any one of claims 1-20.

46. ​​A non-transitory computer-readable storage medium for storing a bit stream generated by the method of any one of claims 21-40.

Images