Intra template matching prediction signaling

Abstract

An apparatus for video decoding is provided. The apparatus includes processing circuitry. The processing circuitry is to receive a video bitstream comprising encoded information associated with a current block, the current block comprising a luma component and a chroma component. The encoded information indicates whether to apply Intra Template Matching Prediction (IntraTMP) to the current block. When the encoded information indicates that the intraTMP is applied to the current block, the processing circuitry is configured to determine (i) a BV of a luma component of the current block based on the intraTMP, and determine (ii) a BV of a chroma component of the current block based on the determined BV of the luma component of the current block. The processing circuitry is to reconstruct the current block based on the determined BV of the luma component and the determined BV of the chroma component.

Description

Incorporation by Reference

[0001] This application claims the benefit of priority to U.S. Provisional Application No. 63 / 460,597, filed on April 19, 2023, entitled “On Signaling of Intra Template Matching Prediction,” which is incorporated herein by reference in its entirety. Technical Field

[0002] Aspects described herein relate generally to video coding. Background Art

[0003] The background description provided herein is intended to present the context of the present application in general. The work of the inventors named herein (to the extent described in this background section), as well as various aspects described in the specification that may not be considered prior art at the time of filing, are neither explicitly nor implicitly admitted to be prior art against the present application.

[0004] Image / video compression can help transmit image / video data between different devices, storage devices, and networks with minimal quality degradation. In some examples, video codec technology can compress video based on spatial and temporal redundancy. In an example, a video codec can use a technique called intra-frame prediction, which can compress an image based on spatial redundancy. For example, intra-frame prediction can use reference data in the current image being reconstructed for sample prediction. In another example, a video codec can use a technique called inter-frame prediction, which can compress an image based on temporal redundancy. For example, inter-frame prediction can predict samples in the current image based on a previously reconstructed image through motion compensation. Motion compensation can be indicated by a motion vector (MV). Summary of the invention

[0005] Aspects of the present application include code streams, methods, and apparatus for video encoding / decoding. In some examples, an apparatus for video encoding / decoding includes a processing circuit.

[0006] According to one aspect of the present application, a method for processing visual media data is provided. In the method, conversion between a visual media file and a code stream of visual media data is performed according to a format rule. The code stream includes encoded information associated with a current block in a current image, and the current block includes a luminance component and a chrominance component. The format rule stipulates that the encoded information indicates (i) whether intra-frame template matching prediction (intraTMP) is applied to the luminance component or the chrominance component when the luminance component and the chrominance component are in a dual-tree structure, and (ii) whether intraTMP is applied to the luminance component and the chrominance component of the current block when the luminance component and the chrominance component are in a single-tree structure. When the encoded information indicates that intraTMP is applied to the current block, (i) a block vector (BV) of the luminance component of the current block is determined based on the intraTMP, and (ii) the BV of the chrominance component of the current block is determined based on using the determined BV of the luminance component as the starting BV of the search range of the intraTMP for the chrominance component. The format rule stipulates that the current block is processed based on the determined BV of the luma component and the determined BV of the chroma components.

[0007] In the example, the format rule stipulates that at least two candidate BVs are determined within the search range. The format rule stipulates that the template matching cost between the template of the chrominance component of the current block and the template of each of the at least two candidate BVs is determined. The format rule stipulates that the candidate BV with the smallest template matching cost between the template of the chrominance component and the template of the chrominance component among the at least two candidate BVs is determined as the BV of the chrominance component.

[0008] In an example, the format rule stipulates that the determined BV of the luma component is scaled according to the chroma downsampling format to obtain the start BV of the search range of the chroma component.

[0009] In an example, the codestream further includes a first syntax element indicating whether the current block is encoded with intra block copy (IBC). The format rule stipulates that when the first syntax element indicates that the current block is encoded with IBC, the encoded information indicates whether intraTMP is applied to the current block.

[0010] According to another aspect of the present application, a method for video encoding is provided. In the method, it is determined whether to apply intraTMP to a current block in a current image. The current block includes a luminance component and a chrominance component, and intraTMP is used to determine a prediction block of the current block based on a template matching cost between a template of the current block and a template of a prediction block. When it is determined that intraTMP is applied to the current block, the BV of the luminance component of the current block is determined based on the intraTMP. The BV of the chrominance component of the current block is determined by using the determined BV of the luminance component as the starting BV within the search range of the intraTMP for the chrominance component. Based on the determined BV of the luminance component and the determined BV of the chrominance component, the current block is encoded into a video bitstream.

[0011] In an example, a syntax element for indicating whether intraTMP is applied to the current block is encoded into a video bitstream.

[0012] In an example, at least two candidate BVs are determined within the search range. A template matching cost between the template of the chrominance component of the current block and the template of each of the at least two candidate BVs is determined. The candidate BV with the smallest template matching cost between the template of the chrominance component and the template of the chrominance component among the at least two candidate BVs is determined as the BV of the chrominance component.

[0013] According to another aspect of the present application, a device for video decoding is provided. The device includes a processing circuit. The processing circuit is used to receive a video code stream. The video code stream includes encoded information associated with a current block in a current image. The current block includes a luminance component and a chrominance component, and the encoded information indicates whether intra-frame template matching prediction (intraTMP) is applied to the current block, and intraTMP is used to determine the prediction block of the current block based on the template matching cost between the template of the current block and the template of the prediction block. When the encoded information indicates that intraTMP is applied to the current block, the processing circuit is used to: (i) determine the block vector (BV) of the luminance component of the current block based on intraTMP, and (ii) determine the BV of the chrominance component of the current block based on the determined BV of the luminance component of the current block. The processing circuit is used to reconstruct the current block based on the determined BV of the luminance component and the determined BV of the chrominance component.

[0014] In an example, the encoded information comprises a first syntax element indicating whether intraTMP is applied to a luma component of the current block and a second syntax element indicating whether intraTMP is applied to a chroma component of the current block.

[0015] In an example, the processing circuit is used to set the determined BV of the luminance component as the starting BV of the search area of ​​the chrominance component. The processing circuit is used to determine at least two candidate BVs in the search area. The processing circuit is used to determine the template matching cost between the template of the chrominance component of the current block and the template of each of the at least two candidate BVs. The processing circuit is used to determine the candidate BV with the smallest template matching cost between the template of the chrominance component and the template of the chrominance component among the at least two candidate BVs as the BV of the chrominance component.

[0016] In an example, the processing circuit is configured to scale the determined BV of the luma component according to the chroma downsampling format to determine the starting BV of the chroma components.

[0017] In an example, the encoded information indicates whether intraTMP is applied to a luma component or a chroma component when the luma component and the chroma component are in a dual-tree structure.

[0018] In an example, the encoded information indicates whether intraTMP is applied to the luma component and the chroma component of the current block when the luma component and the chroma component are in a single tree structure.

[0019] In an example, the video code stream further includes first syntax information indicating whether the current block is encoded with intra block copy (IBC), and when the first syntax information indicates that the current block is encoded with IBC, the encoded information indicates whether intraTMP is applied to the current block.

[0020] In an example, the processing circuit is configured to determine merged data of the IBC included in the video code stream when the encoded information indicates that the intraTMP is not applied to the current block.

[0021] In an example, the video bitstream further includes second syntax information indicating whether to include the merged data of IBC in the video bitstream. The processing circuit is used to determine whether the second syntax information indicates to include the merged data of IBC in the video bitstream when the encoded information indicates that intraTMP is not applied to the current block. The processing circuit is used to determine the merged data of IBC from the video bitstream when the second syntax information indicates to include the merged data of IBC in the video bitstream.

[0022] In an example, the video code stream further includes third syntax information indicating whether the current block is encoded with IBC or intraTMP. When the third syntax information indicates that the current block is encoded with IBC or intraTMP, the encoded information indicates whether intraTMP is applied to the current block.

[0023] In an example, the processing circuit is to, when the encoded information indicates that intraTMP is applied to the current block, determine intraTMP information associated with the intraTMP, wherein the intraTMP information includes a width of the current block and a height of the current block.

[0024] In an example, the processing circuit is configured to, when the encoded information indicates that intraTMP is applied to the current block, determine a prediction block for the current block based on the intraTMP. The processing circuit is configured to determine that a merge candidate list for the current block includes the prediction block and at least one IBC candidate.

[0025] In an example, the processing circuit is used to determine a normalized template matching cost of the prediction block and a normalized template matching cost of each IBC candidate in at least one IBC candidate. The normalized template matching cost is equal to the template matching cost associated with the prediction block divided by the size of the template of the prediction block. The normalized template matching cost of each IBC candidate in at least one IBC candidate is equal to the template matching cost associated with the corresponding IBC candidate divided by the size of the template of the corresponding IBC candidate. The processing circuit is used to reorder the prediction block and at least one IBC candidate in the merge candidate list based on the normalized template matching cost.

[0026] Aspects of the present application also provide an apparatus for video encoding. The apparatus for video encoding includes a processing circuit, which is configured to implement any one of the methods for video encoding.

[0027] Aspects of the present application also provide a method for video decoding, which includes any one of the methods implemented by the apparatus for video decoding.

[0028] Aspects of the present application also provide a non-transitory computer-readable medium storing instructions, which, when executed by a computer, causes the computer to perform any of the methods for video decoding / encoding. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Other features, properties and various advantages of the disclosed subject matter will become more apparent from the following detailed description and accompanying drawings, in which:

[0030] Figure 1 An example of a block diagram of a communication system (100) is shown.

[0031] Figure 2 An example of a block diagram of a decoder is shown.

[0032] Figure 3 An example of a block diagram of an encoder is shown.

[0033] Figure 4Intra Template Matching Prediction (IntraTMP) illustrating some aspects of the present application.

[0034] Figure 5 An overview flow chart showing the decoding process of some aspects of the present application.

[0035] Figure 6 An overview flow chart showing the encoding process of some aspects of the present application.

[0036] Figure 7 A computer system in accordance with one aspect of the present application is shown. DETAILED DESCRIPTION

[0037] Figure 1 A block diagram of a video processing system (100) in some examples is shown. The video processing system (100) is an example of an application of the disclosed subject matter, including a video encoder and a video decoder in a streaming environment. The subject matter disclosed in this application is equally applicable to other video-enabled applications, including, for example, video conferencing, digital TV, streaming services, storing compressed video on digital media including CDs, DVDs, memory sticks, etc., and the like.

[0038] The video processing system (100) includes an acquisition subsystem (113), which may include a video source (101) such as a digital camera, which creates an uncompressed video image stream (102). In an embodiment, the video image stream (102) includes samples captured by a digital camera. Compared to the encoded video data (104) (or the encoded video code stream), the video image stream (102) is depicted as a thick line to emphasize the high data volume of the video image stream, and the video image stream (102) can be processed by an electronic device (120), which includes a video encoder (103) coupled to the video source (101). The video encoder (103) may include hardware, software, or a combination of hardware and software to implement or implement various aspects of the disclosed subject matter as described in more detail below. Compared to the video image stream (102), the encoded video data (104) (or the encoded video code stream (104)) is depicted as a thin line to emphasize the lower amount of data of the encoded video data (104) (or the encoded video code stream (104)), which can be stored on the streaming server (105) for future use. At least one streaming client subsystem, such as Figure 1A client subsystem (106) and a client subsystem (108) in a streaming server (105) may access a streaming server (105) to retrieve a copy (107) and a copy (109) of the encoded video data (104). The client subsystem (106) may include, for example, a video decoder (110) in an electronic device (130). The video decoder (110) decodes the incoming copy (107) of the encoded video data and generates an output video image stream (111) that can be presented on a display (112) (e.g., a display screen) or another presentation device (not depicted). In some streaming systems, the encoded video data (104), the video data (107), and the video data (109) (e.g., a video bitstream) may be encoded according to certain video encoding / compression standards. Examples of such standards include ITU-T H.265. In an embodiment, the video coding standard under development is informally referred to as Versatile Video Coding (VVC), and the present application may be used in the context of the VVC standard.

[0039] It should be noted that the electronic device (120) and the electronic device (130) may include other components (not shown). For example, the electronic device (120) may include a video decoder (not shown), and the electronic device (130) may also include a video encoder (not shown).

[0040] Figure 2 1 is a block diagram of a video decoder (210) according to an embodiment disclosed in the present application. The video decoder (210) may be provided in an electronic device (230). The electronic device (230) may include a receiver (231) (e.g., a receiving circuit). The video decoder (210) may be used to replace Figure 1 A video decoder (110) of an embodiment.

[0041] The receiver (231) may receive at least one encoded video sequence to be decoded by the video decoder (210); in the same or another embodiment, one encoded video sequence is received at a time, wherein the decoding of each encoded video sequence is independent of the other encoded video sequences. The encoded video sequence may be received from a channel (201), which may be a hardware / software link to a storage device storing the encoded video data. The receiver (231) may receive the encoded video data as well as other data, such as encoded audio data and / or auxiliary data streams that may be forwarded to their respective consuming entities (not shown). The receiver (231) may separate the encoded video sequence from the other data. To prevent network jitter, a buffer memory (215) may be coupled between the receiver (231) and the entropy decoder / parser (220) (hereinafter referred to as "parser (220)"). In some applications, the buffer memory (215) is part of the video decoder (210). In other cases, the buffer memory (215) may be provided external to the video decoder (210) (not shown). In other cases, a buffer memory (not shown) is provided outside the video decoder (210) to prevent network jitter, for example, and another buffer memory (215) may be configured inside the video decoder (210) to handle broadcast timing, for example. When the receiver (231) receives data from a storage / forward device with sufficient bandwidth and controllability or from an isochronous network, it may not be necessary to configure the buffer memory (215), or the buffer memory may be made smaller. Of course, in order to use on a service packet network such as the Internet, a buffer memory (215) may also be required, and the buffer memory may be relatively large and have an adaptive size, and may be at least partially implemented in an operating system or a similar element (not shown) outside the video decoder (210).

[0042] The video decoder (210) may include a parser (220) to reconstruct symbols (221) from the encoded video sequence. The types of symbols include information for managing the operation of the video decoder (210) and potential information for controlling a display device such as a display device (212) (e.g., a display screen) that is not part of the electronic device (230) but can be coupled to the electronic device (230), such as Figure 2As shown in . The control information for the display device may be a parameter set fragment (not indicated) of Supplemental Enhancement Information (SEI message) or Video Usability Information (VUI). The parser (220) may parse / entropy decode the received coded video sequence. The encoding of the coded video sequence may be performed according to a video coding technique or standard, and may follow various principles, including variable length coding, Huffman coding, arithmetic coding with or without context sensitivity, and the like. The parser (220) may extract a subgroup parameter set of at least one subgroup of the subgroups of pixels in the video decoder from the coded video sequence based on at least one parameter corresponding to the group. The subgroup may include a Group of Pictures (GOP), a picture, a tile, a slice, a macroblock, a Coding Unit (CU), a block, a Transform Unit (TU), a Prediction Unit (PU), and the like. The parser (220) may also extract information from the encoded video sequence, such as transform coefficients, quantizer parameter values, motion vectors, and so on.

[0043] The parser (220) may perform entropy decoding / parsing operations on the video sequence received from the buffer memory (215), thereby creating symbols (221).

[0044] Depending on the type of coded video picture or part of coded video picture (e.g., inter-frame and intra-frame pictures, inter-frame blocks and intra-frame blocks) and other factors, the reconstruction of the symbol (221) may involve multiple different units. Which units are involved and how they are involved can be controlled by subgroup control information parsed by the parser (220) from the coded video sequence. For the sake of brevity, such subgroup control information flow between the parser (220) and the multiple units below is not described.

[0045] In addition to the functional blocks already mentioned, the video decoder (210) can be conceptually subdivided into several functional units as described below. In a practical embodiment operating under commercial constraints, many of these units interact closely with each other and can be integrated with each other. However, for the purpose of describing the disclosed subject matter, the conceptual subdivision into the following functional units is appropriate.

[0046] The first unit is a sealer / inverse transform unit (251). The sealer / inverse transform unit (251) receives quantized transform coefficients as symbols (221) from the parser (220) and control information, including which transform method to use, block size, quantization factor, quantization scaling matrix, etc. The sealer / inverse transform unit (251) can output a block including sample values, which can be input into an aggregator (255).

[0047] In some cases, the output samples of the scaler / inverse transform unit (251) may belong to an intra-coded block. An intra-coded block is a block that does not use predictive information from a previously reconstructed image, but may use predictive information from a previously reconstructed portion of a current image. Such predictive information may be provided by an intra-image prediction unit (252). In some cases, the intra-image prediction unit (252) generates surrounding blocks of the same size and shape as the block being reconstructed using reconstructed information extracted from a current image buffer (258). For example, the current image buffer (258) buffers a partially reconstructed current image and / or a fully reconstructed current image. In some cases, the aggregator (255) adds the prediction information generated by the intra-frame prediction unit (252) to the output sample information provided by the scaler / inverse transform unit (251) on a per-sample basis.

[0048] In other cases, the output samples of the scaler / inverse transform unit (251) may belong to an inter-frame coded and potentially motion compensated block. In this case, the motion compensated prediction unit (253) may access the reference image memory (257) to extract samples for prediction. After the extracted samples are motion compensated according to the symbols (221), these samples may be added to the output of the scaler / inverse transform unit (251) (in this case referred to as residual samples or residual signals) by the aggregator (255) to generate output sample information. The acquisition of the predicted samples by the motion compensated prediction unit (253) from the address in the reference image memory (257) may be controlled by a motion vector, and the motion vector is provided to the motion compensated prediction unit (253) in the form of the symbols (221), for example, including X, Y and reference image components. Motion compensation may also include interpolation of sample values ​​extracted from the reference image memory (257) when using sub-sample accurate motion vectors, motion vector prediction mechanisms, etc.

[0049] The output samples of the aggregator (255) may be used by various loop filtering techniques in a loop filter unit (256). The video compression techniques may include in-loop filter techniques controlled by parameters included in the encoded video sequence (also referred to as the encoded video bitstream) and available to the loop filter unit (256) as symbols (221) from the parser (220). However, in other embodiments, the video compression techniques may also be responsive to meta-information obtained during decoding of a previous (in decoding order) portion of the encoded image or encoded video sequence, as well as to previously reconstructed and loop filtered sample values.

[0050] The output of the loop filter unit (256) may be a sample stream that may be output to a display device (212) and stored in a reference picture memory (257) for subsequent inter-picture prediction.

[0051] Once fully reconstructed, certain coded pictures may be used as reference pictures for future prediction. For example, once the coded picture corresponding to the current picture is fully reconstructed, and the coded picture is identified as a reference picture (e.g., by the parser (220)), the current picture buffer (258) may become part of the reference picture memory (257), and a new current picture buffer may be reallocated before starting reconstruction of a subsequent coded picture.

[0052] The video decoder (210) may perform decoding operations according to a predetermined video compression technique, such as in the ITU-T H.265 standard. The encoded video sequence may conform to the syntax specified by the video compression technique or standard used in the sense that the encoded video sequence follows the syntax of the video compression technique or standard and the profile recorded in the video compression technique or standard. Specifically, the profile may select certain tools from all the tools available in the video compression technique or standard as the only tools available for use under the profile. For compliance, the complexity of the encoded video sequence is also required to be within the range defined by the hierarchy of the video compression technique or standard. In some cases, the hierarchy limits the maximum picture size, the maximum frame rate, the maximum reconstruction sampling rate (measured in, for example, mega samples per second), the maximum reference picture size, etc. In some cases, the limits set by the hierarchy may be further limited by the Hypothetical Reference Decoder (HRD) specification and metadata of the HRD buffer management signaled in the encoded video sequence.

[0053] In one aspect, the receiver (231) may receive additional (redundant) data along with the encoded video. The additional data may be part of the encoded video sequence. The additional data may be used by the video decoder (210) to properly decode the data and / or more accurately reconstruct the original video data. The additional data may be in the form of, for example, temporal, spatial or signal-to-noise ratio (SNR) enhancement layers, redundant slices, redundant pictures, forward error correction codes, etc.

[0054] Figure 3 An example of a block diagram of a video encoder (303) is shown. The video encoder (303) is provided in an electronic device (320). The electronic device (320) includes a transmitter (340) (eg, a transmission circuit). The video encoder (303) may be used to replace Figure 1 A video encoder (103) in an embodiment.

[0055] The video encoder (303) can be used to obtain the video source (301) (not Figure 3 In another embodiment, the video source (301) is a part of the electronic device (320) to receive video samples, and the video source can collect video images to be encoded by the video encoder (303). In another embodiment, the video source (301) is a part of the electronic device (320).

[0056] The video source (301) may provide a source video sequence in the form of a digital video sample stream to be encoded by the video encoder (303), wherein the digital video sample stream may have any suitable bit depth (e.g., 8-bit, 10-bit, 12-bit, ...), any color space (e.g., BT.601 Y CrCB, RGB, ...), and any suitable sampling structure (e.g., Y CrCb 4:2:0, Y CrCb 4:4:4). In a media service system, the video source (301) may be a storage device storing previously prepared videos. In a video conferencing system, the video source (301) may be a camera that captures local image information as a video sequence. The video data may be provided as a plurality of individual images that are given motion when viewed sequentially. The images themselves may be constructed as a spatial array of pixels, wherein each pixel may include at least one sample depending on the sampling structure, color space, etc. used. The following description focuses on the samples.

[0057] According to one aspect, the video encoder (303) can encode and compress images of a source video sequence into an encoded video sequence (343) in real time or under any other time constraints as required. Implementing the appropriate encoding speed is a function of the controller (350). In some aspects, the controller (350) controls other functional units as described below and is functionally coupled to these units. For the sake of simplicity, couplings are not shown in the figure. Parameters set by the controller (350) may include rate control related parameters (picture skipping, quantizer, lambda value of rate distortion optimization techniques, etc.), picture size, group of pictures (GOP) layout, maximum motion vector search range, etc. The controller (350) can be used to have other suitable functions that are related to the video encoder (303) optimized for a certain system design.

[0058] In some embodiments, the video encoder (303) operates in an encoding loop. As a simplified description, in embodiments, the encoding loop may include a source encoder (330) (e.g., responsible for creating symbols, such as a symbol stream, based on an input image to be encoded and a reference image) and a (local) decoder (333) embedded in the video encoder (303). The decoder (333) reconstructs the symbols to create sample data in a manner similar to how the (remote) decoder creates sample data. The reconstructed sample stream (sample data) is input to a reference image memory (334). Since the decoding of the symbol stream produces bit-accurate results that are independent of the decoder location (local or remote), the contents of the reference image memory (334) are also bit-accurate between the local encoder and the remote encoder. In other words, the reference image samples "seen" by the prediction part of the encoder are exactly the same as the sample values ​​that the decoder will "see" when using the prediction during decoding. This basic principle of reference image synchronization (and the drift that occurs when synchronization cannot be maintained, such as due to channel errors) is also used in some related technologies.

[0059] The operation of the "local" decoder (333) may be combined with, for example, Figure 2 The "remote" decoder described in detail for the video decoder (210) is identical. However, additional brief reference is made to Figure 2 , when symbols are available and the entropy encoder (345) and parser (220) are capable of losslessly encoding / decoding the symbols into an encoded video sequence, the entropy decoding portion of the video decoder (210), including the buffer memory (215) and the parser (220), may not be fully implemented in the local decoder (333).

[0060] According to one aspect, any decoder technology other than the parsing / entropy decoding present in the decoder is present in the corresponding encoder in the same or substantially the same functional form. Accordingly, the application focuses on decoder operation. The description of encoder technology can be simplified because encoder technology is mutually inverse to the decoder technology described comprehensively. A more detailed description is needed in some areas and is provided below.

[0061] During operation, in some embodiments, the source encoder (330) may perform motion compensated predictive coding. The motion compensated predictive coding predictively encodes an input image with reference to at least one previously encoded image from a video sequence designated as a "reference image." In this manner, the encoding engine (332) encodes the difference between a pixel block of the input image and a pixel block of a reference image that may be selected as a prediction reference for the input image.

[0062] The local video decoder (333) may decode the encoded video data of the picture that may be designated as the reference picture based on the symbol created by the source encoder (330). The operation of the encoding engine (332) may be a lossy process. When the encoded video data is available at the video decoder ( Figure 3 When the video sequence is decoded at a remote location (not shown), the reconstructed video sequence may typically be a copy of the source video sequence with some errors. The local video decoder (333) replicates the decoding process that may be performed by the video decoder on the reference picture and may cause the reconstructed reference picture to be stored in the reference picture memory (334). In this way, the video encoder (303) may locally store a copy of the reconstructed reference picture that has common content (absent transmission errors) with the reconstructed reference picture to be obtained by the remote video decoder.

[0063] The predictor (335) may perform a prediction search for the encoding engine (332). That is, for a new image to be encoded, the predictor (335) may search the reference image memory (334) for sample data (as candidate reference pixel blocks) or certain metadata, such as reference image motion vectors, block shapes, etc., that may serve as appropriate prediction references for the new image. The predictor (335) may operate on a pixel-by-pixel-block basis based on sample blocks to find a suitable prediction reference. In some cases, based on the search results obtained by the predictor (335), it may be determined that the input image may have prediction references taken from a plurality of reference images stored in the reference image memory (334).

[0064] The controller (350) may manage encoding operations of the source encoder (330), including, for example, setting parameters and subgroup parameters for encoding video data.

[0065] The outputs of all the above functional units may be entropy encoded in an entropy encoder (345). The entropy encoder (345) applies lossless compression to the symbols generated by the various functional units according to techniques such as Huffman coding, variable length coding, arithmetic coding, etc., thereby converting the symbols into a coded video sequence.

[0066] The transmitter (340) may buffer the encoded video sequence created by the entropy encoder (345) in preparation for transmission over a communication channel (360), which may be a hardware / software link to a storage device where the encoded video data will be stored. The transmitter (340) may combine the encoded video data from the video encoder (303) with other data to be transmitted, such as encoded audio data and / or ancillary data streams (source not shown).

[0067] The controller (350) may manage the operation of the video encoder (303). During encoding, the controller (350) may assign a certain coded picture type to each coded picture, but this may affect the coding techniques that can be applied to the corresponding picture. For example, a picture may generally be assigned to any of the following picture types:

[0068] An Intra picture (I picture) may be a picture that can be encoded and decoded without using any other picture in the sequence as a prediction source. Some video codecs allow different types of Intra pictures, including, for example, Independent Decoder Refresh ("IDR") pictures.

[0069] Predictive pictures (P pictures) can be encoded and decoded using intra prediction or inter prediction, which uses motion vectors and reference indices to predict sample values ​​for each block.

[0070] Bi-directional predictive pictures (B pictures), which can be pictures that can be encoded and decoded using intra-frame prediction or inter-frame prediction, which uses motion vectors and reference indices to predict sample values ​​for each block. Similarly, multiple predictive pictures can use more than two reference pictures and associated metadata for reconstruction of a single block.

[0071] The source picture may typically be spatially subdivided into blocks of samples (e.g., blocks of 4×4, 8×8, 4×8 or 16×16 samples) and coded block by block. These blocks may be predictively coded with reference to other (already coded) blocks, which are determined according to the coding allocation applied to the block's corresponding picture. For example, blocks of an I picture may be non-predictively coded, or they may be predictively coded (spatial prediction or intra prediction) with reference to already coded blocks of the same picture. Blocks of pixels of a P picture may be predictively coded by spatial prediction with reference to one previously coded reference picture or by temporal prediction. Blocks of a B picture may be predictively coded by spatial prediction with reference to one or two previously coded reference pictures or by temporal prediction.

[0072] The video encoder (303) may perform encoding operations according to a predetermined video encoding technique or standard, such as ITU-T H.265 Recommendation. In operation, the video encoder (303) may perform various compression operations, including predictive encoding operations that exploit temporal and spatial redundancy in an input video sequence. Thus, the encoded video data may conform to the syntax specified by the video encoding technique or standard used.

[0073] In one aspect, the transmitter (340) may transmit additional data when transmitting the encoded video. The source encoder (330) may include such data as part of the encoded video sequence. The additional data may include temporal / spatial / SNR enhancement layers, other forms of redundant data such as redundant pictures and slices, SEI messages, VUI parameter set fragments, etc.

[0074] The captured video may be taken as a plurality of source images (video images) in a temporal sequence. Intra-image prediction (often simplified to intra-frame prediction) exploits spatial correlations in a given image, while inter-image prediction exploits (temporal or other) correlations between images. In an embodiment, a particular image being encoded / decoded is divided into blocks, and the particular image being encoded / decoded is referred to as the current image. When a block in the current image is similar to a reference block in a reference image that was previously encoded in the video and is still buffered, the block in the current image may be encoded by a vector called a motion vector. The motion vector points to a reference block in a reference image, and in the case where multiple reference images are used, the motion vector may have a third dimension that identifies the reference image.

[0075] In some aspects, bidirectional prediction techniques may be used in inter-frame image prediction. According to the bidirectional prediction technique, two reference images are used, such as a first reference image and a second reference image, both of which precede the current image in the video in decoding order (but may be in the past and future, respectively, in display order). A block in the current image may be encoded by a first motion vector pointing to a first reference block in the first reference image and a second motion vector pointing to a second reference block in the second reference image. Specifically, the block may be predicted by a combination of the first reference block and the second reference block.

[0076] In addition, merge mode technology can be used in inter-picture prediction to improve coding efficiency.

[0077] According to some aspects of the present application, predictions such as inter-image prediction and intra-image prediction are performed in units of blocks. For example, according to the HEVC standard, an image in a video image sequence is divided into coding tree units (CTUs) for compression, and the CTUs in the image have the same size, such as 64×64 pixels, 32×32 pixels, or 16×16 pixels. In general, a CTU includes three coding tree blocks (CTBs), which are a luminance CTB and two chrominance CTBs. Furthermore, each CTU can be split into at least one coding unit (CU) using a quadtree. For example, a 64×64 pixel CTU can be split into a 64×64 pixel CU, or 4 32×32 pixel CUs, or 16 16×16 pixel CUs. In one aspect, each CU is analyzed to determine a prediction type for the CU, such as an inter-frame prediction type or an intra-frame prediction type. In addition, depending on temporal and / or spatial predictability, the CU is split into at least one prediction unit (PU). Typically, each PU includes a luma prediction block (PB) and two chroma PBs. In an embodiment, the prediction operation in encoding (encoding / decoding) is performed in units of prediction blocks. Taking the luma prediction block as an example of a prediction block, the prediction block includes a matrix of pixel values ​​(e.g., luma values), such as 8×8 pixels, 16×16 pixels, 8×16 pixels, 16×8 pixels, and the like.

[0078] It should be noted that the video encoder (103), video encoder (303), and video decoder (110) and video decoder (410) may be implemented using any suitable technology. In one aspect, the video encoder (303), video encoder (503), and video encoder (603), and video decoder (110) and video decoder (210) may be implemented using at least one integrated circuit. In another aspect, the video encoder (103), video encoder (303), and video decoder (110) and video decoder (210) may be implemented using at least one processor that executes software instructions.

[0079] Aspects of the present application provide techniques for improving signaling of IntraTMP.

[0080] Intra template matching prediction (also called intraTMP) is a special intra prediction mode, for example, copying the best prediction block from the reconstructed part of the current frame, the L-shaped template of which matches the current template (e.g., the template of the current block). For a predefined search range, the encoder can search for the template most similar to the current template in the reconstructed part of the current frame, and use the corresponding block as the prediction block, where the most similar template is associated with the corresponding block, and the current template is associated with the current block. The encoder then signals the use of the intraTMP mode, and the same prediction operation can be performed on the decoder side. The matching block (or corresponding block) (402) can be as shown in Figure 4 As shown, and serves as the matching area of ​​the current CU (404).

[0081] like Figure 4 As shown, a prediction signal can be generated by matching the L-shaped causal neighbors (or L-shaped template) of the current block (404) with another block in a predefined search area. Examples of predefined search areas may include R1 (current CTU), R2 (top left CTU), R3 (top CTU), and R4 (left CTU).

[0082] In one aspect, the sum of absolute differences (SAD) is used as a cost function in the IntraTMP mode. Within each search region, the decoder can search for a template (406) of a block (402) having a minimum SAD with a current template (408) of a current block (404), and use the block with the minimum SAD as a corresponding block of the current block. The corresponding block can further be used as a prediction block for the current block (404).

[0083] The size of all search regions (e.g., SearchRange_w, SearchRange_h) can be set to be proportional to the block size (e.g., BlkW, BlkH) of the current block. Accordingly, a fixed number of SAD comparisons can be obtained in each pixel. For example, the size of the search region (or search range) can be defined in Equation 1 and Equation 2 as follows: SearchRange_w = a * BlkW Equation (1) SearchRange_h = a * BlkH Equation (2) Where "a" is a constant that controls the balance between gain and complexity of the search process. In the example, "a" is equal to 5.

[0084] In one aspect, to speed up the template matching process, the search range of all search areas may be subsampled by a factor of 2. The reduced search range may result in a 4-fold reduction in template matching searches. After the best match is found, a further refinement process may be performed. Refinement may be accomplished by performing a second template matching search within a reduced range around the best match. The reduced range may be defined as min(BlkW, BlkH) / 2.

[0085] The intra template matching tool may be enabled for CUs with width and height less than or equal to 64. The maximum CU size for intra template matching may be configurable.

[0086] In one aspect, when decoder-side intra mode derivation (DIMD) is not used for the current CU, the intra template matching prediction mode may be signaled at the CU level through a dedicated flag.

[0087] In one aspect, such as in JVET-AA0044 and JVET-AB0131, intraTMP for chroma components is proposed when chroma separated trees are enabled in intra slices. In an example, a CU level flag is signaled to indicate whether intraTMP is applied to both chroma components.

[0088] IntraTMP is a special intra prediction mode for luma components. For example, in JVET-AA0044 and JVET-AB0131, it is proposed to signal a flag to use intraTMP as one of the chroma prediction modes. However, when intraTMP mode is activated in a single tree structure, the syntax (e.g., the above flag) may be redundant. In addition, when luma and chroma use different BVs, there may be potential subjective quality artifacts such as bleeding.

[0089] In the present application, signaling information (e.g., a signaling flag) is provided to indicate whether intraTMP is applied in a coded block unit or a prediction block unit for both luminance and chrominance in a single tree structure. For a separate tree (e.g., dual tree) structure, the signal flag may indicate whether intraTMP is applied to a coded luminance component or a coded chrominance component, and when the coded component is a chrominance channel and the intraTMP mode is selected, intraTMP may be applied to Cb and Cr.

[0090] In one aspect, a syntax such as intra block copy (IBC) mode, intra prediction mode, and inter prediction mode may be signaled to indicate whether intraTMP is applied to a coded component. For a dual tree structure, the signaled syntax may indicate whether intraTMP is applied to a luma component or a chroma component. For a single tree structure, the signaled syntax may indicate whether intraTMP is applied to a coded block, where the coded block includes a luma component and a chroma component.

[0091] In the example, a variable such as modeType may be applied to specify whether to use intra coding mode (MODE_INTRA), intraTMP coding mode (MODE_INTRATMP), IBC coding mode (MODE_IBC), palette coding mode (MODE_PLT), and inter coding mode. When intra coding mode, intraTMP coding mode, IBC coding mode, palette coding mode, and inter coding mode may be used, modeType may be set to MODE_TYPE_ALL. When only intra coding mode, intraTMP coding mode, IBC coding mode, and palette coding mode may be used, modeType may be set to MODE_TYPE_INTRA. When only inter coding mode may be used, modeType may be set to MODE_TYPE_INTER for coding units inside coding tree nodes.

[0092] In one aspect, when at least one coded component of a coded block is coded in IBC mode, a flag may be signaled to indicate whether intraTMP is applied.

[0093] In an example, an intraTMP flag, such as intra_tmp_flag (or encoded information), may be signaled in a syntax table (such as Table 1). As shown in Table 1, when intra_tmp_flag is true, the related syntax of IntraTMP may be signaled. When intra_tmp_flag is false, the merged data of the IBC mode may be signaled, such as x0, y0, cbWidth, cbHeight, and chType.

[0094] Table 1. Syntax table for improved IntraTMP signaling

[0095] Still referring to Table 1, when CuPredMode[chType][x0][y0] is not equal to MODE_IBC, intra_tmp_flag[x0][y0] can be inferred to be 0. When CuPredMode (or the first syntax information) is equal to MODE_IBC, it indicates that the current block is encoded by IBC. Intra_tmp_flag is further signaled to indicate whether the current block is encoded by intraTMP. If intra_tmp_flag is true, intraTMP data such as x0, y0, cbWidth, cbHeight, chType can be signaled.

[0096] In another example, intra_tmp_flag may be signaled separately from the merged data of IBC as shown in Table 2. For example, a merge flag such as general_merge_flag (or the second syntax information) may be signaled to indicate the merged data of IBC.

[0097] Table 2. Syntax table for improved IntraTMP signaling

[0098] Still referring to Table 2, when CuPredMode[chType][x0][y0] is not equal to MODE_IBC, intra_tmp_flag[x0][y0] may be inferred to be 0. When general_merge_flag is true, merge data for IBC mode may be signaled.

[0099] In one aspect, as shown in Table 3, intraTMP may be considered a sub-mode of IBC, and intraTMP may be signaled immediately after the IBC flag.

[0100] Table 3. Syntax table for improved IntraTMP signaling

[0101] As shown in Table 3, when the IBC flag (such as pred_mode_ibc_flag (or the third syntax information)) is equal to 1, the IBC flag may specify that the current coding unit is encoded in the IBC prediction mode or the intraTMP mode. When pred_mode_ibc_flag is equal to 0, the IBC flag may indicate that the current coding unit is not encoded in the IBC prediction mode or the current coding unit is not encoded in the intraTMP mode.

[0102] In an aspect, if intra_tmp_flag is true, then intraTMP related syntax (eg, intra_tmp_data) may be signaled immediately following the intraTMP flag (eg, intra_tmp_flag).

[0103] In another aspect, the intraTMP related syntax intra_tmp_data may be signaled separately from the merge data of the IBC. An example may be shown in Table 4. When MODE_IBC is true and intra_tmp_flag is true, intra_tmp_data may be signaled. Otherwise, when at least one of MODE_IBC and intra_tmp_flag is false, if general_merge_flag is true, the merge data of the IBC (e.g., merge_data) may be signaled.

[0104] Table 4. Syntax table for improved IntraTMP signaling

[0105] In one aspect, the intraTMP mode may be one of the IBC merge candidates in the merge candidate list and may be indicated by an IBC merge index. For example, a prediction block derived from intraTMP may be used as an IBC merge candidate in the merge candidate list.

[0106] In one aspect, when reordering candidates in the IBC merge candidate list in ascending order by using template matching (TM) cost, a normalized template matching cost such as distortion / template size may be used when the template size of intraTMP is different from that of IBC.

[0107] In one aspect, similar to the intraTMP signaling design in ECM, separate flags may be signaled to indicate whether intraTMP is applied to luma and chroma. The separate flags may be named luma intraTMP flag (or first syntax element) and chroma intraTMP flag (or second syntax element). However, entropy coding of the chroma intraTMP flag may use the luma intraTMP flag as context.

[0108] In one aspect, similar to the intraTMP signaling design in ECM, separate flags can be signaled to indicate whether intraTMP is applied to luma and chroma. The separate flags can be named luma intraTMP flag and chroma intraTMP flag. However, the BV for luma component derived by intraTMP can be used to implement intraTMP for chroma components.

[0109] In one aspect, the BV (eg, block vector) for the chroma components can use the BV of the luma component derived by the intraTMP as a starting point for the BV search in the intraTMP. Additional searches around the starting point BV can be used to further refine the BV of the chroma components.

[0110] In one aspect, the BV (e.g., block vector) for the chroma components can use the BV of the luma component derived by the intraTMP as the starting point for the BV search in the intraTMP. However, the BV of the luma component derived by the intraTMP can be scaled based on the chroma downsampling format. Therefore, the starting point for the BV search for the chroma components in the intraTMP can be the scaled BV of the luma component.

[0111] Figure 5 An overview flow chart of a method (500) of one aspect of the present application is shown. The method (500) can be used in a video decoder. In various aspects, the method (500) is performed by a processing circuit (such as a processing circuit that performs the functions of a video decoder (110), a processing circuit that performs the functions of a video decoder (210), etc.). In some aspects, the method (500) is implemented as software instructions, so when the processing circuit executes the software instructions, the processing circuit performs the method (500). The method starts at (S501) and proceeds to (S510).

[0112] At (S510), a video code stream is received, the video code stream comprising encoded information associated with a current block in a current image. The current block comprises a luminance component and a chrominance component. The encoded information indicates whether intraTMP is applied to the current block, wherein intraTMP is used to determine a prediction block of the current block based on a template matching cost between a template of the current block and a template of a prediction block.

[0113] At (S520), when the encoded information indicates that intraTMP is applied to the current block, a BV of a luma component of the current block is determined based on the intraTMP. A BV of a chroma component of the current block is determined based on the determined BV of the luma component of the current block.

[0114] At (S530), the current block is reconstructed based on the determined BV of the luma component and the determined BV of the chroma components.

[0115] In an example, the encoded information comprises a first syntax element indicating whether intraTMP is applied to a luma component of the current block and a second syntax element indicating whether intraTMP is applied to a chroma component of the current block.

[0116] In the example, the determined BV of the luminance component is set as the starting BV of the search area of ​​the chrominance component. A plurality of candidate BVs are determined in the search area. A template matching cost between the template of the chrominance component of the current block and the template of each of the plurality of candidate BVs is determined. A candidate BV among the plurality of candidate BVs having the smallest template matching cost with the template of the chrominance component is determined as the BV of the chrominance component.

[0117] In an example, the determined BV of the luma component is scaled according to the chroma downsampling format to obtain the starting BV of the chroma component.

[0118] In an example, the encoded information indicates whether intraTMP is applied to a luma component or a chroma component when the luma component and the chroma component are in a dual-tree structure.

[0119] In an example, the encoded information indicates whether intraTMP is applied to both the luma component and the chroma component of the current block when the luma component and the chroma component are in a single tree structure.

[0120] In an example, the video code stream further includes first syntax information indicating whether the current block is encoded with intra block copy (IBC), and the encoded information indicates whether intraTMP is applied to the current block when the first syntax information indicates that the current block is encoded with IBC.

[0121] In an example, when the encoded information indicates that intraTMP is not applied to the current block, merge data of the IBC included in the video code stream is determined.

[0122] In an example, the video bitstream further includes second syntax information indicating whether to include the merged data of IBC in the video bitstream. When the encoded information indicates that intraTMP is not applied to the current block, it is determined whether the second syntax information indicates to include the merged data of IBC in the video bitstream. When the second syntax information indicates to include the merged data of IBC in the video bitstream, the merged data of IBC is determined from the video bitstream.

[0123] In an example, the video code stream further includes third syntax information indicating whether the current block is encoded with IBC or intraTMP. When the third syntax information indicates that the current block is encoded with IBC or intraTMP, the encoded information indicates whether intraTMP is applied to the current block.

[0124] In an example, when the encoded information indicates that intraTMP is applied to the current block, intraTMP information associated with the intraTMP is determined, wherein the intraTMP information includes a width of the current block and a height of the current block.

[0125] In an example, when the encoded information indicates that intraTMP is applied to the current block, a prediction block of the current block is determined based on the intraTMP. A merge candidate list of the current block is determined to include the prediction block and at least one IBC candidate.

[0126] In an example, a normalized template matching cost of the prediction block is determined, and a normalized template matching cost of each IBC candidate in at least one IBC candidate is determined. The normalized template matching cost is equal to the template matching cost associated with the prediction block divided by the size of the template of the prediction block. The normalized template matching cost of each IBC candidate in at least one IBC candidate is equal to the template matching cost associated with the corresponding IBC candidate divided by the size of the template of the corresponding IBC candidate. The prediction block and at least one IBC candidate in the merge candidate list are reordered based on the normalized template matching cost.

[0127] Then, the method proceeds to (S599) and terminates.

[0128] The method (500) may be adjusted appropriately. At least one step in the method (500) may be modified and / or omitted. At least one additional step may be added. Any suitable execution order may be used.

[0129] Figure 6An overview flow chart of a method (600) of one aspect of the present application is shown. The method (600) can be used in a video encoder. In various aspects, the method (600) is performed by a processing circuit (such as a processing circuit that performs the functions of the video encoder (103), a processing circuit that performs the functions of the video encoder (303), etc.). In some aspects, the method (600) is implemented as software instructions, so when the processing circuit executes the software instructions, the processing circuit performs the method (600). The method starts at (S601) and proceeds to (S610).

[0130] At (S610), it is determined whether intraTMP is applied to a current block in a current image. The current block includes a luminance component and a chrominance component, and intraTMP is used to determine a prediction block of the current block based on a template matching cost between a template of the current block and a template of a prediction block.

[0131] At (S620), when it is determined that intraTMP is applied to the current block, the BV of the luma component of the current block is determined based on the intraTMP. The BV of the chroma component of the current block is determined using the determined BV of the luma component as the start BV of the search range of the intraTMP for the chroma component.

[0132] At (S630), the current block is encoded into a video code stream based on the determined BV of the luma component and the determined BV of the chroma component.

[0133] In an example, a syntax element for indicating whether intraTMP is applied to a current block in a video code stream is encoded.

[0134] In the example, at least two candidate BVs are determined within the search range. A template matching cost between the template of the chrominance component of the current block and the template of each of the at least two candidate BVs is determined. The candidate BV with the smallest template matching cost with the template of the chrominance component among the at least two candidate BVs is determined as the BV of the chrominance component.

[0135] Then, the method proceeds to (S699) and terminates.

[0136] The method (600) may be adjusted appropriately. At least one step in the method (600) may be modified and / or omitted. At least one additional step may be added. Any suitable order of execution may be used.

[0137] The above techniques may be implemented as computer software using computer-readable instructions and physically stored in at least one computer-readable medium. For example, Figure 7 A computer system (700) suitable for implementing certain aspects of the disclosed subject matter is shown.

[0138] Computer software may be encoded using any suitable machine code or computer language that may be assembled, compiled, linked or similarly constructed to create code comprising instructions that may be executed directly by at least one computer central processing unit (CPU), graphics processing unit (GPU), etc., or by interpretation, microcode execution, etc.

[0139] The instructions may be executed on various types of computers or components thereof, including, for example, personal computers, tablets, servers, smartphones, gaming devices, IoT devices, etc.

[0140] Figure 7 The components of the computer system (700) shown are examples only and are not intended to impose any limitations on the scope of use or functionality of computer software implementing aspects of the present application. The manner in which the components are deployed should not be interpreted as having any dependency or requirement on any one or combination of the components illustrated in the example aspects of the computer system (700).

[0141] The computer system (700) may include certain human interface input devices. Such human interface input devices may be responsive to input generated by at least one human user through, for example, tactile input (such as keystrokes, swipes, data glove movements), audio input (such as voice, taps), visual input (such as gestures), olfactory input (not depicted). The human interface device may also be used to capture certain media that are not necessarily directly related to a person's conscious input, such as audio (such as speech, music, ambient sounds), images (such as scanned images, photographic images obtained from a still image camera), and videos (such as two-dimensional video, three-dimensional video including stereoscopic video).

[0142] The human-machine input interface device may include at least one of the following (only one of each is depicted): keyboard (701), mouse (702), touchpad (703), touch screen (710), data gloves (not shown), joystick (705), microphone (706), scanner (707), camera (708).

[0143] The computer system (700) may also include certain human-computer interface output devices. Such human-computer interface output devices may stimulate at least one sense of a human user through, for example, tactile output, sound, light, and smell / taste. Such human-computer interface output devices may include tactile output devices (e.g., tactile feedback through a touch screen (710), a data glove (not shown), or a joystick (705), but there may also be tactile feedback devices that are not used as input devices), audio output devices (such as: speakers (709), headphones (not shown)), visual output devices, and printers (not depicted). Visual output devices may be, for example, screens (710), including CRT screens, LCD screens, plasma screens, OLED screens, virtual reality glasses (not depicted), holographic displays, and smoke boxes (not depicted). Each of the above screens may or may not have touch screen input capabilities, and each screen may or may not have tactile feedback capabilities), some of which are capable of outputting two-dimensional visual outputs or three-dimensional or higher outputs through means such as stereo output.

[0144] The computer system (700) may also include human-accessible storage devices and their associated media, such as optical media including CD / DVD ROM / RW (720) with CD / DVD or similar media (721), thumb drives (722), removable hard drives or solid-state drives (723), traditional magnetic media such as tapes and floppy disks (not depicted), dedicated ROM / ASIC / PLD based devices such as security dongles (not depicted), and the like.

[0145] Those skilled in the art will also appreciate that the term "computer-readable media" used in connection with the presently disclosed subject matter does not encompass transmission media, carrier waves, or other volatile signals.

[0146] The computer system (700) may also include an interface (754) connected to at least one communication network (755). The network may be, for example, a wireless network, a wired network, an optical network. The network may further be a local area network, a wide area network, a metropolitan area network, an in-vehicle network and an industrial network, a real-time network, a delay-tolerant network, and the like. Examples of networks include local area networks such as Ethernet, wireless LAN, cellular networks including GSM, 3G, 4G, 5G, LTE, etc., TV wired or wireless wide area digital networks including cable TV, satellite TV, and terrestrial broadcast TV, in-vehicle and industrial networks including CANBus, and the like. Some networks typically require an external network interface adapter attached to some common data port or peripheral bus (749) (e.g., a USB port of the computer system (700)); other networks are typically integrated into the kernel of the computer system (700) by attaching to a system bus as described below (e.g., an Ethernet interface integrated into a PC computer system or a cellular network interface integrated into a smartphone computer system). Using any of these networks, the computer system (700) can communicate with other entities. Such communication can be one-way receive-only (e.g., broadcast TV), one-way send-only (e.g., CANbus to certain CANbus devices), or two-way (e.g., communicating with other computer systems using a local area digital network or a wide area digital network). Certain protocols and protocol stacks may be used on each of those networks and network interfaces as described above.

[0147] The above-mentioned human interface device, human-accessible storage device, and network interface may be attached to the kernel ( 740 ) of the computer system ( 700 ).

[0148] The kernel (740) may include at least one central processing unit (CPU) (741), a graphics processing unit (GPU) (742), a dedicated programmable processing unit in the form of a field programmable gate array (FPGA) (743), hardware accelerators for certain tasks (744), a graphics adapter (750), etc. These devices, along with a read-only memory (ROM) (745), a random access memory (746), an internal mass storage device (such as an internal non-user accessible hard drive), an SSD, etc. (747) may be connected via a system bus (748). In some computer systems, the system bus (748) may be accessible in the form of at least one physical plug to enable expansion through additional CPUs, GPUs, etc. Peripheral devices may be attached directly to the kernel's system bus (748) or to the kernel's system bus (748) via a peripheral bus (749). In an example, a screen (710) may be connected to a graphics adapter (750). Architectures for peripheral buses include PCI, USB, etc.

[0149] The CPU (741), GPU (742), FPGA (743) and accelerator (744) can execute certain instructions, which can be combined to form the above-mentioned computer code. The computer code can be stored in ROM (745) or RAM (746). Transition data can also be stored in RAM (746), while permanent data can be stored in, for example, an internal mass storage device (747). Fast storage and retrieval of any memory device can be achieved by using a cache memory, which can be closely associated with at least one CPU (741), GPU (742), mass storage device (747), ROM (745), RAM (746), etc.

[0150] The computer readable medium may have thereon computer code for performing various computer-implemented operations. The medium and computer code may be those specially designed and constructed for the purposes of this application, or they may be of a type well known and available to those skilled in the art of computer software.

[0151] As an example and not by way of limitation, a computer system having the architecture (700) (and in particular the kernel (740)) can provide functionality due to at least one processor (including a CPU, GPU, FPGA, accelerator, etc.) executing software embodied in at least one tangible, computer-readable medium. Such computer-readable media can be media associated with a user-accessible mass storage device as described above, as well as certain storage devices of the kernel (740) having non-volatile properties, such as a kernel-internal mass storage device (747) or ROM (745). Software implementing various aspects of the present application can be stored in such devices and executed by the kernel (740). Depending on specific needs, the computer-readable medium may include at least one memory device or chip. The software can cause the kernel (740) (and in particular the processor therein (including a CPU, GPU, FPGA, etc.)) to perform specific processes or specific parts of specific processes described herein, including defining data structures stored in RAM (746) and modifying such data structures according to processes defined by the software. In addition or in the alternative, the computer system may provide functionality through logic hardwired or otherwise embodied in circuits (e.g., accelerators (744)) that may operate in place of or in conjunction with software to perform a particular process or a particular portion of a particular process described herein. Where appropriate, references to software may encompass logic and vice versa. Where appropriate, references to computer-readable media may encompass circuits (such as integrated circuits (ICs)) storing software for execution, circuits embodying logic for execution, or both. The present application encompasses any suitable combination of hardware and software.

[0152] The use of "at least one of" or "one of" in this application is intended to include any one or combination of the elements described. For example, references to at least one of A, B, or C; at least one of A, B, and C; at least one of A, B, and / or C; and at least one of A through C are intended to include only A, only B, only C, or any combination thereof. References to one of A or B and one of A and B are intended to include A or B or (A and B). When applicable, such as when the elements are not mutually exclusive, the use of "one of" does not exclude any combination of the elements described.

[0153] Although the present application has described several examples of various aspects, there are changes, permutations and various alternative equivalents that fall within the scope of the present application. Therefore, it should be understood that those skilled in the art will be able to design many systems and methods that, although not explicitly shown or described herein, embody the principles of the present application and are therefore within the spirit and scope of the present application.

Claims

1. A method for processing visual media data, characterized in that: include: Conversion between the visual media file and the code stream of the visual media data is performed according to the format rules, wherein: The code stream includes encoded information associated with a current block in a current image, the current block including a luminance component and a chrominance component; and The formatting rules state that: The encoded information indicates (i) whether to apply intra template matching prediction (intraTMP) to the luma component or the chroma component when the luma component and the chroma component are in a dual tree structure, and (ii) whether to apply the intraTMP to the luma component and the chroma component of the current block when the luma component and the chroma component are in a single tree structure, When the encoded information indicates that the intraTMP is applied to the current block, (i) a block vector (BV) of the luma component of the current block is determined based on the intraTMP, and (ii) the BV of the chroma component of the current block is determined based on using the determined BV of the luma component as a start BV of a search range of the intraTMP for the chroma component, and The current block is processed based on the determined BV of the luma component and the determined BV of the chroma components.

2. The method according to claim 1, characterized in that The formatting rules state that: determining at least two candidate BVs within the search range; determining a template matching cost between a template of the chroma component of the current block and a template of each of the at least two candidate BVs; as well as The candidate BV having the smallest template matching cost with the template of the chrominance component among the at least two candidate BVs is determined as the BV of the chrominance component.

3. The method according to claim 1 or 2, characterized in that: The format rule stipulates that the determined BV of the luminance component is scaled according to a chroma downsampling format to obtain the starting BV of the search range of the chroma component.

4. The method according to any one of claims 1 to 3, characterized in that: The code stream further includes a first syntax element, the first syntax element indicating whether to encode the current block with intra block copy (IBC), and The format rule stipulates that, when the first syntax element indicates that the current block is encoded with the IBC, the encoded information indicates whether the intraTMP is applied to the current block.

5. A video encoding method, characterized in that: include: determining whether to apply intra template matching prediction (intraTMP) to a current block in a current image, the current block including a luma component and a chroma component, the intraTMP being used to determine a prediction block for the current block based on a template matching cost between a template of the current block and a template of a prediction block; When it is determined that the intraTMP is applied to the current block, (i) a block vector (BV) of the luminance component of the current block is determined based on the intraTMP, and (ii) the BV of the chrominance component of the current block is determined by using the determined BV of the luminance component as a starting BV of a search range of the intraTMP for the chrominance component; as well as Based on the determined BV of the luminance component and the determined BV of the chrominance component, the current block is encoded into a video bitstream.

6. The method according to claim 5, characterized in that Further including: A syntax element for indicating whether the intraTMP is applied to the current block is encoded into the video bitstream.

7. The method according to claim 6, characterized in that The determining the BV of the chrominance component further comprises: determining at least two candidate BVs within the search range; determining a template matching cost between a template of the chroma component of the current block and a template of each of the at least two candidate BVs; and The candidate BV having the smallest template matching cost with the template of the chrominance component among the at least two candidate BVs is determined as the BV of the chrominance component.

8. A device for video decoding, characterized in that: include: A processing circuit, the processing circuit is used to: receiving a video bitstream, the video bitstream comprising encoded information associated with a current block in a current image, the current block comprising a luminance component and a chrominance component, the encoded information indicating whether to apply intra template matching prediction (intraTMP) to the current block, the intraTMP being used to determine a prediction block of the current block based on a template matching cost between a template of the current block and a template of a prediction block; When the encoded information indicates that the intraTMP is applied to the current block, (i) determining a block vector (BV) of the luma component of the current block based on the intraTMP, and (ii) determining a BV of the chroma component of the current block based on the determined BV of the luma component of the current block; as well as The current block is reconstructed based on the determined BV of the luma component and the determined BV of the chroma component.

9. The device according to claim 8, characterized in that The encoded information includes a first syntax element and a second syntax element, the first syntax element indicating whether the intraTMP is applied to the luma component of the current block, and the second syntax element indicating whether the intraTMP is applied to the chroma component of the current block.

10. The device according to claim 8 or 9, characterized in that The processing circuit is used for: Setting the determined BV of the luminance component as the starting BV of the search area of ​​the chrominance component; determining at least two candidate BVs in the search area; determining a template matching cost between a template of the chroma component of the current block and a template of each of the at least two candidate BVs; as well as The candidate BV having the smallest template matching cost with the template of the chrominance component among the at least two candidate BVs is determined as the BV of the chrominance component.

11. The device according to claim 10, characterized in that The processing circuit is used for: The determined BV of the luma component is scaled according to a chroma downsampling format to determine the starting BV of the chroma component.

12. The device according to any one of claims 8 to 11, characterized in that The encoded information indicates that when the luminance component and the chrominance component are in a dual-tree structure, the intraTMP is applied to the luminance component or the chrominance component.

13. The device according to any one of claims 8 to 12, characterized in that The encoded information indicates whether the intraTMP is applied to the luminance component and the chrominance component of the current block when the luminance component and the chrominance component are in a single tree structure.

14. The device according to any one of claims 8 to 13, characterized in that: The video code stream further includes first syntax information, wherein the first syntax information indicates whether to encode the current block using intra-frame block copy (IBC), and When the first syntax information indicates that the current block is encoded with the IBC, the encoded information indicates whether the intraTMP is applied to the current block.

15. The device according to claim 14, characterized in that The processing circuit is used for: When the encoded information indicates that the intraTMP is not applied to the current block, the merged data of the IBC included in the video code stream is determined.

16. The device according to claim 14 or 15, characterized in that: The video code stream further includes second syntax information, wherein the second syntax information indicates whether to include the merged data of the IBC in the video code stream, and The processing circuit is used for: When the encoded information indicates that the intraTMP is not applied to the current block, determining whether the second syntax information indicates that the merged data of the IBC is included in the video bitstream; as well as When the second syntax information indicates that the merged data of the IBC is included in the video code stream, the merged data of the IBC is determined from the video code stream.

17. The device according to any one of claims 8 to 14, characterized in that: The video code stream further includes third syntax information, wherein the third syntax information indicates whether to encode the current block using intra-frame block copy (IBC) or the intraTMP, and When the third syntax information indicates that the current block is encoded with the IBC or the intraTMP, the encoded information indicates whether the intraTMP is applied to the current block.

18. The device according to any one of claims 8 to 17, characterized in that The processing circuit is used for: When the encoded information indicates that the intraTMP is applied to the current block, intraTMP information associated with the intraTMP is determined, the intraTMP information including a width of the current block and a height of the current block.

19. The device according to any one of claims 14 to 17, characterized in that The processing circuit is used for: When the encoded information indicates that the intraTMP is applied to the current block, determining the prediction block of the current block based on the intraTMP; as well as It is determined that a merge candidate list of the current block includes the prediction block and at least one IBC candidate.

20. The device according to claim 19, characterized in that The processing circuit is used for: determining a normalized template matching cost of the prediction block and a normalized template matching cost of each of the at least one IBC candidate, the normalized template matching cost being equal to the template matching cost associated with the prediction block divided by a size of the template of the prediction block, and the normalized template matching cost of each of the at least one IBC candidate being equal to the template matching cost associated with the corresponding IBC candidate divided by a size of the template of the corresponding IBC candidate; as well as The prediction block and the at least one IBC candidate in the merge candidate list are reordered based on the normalized template matching cost.

Images