Sub-block level intra template matching prediction

Abstract

An apparatus for video decoding is provided. The apparatus includes processing circuitry. The processing circuitry is configured to receive a video bitstream including encoded information of a current block in a current picture and a template of the current block. The template of the current block includes samples adjacent to the current block, and the current block includes a first sub-block and a second sub-block adjacent to the first sub-block. The processing circuitry is configured to determine a template of the second sub-block based on at least one of (i) the template of the current block and (ii) the reconstructed samples of the first sub-block. The processing circuitry is configured to reconstruct the second sub-block based on the determined template in accordance with Intratemplate Matching Prediction (IntraTMP).

Description

Incorporation by Reference

[0001] This application claims the benefit of priority to U.S. Provisional Application No. 63 / 460,892, filed on April 20, 2023, entitled “Apply Intra Template Matching Prediction at Subblock Level,” which is incorporated herein by reference in its entirety. Technical Field

[0002] This disclosure describes various aspects generally related to video encoding and decoding. Background Art

[0003] The background description provided herein is intended to generally present the context of the present disclosure. The work of the inventors currently named (to the extent that the work is described in this background section), as well as aspects of the description that may not be considered prior art at the time of filing, are neither explicitly nor implicitly admitted to be prior art against the present disclosure.

[0004] Image / video compression can help transmit image / video data between different devices, storage devices, and networks while minimizing quality degradation. In some examples, video codec technology can compress video based on spatial and temporal redundancy. In an example, a video codec can use a technique called intra-frame prediction, which can compress an image based on spatial redundancy. For example, intra-frame prediction can use reference data of the current picture being reconstructed for sample prediction. In another example, a video codec can use a technique called inter-frame prediction, which can compress an image based on temporal redundancy. For example, inter-frame prediction can predict samples in the current picture based on a previously reconstructed picture through motion compensation. Motion compensation can be indicated by a motion vector (MV). Summary of the invention

[0005] Aspects of the present disclosure include code streams, methods, and apparatus for video encoding / decoding. In some examples, an apparatus for video encoding / decoding includes a processing circuit system.

[0006] According to one aspect of the present disclosure, a method for processing visual media data is provided. In the method, a code stream of visual media data including a current picture is processed according to a format rule. The code stream includes encoded information of a current block in the current picture and a template of the current block. The template of the current block has an L shape and includes samples adjacent to the top and left sides of the current block. The format rule stipulates that the current block is partitioned into a plurality of sub-blocks based on a candidate partitioning scheme indicated by an index signaled in the encoded information. The format rule stipulates that a reconstructed sample of a first sub-block among a plurality of sub-blocks is determined based on an intra-frame template matching prediction (intraTMP), so that the predicted sub-block is determined based on a cost value between the template of the first sub-block and the template of the predicted sub-block of the first sub-block. The format rule stipulates that a template of a second sub-block adjacent to the first sub-block among a plurality of sub-blocks is determined based on at least one of (i) the template of the current block and (ii) the reconstructed sample of the first sub-block. The template of the second sub-block includes at least a portion of the template of the current block. At least a portion of the template of the current block included in the template of the second sub-block is determined based on which side of the second sub-block is adjacent to the template of the current block. The format rules stipulate that the second sub-block is processed according to intraTMP based on a certain template.

[0007] In an example, the template of the second sub-block includes the reconstructed sample of the first sub-block located at a first side of the second sub-block and a portion of the template of the current block adjacent to a second side of the second sub-block.

[0008] In an example, the template of the second sub-block includes a portion of the reconstructed sample of the first sub-block adjacent to a first side of the second sub-block and a portion of the template of the current block adjacent to a second side of the second sub-block.

[0009] In an example, the template of the second sub-block includes a portion of the template of the current block adjacent to the first side of the second sub-block.

[0010] In an example, the template of the second sub-block includes a portion of the template of the current block having a width equal to a width of the first side of the second sub-block.

[0011] According to another aspect of the present disclosure, a method for video encoding is provided. In the method, a current block of a current picture is partitioned into a plurality of sub-blocks, wherein the current block includes a template having an L shape and including samples adjacent to the top and left sides of the current block. A prediction sub-block is determined for a first sub-block of the plurality of sub-blocks to obtain a prediction sub-block of the first sub-block based on an intraTMP. A template of a second sub-block adjacent to the first sub-block of the plurality of sub-blocks is determined based on at least one of (i) the template of the current block and (ii) the samples of the first sub-block. According to the intraTMP, the second sub-block is encoded based on the determined template.

[0012] In an example, the template of the second sub-block includes samples of the first sub-block located at a first side of the second sub-block and a portion of the template of the current block adjacent to a second side of the second sub-block.

[0013] In an example, the template of the second sub-block includes a portion of the samples of the first sub-block adjacent to a first side of the second sub-block and a portion of the template of the current block adjacent to a second side of the second sub-block.

[0014] According to another aspect of the present disclosure, a device for video decoding is provided. The device includes a processing circuit system. The processing circuit system is configured to receive a video code stream including encoded information of a current block in a current picture and a template of the current block. The template of the current block includes samples adjacent to the current block, and the current block includes a first subblock and a second subblock adjacent to the first subblock. The processing circuit system is configured to determine a template of a second subblock based on at least one of (i) the template of the current block and (ii) the reconstructed sample of the first subblock. The processing circuit system is configured to reconstruct the second subblock based on the determined template according to intraTMP.

[0015] In an example, the template of the second sub-block includes the reconstructed sample of the first sub-block located at a first side of the second sub-block and a portion of the template of the current block adjacent to a second side of the second sub-block.

[0016] In an example, the template of the second sub-block includes a portion of the reconstructed sample of the first sub-block adjacent to a first side of the second sub-block and a portion of the template of the current block adjacent to a second side of the second sub-block.

[0017] In an example, the template of the second sub-block includes a portion of the template of the current block adjacent to the first side of the second sub-block.

[0018] In an example, the template of the second sub-block includes a portion of the template of the current block having a width equal to a width of the first side of the second sub-block.

[0019] In an example, when the first side of the second sub-block is the left side of the second sub-block, the portion of the template of the current block includes: a first sample determined based on the width of the left side of the second sub-block, and a second sample located above the first sample. In an example, when the first side of the second sub-block is the top side of the second sub-block, the portion of the template of the current block includes: a third sample determined based on the width of the top side of the second sub-block, and a fourth sample located to the left of the third sample.

[0020] In an example, the processing circuitry is configured to partition the current block into a plurality of sub-blocks including a first sub-block and a second sub-block based on one of a plurality of candidate partitioning schemes indicated by an index included in the encoded information.

[0021] In an example, the processing circuit system is configured to determine a plurality of candidate prediction sub-blocks within a search range of the second sub-block. The processing circuit system is configured to determine a cost value between a template of the second sub-block and a template of each of the plurality of candidate prediction sub-blocks. The processing circuit system is configured to determine one or more prediction sub-blocks from the plurality of candidate prediction sub-blocks, the one or more prediction sub-blocks corresponding to one or more minimum cost values ​​among the cost values ​​between the template of the second sub-block and the templates of the plurality of candidate prediction sub-blocks. The processing circuit system is configured to reconstruct the second sub-block based on the one or more prediction sub-blocks.

[0022] In an example, when the first subblock is located at the left side of the second subblock, the left boundary of the search range is reduced by the width of the first subblock. In an example, when the first subblock is located at the top side of the second subblock, the top boundary of the search range is reduced by the height of the first subblock.

[0023] In an example, each of the plurality of sub-blocks includes one of a rectangular shape, a square shape, and a trapezoidal shape partitioned by a geometric partitioning mode (GPM).

[0024] In an example, the processing circuitry is configured to partition the current block into N parts in a horizontal direction and partition the current block into M parts in a vertical direction such that the number of the plurality of sub-blocks is equal to N×M.

[0025] In an example, the number of samples in the first sub-block is different from the number of samples in the second sub-block.

[0026] Aspects of the present disclosure also provide an apparatus for video encoding. The apparatus for video encoding includes a processing circuit system configured to implement any one of the methods for video encoding.

[0027] Aspects of the present disclosure also provide a method for video decoding, which includes any one of the methods implemented by the apparatus for video decoding.

[0028] Aspects of the present disclosure also provide a non-transitory computer-readable medium storing instructions, which, when executed by a computer, causes the computer to perform any of the methods for video decoding / encoding. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Other features, properties and various advantages of the disclosed subject matter will become more apparent from the following detailed description and accompanying drawings, in which:

[0030] Figure 1 is a schematic diagram of an example of a block diagram of a communication system (100).

[0031] Figure 2 is a diagram showing an example of a block diagram of a decoder.

[0032] Figure 3 is a diagram of an example of a block diagram of an encoder.

[0033] Figure 4 is a schematic diagram of intra template matching prediction (IntraTMP) according to some aspects of the present disclosure.

[0034] Figure 5 is a schematic diagram of a sub-block based IntraTMP according to some aspects of the present disclosure.

[0035] FIG. 6A to FIG. 6E It is a schematic diagram of a template of a coding block and sub-blocks of a coding block.

[0036] 7A to 7E is a schematic diagram of sub-block based IntraTMP via intermediate results of sub-block prediction.

[0037] FIG. 8A to FIG. 8B is a schematic diagram of a first template of a sub-block of a coding block.

[0038] FIG. 9A to FIG. 9B is a schematic diagram of a second template of a sub-block of a coding block.

[0039] FIG. 10A to FIG. 10B is a schematic diagram of a third template of a sub-block of a coding block.

[0040] FIG. 11A to FIG. 11B is a schematic diagram of the partition shapes of sub-blocks in a coding block based on the geometric partition mode (GPM).

[0041] Fig.12 A flow chart outlining a decoding process according to some aspects of the present disclosure is shown.

[0042] Fig.13 A flow chart outlining an encoding process according to some aspects of the present disclosure is shown.

[0043] Fig.14 is a schematic diagram of a computer system according to one aspect. DETAILED DESCRIPTION

[0044] Figure 1 A block diagram of a video processing system (100) in some examples is shown. The video processing system (100) is an example of an application of the disclosed subject matter, a video encoder and a video decoder in a streaming environment. The disclosed subject matter is equally applicable to other video-enabled applications including, for example, video conferencing, digital TV, streaming services, storing compressed video on digital media including CDs, DVDs, memory sticks, etc., and the like.

[0045] The video processing system (100) includes an acquisition subsystem (113), which may include a video source (101) such as a digital camera, which creates, for example, an uncompressed video picture stream (102). In one example, the video picture stream (102) includes samples captured by the digital camera. The video picture stream (102) is depicted as a bold line to emphasize that it has a higher data volume than the encoded video data (104) (or the encoded video bitstream). The video picture stream (102) is depicted as a bold line to emphasize the high data volume of the video picture stream. The video picture stream (102) can be processed by an electronic device (120), which includes a video encoder (103) coupled to the video source (101). The video encoder (103) can include hardware, software, or a combination of hardware and software to implement or implement various aspects of the disclosed subject matter as described in more detail below. The encoded video data (104) (or the encoded video code stream (104)) is depicted as a thin line to emphasize that it has a lower data volume compared to the video picture stream (102), which can be stored on the streaming server (105) for future use. One or more streaming client subsystems, such as Figure 1 A client subsystem (106) and a client subsystem (108) in a streaming server (105) may access a streaming server (105) to retrieve a copy (107) and a copy (109) of encoded video data (104). The client subsystem (106) may include, for example, a video decoder (110) in an electronic device (130). The video decoder (110) decodes the incoming copy (107) of the encoded video data and generates an output video picture stream (111) that can be presented on a display (112) (e.g., a display screen) or another presentation device (not depicted). In some streaming systems, the encoded video data (104), the video data (107), and the video data (109) (e.g., a video bitstream) may be encoded according to certain video encoding / compression standards. Examples of such standards include ITU-T Recommendation H.265. In one example, the video coding standard under development is informally referred to as Versatile Video Coding (VVC), and the present application may be used in the context of the VVC standard.

[0046] It should be noted that the electronic device (120) and the electronic device (130) may include other components (not shown). For example, the electronic device (120) may include a video decoder (not shown), and the electronic device (130) may also include a video encoder (not shown).

[0047] Figure 2An example of a block diagram of a video decoder (210) is shown. The video decoder (210) may be provided in an electronic device (230). The electronic device (230) may include a receiver (231) (eg, a receiving circuit). The video decoder (210) may be used to replace Figure 1 A video decoder (110) is shown in an example.

[0048] A receiver (231) may receive one or more encoded video sequences, such as contained in a bitstream, to be decoded by a video decoder (210); in one aspect, one encoded video sequence is received at a time, wherein the decoding of each encoded video sequence is independent of the decoding of other encoded video sequences. The encoded video sequence may be received from a channel (201), which may be a hardware / software link to a storage device storing the encoded video data. The receiver (231) may receive the encoded video data as well as other data, such as encoded audio data and / or auxiliary data streams that may be forwarded to their respective consuming entities (not shown). The receiver (231) may separate the encoded video sequence from the other data. To prevent network jitter, a buffer memory (215) may be coupled between the receiver (231) and an entropy decoder / parser (220) (hereinafter referred to as "parser (220)"). In some applications, the buffer memory (215) is part of the video decoder (210). In other cases, the buffer memory (215) may be provided outside the video decoder (210) (not shown). In other cases, a buffer memory (not shown) is provided outside the video decoder (210) to, for example, prevent network jitter, and another buffer memory (215) may be provided inside the video decoder (210) to, for example, handle broadcast timing. When the receiver (231) receives data from a storage / forward device with sufficient bandwidth and controllability, or from an isochronous network, it may not be necessary to configure the buffer memory (215), or the buffer memory may be made smaller. For use on a best-effort service packet network such as the Internet, a buffer memory (215) may also be required, and the buffer memory may be relatively large and may have an adaptive size, and may be at least partially implemented in an operating system or a similar element (not shown) outside the video decoder (210).

[0049] The video decoder (210) may include a parser (220) to reconstruct symbols (221) from the encoded video sequence. The types of symbols include information for managing the operation of the video decoder (210) and potential information for controlling a display device such as a display device (212) (e.g., a display screen) that is not part of the electronic device (230) but can be coupled to the electronic device (230), such as Figure 2As shown in . The control information for the display device may be a parameter set fragment (not indicated) of Supplemental Enhancement Information (SEI message) or Video Usability Information (VUI). The parser (220) may parse / entropy decode the received coded video sequence. The encoding of the coded video sequence may be performed according to a video coding technique or standard, and may follow various principles, including variable length coding, Huffman coding, arithmetic coding with or without context sensitivity, and the like. The parser (220) may extract a subgroup parameter set of at least one subgroup of the subgroups of pixels in the video decoder from the coded video sequence based on at least one parameter corresponding to the group. The subgroup may include a Group of Pictures (GOP), a picture, a tile, a slice, a macroblock, a Coding Unit (CU), a block, a Transform Unit (TU), a Prediction Unit (PU), and the like. The parser (220) may also extract information from the encoded video sequence, such as transform coefficients, quantizer parameter values, motion vectors, and so on.

[0050] The parser (220) may perform entropy decoding / parsing operations on the video sequence received from the buffer memory (215), thereby creating symbols (221).

[0051] Depending on the type of coded video picture or part of coded video picture (e.g., inter-picture and intra-picture, inter-block and intra-block) and other factors, the reconstruction of symbol (221) may involve multiple different units. Which units are involved and how they are involved may be controlled by subgroup control information parsed by parser (220) from the coded video sequence. For the sake of brevity, such subgroup control information flow between parser (220) and the multiple units below is not described.

[0052] In addition to the functional blocks already mentioned, the video decoder (210) can be conceptually subdivided into several functional units as described below. In a practical embodiment operating under commercial constraints, many of these units interact closely with each other and can be integrated with each other. However, for the purpose of describing the disclosed subject matter, the conceptual subdivision into the following functional units is appropriate.

[0053] The first unit is a sealer / inverse transform unit (251). The sealer / inverse transform unit (251) receives quantized transform coefficients as symbols (221) from the parser (220) and control information, including which transform method to use, block size, quantization factor, quantization scaling matrix, etc. The sealer / inverse transform unit (251) can output a block including sample values, which can be input into an aggregator (255).

[0054] In some cases, the output samples of the scaler / inverse transform unit (251) may belong to an intra-coded block. The intra-coded block is a block that does not use predictive information from a previously reconstructed picture, but can use predictive information from a previously reconstructed portion of the current picture. Such predictive information may be provided by an intra-prediction unit (252). In some cases, the intra-prediction unit (252) uses surrounding reconstructed information extracted from the current picture buffer (258) to generate a block of the same size and shape as the block being reconstructed. For example, the current picture buffer (258) buffers a partially reconstructed current picture and / or a fully reconstructed current picture. In some cases, the aggregator (255) adds the prediction information generated by the intra-prediction unit (252) to the output sample information provided by the scaler / inverse transform unit (251) on a per-sample basis.

[0055] In other cases, the output samples of the scaler / inverse transform unit (251) may belong to an inter-frame coded and potentially motion compensated block. In this case, the motion compensated prediction unit (253) may access the reference picture memory (257) to extract samples for prediction. After the extracted samples are motion compensated according to the symbols (221) belonging to the block, these samples may be added to the output of the scaler / inverse transform unit (251) (in this case referred to as residual samples or residual signals) by the aggregator (255) to generate output sample information. The acquisition of the predicted samples by the motion compensated prediction unit (253) from the address in the reference picture memory (257) may be controlled by a motion vector, and the motion vector is provided to the motion compensated prediction unit (253) in the form of the symbols (221), for example, including X, Y and reference picture components. Motion compensation may also include interpolation of sample values ​​extracted from the reference picture memory (257) when using sub-sample accurate motion vectors, motion vector prediction mechanisms, etc.

[0056] The output samples of the aggregator (255) may be employed by various loop filtering techniques in a loop filter unit (256). The video compression techniques may include in-loop filter techniques controlled by parameters included in an encoded video sequence (also referred to as an encoded video bitstream) and available to the loop filter unit (256) as symbols (221) from the parser (220). The video compression may also be responsive to meta-information obtained during decoding of an encoded picture or a previous (in decoding order) portion of an encoded video sequence, and to previously reconstructed and loop filtered sample values.

[0057] The output of the loop filter unit (256) may be a sample stream that may be output to a display device (212) and stored in a reference picture memory (257) for subsequent inter-picture prediction.

[0058] Once fully reconstructed, certain coded pictures may be used as reference pictures for future prediction. For example, once the coded picture corresponding to the current picture is fully reconstructed, and the coded picture is identified as a reference picture (e.g., by the parser (220)), the current picture buffer (258) may become part of the reference picture memory (257), and a new current picture buffer may be reallocated before starting to reconstruct a subsequent coded picture.

[0059] The video decoder (210) may perform decoding operations according to a predetermined video compression technique in a standard such as ITU-T H.265 Recommendation. The encoded video sequence may conform to the syntax specified by the video compression technology or standard used in the sense that the encoded video sequence follows the syntax of the video compression technology or standard and the profile recorded in the video compression technology or standard. Specifically, the profile may select certain tools from all the tools available in the video compression technology or standard as the only tools available for use under the profile. For compliance, the complexity of the encoded video sequence is also required to be within the range defined by the hierarchy of the video compression technology or standard. In some cases, the hierarchy limits the maximum picture size, the maximum frame rate, the maximum reconstruction sampling rate (measured in, for example, mega samples per second), the maximum reference picture size, etc. In some cases, the limits set by the hierarchy may be further limited by the Hypothetical Reference Decoder (HRD) specification and metadata of the HRD buffer management signaled in the encoded video sequence.

[0060] In one aspect, the receiver (231) may receive additional (redundant) data along with the encoded video. The additional data may be part of the encoded video sequence. The additional data may be used by the video decoder (210) to properly decode the data and / or more accurately reconstruct the original video data. The additional data may be in the form of, for example, temporal, spatial or signal-to-noise ratio (SNR) enhancement layers, redundant slices, redundant pictures, forward error correction codes, etc.

[0061] Figure 3 An example of a block diagram of a video encoder (303) is shown. The video encoder (303) is disposed in an electronic device (320). The electronic device (320) includes a transmitter (340) (eg, a transmission circuit). The video encoder (303) may be used to replace Figure 1 A video encoder (103) in an embodiment.

[0062] The video encoder (303) can be used to obtain the video source (301) (not Figure 3 In another embodiment, the video source (301) is a part of the electronic device (320) in the example, and receives the video samples. The video source can collect video images to be encoded by the video encoder (303). In another embodiment, the video source (301) is a part of the electronic device (320).

[0063] The video source (301) may provide a source video sequence in the form of a digital video sample stream to be encoded by the video encoder (303), wherein the digital video sample stream may have any suitable bit depth (e.g., 8-bit, 10-bit, 12-bit, ...), any color space (e.g., BT.601 Y CrCB, RGB, ...), and any suitable sampling structure (e.g., Y CrCb 4:2:0, Y CrCb 4:4:4). In a media service system, the video source (301) may be a storage device storing previously prepared videos. In a video conferencing system, the video source (301) may be a camera that captures local image information as a video sequence. The video data may be provided as a plurality of individual pictures that are given motion when viewed sequentially. The pictures themselves may be constructed as a spatial pixel array, wherein each pixel may include one or more samples depending on the sampling structure, color space, etc. used. The following description focuses on the samples.

[0064] According to one aspect, the video encoder (303) can encode and compress pictures of a source video sequence into an encoded video sequence (343) in real time or under any other time constraints required by the application. Implementing the appropriate encoding speed is a function of the controller (350). In some embodiments, the controller (350) controls other functional units as described below and is functionally coupled to these units. For the sake of simplicity, the coupling is not marked in the figure. The parameters set by the controller (350) may include rate control related parameters (picture skipping, quantizer, lambda value of rate distortion optimization technology, etc.), picture size, group of pictures (GOP) layout, maximum motion vector search range, etc. The controller (350) can be used to have other suitable functions that are related to the video encoder (303) optimized for a certain system design.

[0065] In some aspects, the video encoder (303) operates in a coding loop. As a simple description, in one example, the coding loop may include a source encoder (330) (e.g., responsible for creating symbols, such as a symbol stream, based on an input picture to be encoded and a reference picture) and a (local) decoder (333) embedded in the video encoder (303). The decoder (333) reconstructs the symbols to create sample data in a manner similar to how the (remote) decoder creates sample data. The reconstructed sample stream (sample data) is input to a reference picture memory (334). Since the decoding of the symbol stream produces bit-accurate results that are independent of the decoder location (local or remote), the contents of the reference picture memory (334) are also bit-accurate between the local encoder and the remote encoder. In other words, the reference picture samples "seen" by the prediction part of the encoder are exactly the same as the sample values ​​that the decoder will "see" when using the prediction during decoding. This basic principle of reference picture synchronization (and the drift that occurs when synchronization cannot be maintained, such as due to channel errors) is also used in some related technologies.

[0066] The operation of the "local" decoder (333) may be combined with, for example, Figure 2 The "remote" decoder described in detail for the video decoder (210) is identical. However, additional brief reference is made to Figure 2 , when symbols are available and the entropy encoder (345) and parser (220) are capable of losslessly encoding / decoding the symbols into the encoded video sequence, the entropy decoding portion of the video decoder (210), including the buffer memory (215) and the parser (220), may not be fully implemented in the local decoder (333).

[0067] In one aspect, decoder techniques other than parsing / entropy decoding present in a decoder are also present in a corresponding encoder in the same or substantially the same functional form. Therefore, the present application focuses on decoder operation. The description of encoder techniques can be simplified because encoder techniques are mutually inverse to the decoder techniques described comprehensively. In some areas, a more detailed description is provided below.

[0068] During operation, in some examples, the source encoder (330) may perform motion compensated predictive coding. The motion compensated predictive coding predictively encodes an input picture with reference to one or more previously encoded pictures from a video sequence designated as "reference pictures." In this manner, the encoding engine (332) encodes the difference between a pixel block of the input picture and a pixel block of a reference picture that may be selected as a prediction reference for the input picture.

[0069] The local video decoder (333) may decode the encoded video data of the picture that may be designated as the reference picture based on the symbol created by the source encoder (330). The operation of the encoding engine (332) may be a lossy process. When the encoded video data is available at the video decoder ( Figure 3 When the video sequence is decoded at a remote location (not shown), the reconstructed video sequence may typically be a copy of the source video sequence with some errors. The local video decoder (333) replicates the decoding process that may be performed by the video decoder on the reference picture and may cause the reconstructed reference picture to be stored in the reference picture memory (334). In this way, the video encoder (303) may locally store a copy of the reconstructed reference picture that has common content (absent transmission errors) with the reconstructed reference picture to be obtained by the remote video decoder.

[0070] The predictor (335) may perform a prediction search for the encoding engine (332). That is, for a new picture to be encoded, the predictor (335) may search the reference picture memory (334) for sample data (as candidate reference pixel blocks) or certain metadata, such as reference picture motion vectors, block shapes, etc., that may serve as appropriate prediction references for the new picture. The predictor (335) may operate pixel-by-pixel based on sample blocks to find a suitable prediction reference. In some cases, based on the search results obtained by the predictor (335), it may be determined that the input picture may have prediction references taken from a plurality of reference pictures stored in the reference picture memory (334).

[0071] The controller (350) may manage encoding operations of the source encoder (330), including, for example, setting parameters and subgroup parameters for encoding video data.

[0072] The outputs of all the above functional units may be entropy encoded in an entropy encoder (345). The entropy encoder (345) performs lossless compression on the symbols generated by the various functional units according to techniques such as Huffman coding, variable length coding, arithmetic coding, etc., thereby converting the symbols into a coded video sequence.

[0073] The transmitter (340) may buffer the encoded video sequence created by the entropy encoder (345) in preparation for transmission over a communication channel (360), which may be a hardware / software link to a storage device where the encoded video data will be stored. The transmitter (340) may combine the encoded video data from the video encoder (303) with other data to be transmitted, such as encoded audio data and / or ancillary data streams (source not shown).

[0074] The controller (350) may manage the operation of the video encoder (303). During encoding, the controller (350) may assign a certain coded picture type to each coded picture, but this may affect the coding techniques that can be applied to the corresponding picture. For example, a picture may generally be assigned to any of the following picture types:

[0075] An intra picture (I picture) may be a picture that can be encoded and decoded without using any other picture in the sequence as a prediction source. Some video codecs allow different types of intra pictures, including, for example, Independent Decoder Refresh ("IDR") pictures.

[0076] A predictive picture (P picture) may be a picture that may be encoded and decoded using intra prediction or inter prediction, which uses a motion vector and a reference index to predict sample values ​​of each block.

[0077] Bidirectional predictive pictures (B pictures), which can be pictures that can be encoded and decoded using intra prediction or inter prediction, which uses up to two motion vectors and reference indices to predict the sample values ​​of each block. Similarly, multiple predictive pictures can use more than two reference pictures and associated metadata for reconstruction of a single block.

[0078] The source picture may typically be spatially subdivided into a number of blocks of samples (e.g., blocks of 4×4, 8×8, 4×8, or 16×16 samples each), and coded block by block. These blocks may be predictively coded with reference to other (already coded) blocks, which are determined according to the coding allocation applied to the block's corresponding picture. For example, blocks of an I picture may be non-predictively coded, or the blocks may be predictively coded (spatial prediction or intra-frame prediction) with reference to already coded blocks of the same picture. Blocks of pixels of a P picture may be predictively coded by spatial prediction with reference to one previously coded reference picture or by temporal prediction. Blocks of a B picture may be predictively coded by spatial prediction with reference to one or two previously coded reference pictures or by temporal prediction.

[0079] The video encoder (303) may perform encoding operations according to a predetermined video encoding technique or standard, such as ITU-T H.265 Recommendation. In operation, the video encoder (303) may perform various compression operations, including predictive encoding operations that exploit temporal and spatial redundancy in an input video sequence. Thus, the encoded video data may conform to the syntax specified by the video encoding technique or standard used.

[0080] In one aspect, the transmitter (340) may transmit additional data along with the encoded video. The source encoder (330) may include such data as part of the encoded video sequence. The additional data may include temporal / spatial / SNR enhancement layers, redundant pictures and slices, other forms of redundant data, SEI messages, VUI parameter set fragments, etc.

[0081] The captured video may be taken as a plurality of source pictures (video pictures) in a temporal sequence. Intra-picture prediction (often simplified to intra-prediction) exploits spatial correlations in a given picture, while inter-picture prediction exploits (temporal or other) correlations between pictures. In an embodiment, a particular picture being encoded / decoded is divided into blocks, and the particular picture being encoded / decoded is referred to as the current picture. When a block in the current picture is similar to a reference block in a reference picture that was previously encoded in the video and is still buffered, the block in the current picture may be encoded by a vector called a motion vector. The motion vector points to a reference block in a reference picture, and in the case where multiple reference pictures are used, the motion vector may have a third dimension that identifies the reference picture.

[0082] In some aspects, bidirectional prediction techniques may be used in inter-picture prediction. According to the bidirectional prediction technique, two reference pictures are used, such as a first reference picture and a second reference picture that are both before the current picture in the video in decoding order (but may be in the past and future, respectively, in display order). A block in the current picture may be encoded by a first motion vector pointing to a first reference block in the first reference picture and a second motion vector pointing to a second reference block in the second reference picture. The block may be predicted by a combination of the first reference block and the second reference block.

[0083] In addition, merge mode technology can be used in inter-picture prediction to improve coding efficiency.

[0084] According to some aspects disclosed in the present application, predictions such as inter-picture prediction and intra-picture prediction are performed in units of blocks. For example, according to the HEVC standard, pictures in a video picture sequence are divided into coding tree units (CTUs) for compression, and the CTUs in the pictures have the same size, such as 64×64 pixels, 32×32 pixels, or 16×16 pixels. In general, a CTU includes three coding tree blocks (CTBs), which are a luminance CTB and two chrominance CTBs. Furthermore, each CTU can be split into one or more coding units (CUs) using a quadtree. For example, a 64×64 pixel CTU can be split into a 64×64 pixel CU, or 4 32×32 pixel CUs, or 16 16×16 pixel CUs. In an embodiment, each CU is analyzed to determine a prediction type for the CU, such as an inter-prediction type or an intra-prediction type. In addition, depending on temporal and / or spatial predictability, the CU is split into one or more prediction units (PUs). Typically, each PU includes a luma prediction block (PB) and two chroma PBs. In one aspect, the prediction operation in encoding (encoding / decoding) is performed in units of prediction blocks. Taking the luma prediction block as an example of a prediction block, the prediction block includes a matrix of pixel values ​​(e.g., luma values), such as 8×8 pixels, 16×16 pixels, 8×16 pixels, 16×8 pixels, and the like.

[0085] Note that the video encoders (103) and (303) and the video decoders (110) and (210) can be implemented using any suitable technology. In one aspect, the video encoders (103) and (303) and the video decoders (110) and (210) can be implemented using one or more integrated circuits. In another aspect, the video encoders (103) and (303) and the video decoders (110) and (210) can be implemented using one or more processors that execute software instructions.

[0086] Aspects of the present disclosure provide techniques for improving signaling of IntraTMP.

[0087] Intra-frame template matching prediction (also known as IntraTMP) is a special intra-frame prediction mode that copies the best prediction block from the reconstructed part of the current frame, whose L-shaped template matches the current template (e.g., the template of the current block). For a predefined search range, the encoder searches for the template most similar to the current template in the reconstructed part of the current frame, and uses the corresponding block as the prediction block, where the most similar template is associated with the corresponding block and the current template is associated with the current block. The encoder then signals the use of the intraTMP mode, and the same prediction operation can be performed on the decoder side. The matching block (or corresponding block) (402) can be at Figure 4 , and serves as the matching area for the current CU ( 404 ).

[0088] like Figure 4 As shown, a prediction signal can be generated by matching an L-shaped causal neighbor (or L-shaped template) of the current block (404) with another block in a predefined search area. Examples of predefined search areas may include R1 (current CTU), R2 (top left CTU), R3 (top CTU), and R4 (left CTU).

[0089] The matching is performed based on a cost function. In one aspect, the sum of absolute differences (SAD) is used as the cost function in the IntraTMP mode. Within each search area, the decoder can search for a template (406) of a block (402) having a minimum SAD relative to a current template (408) of a current block (404), and use the block with the minimum SAD as a corresponding block for the current block. The corresponding block can further serve as a prediction block for the current block (404).

[0090] The sizes of all search regions (e.g., SearchRange_w, SearchRange_h) can be set to be proportional to the block size (e.g., BlkW, BlkH) of the current block. Therefore, a fixed number of SAD comparisons can be obtained in each pixel. For example, the size of the search region (or search range) can be defined in Equation 1 and Equation 2 as follows: SearchRange_w = a * BlkW Equation (1) SearchRange_h = a * BlkH Equation (2) Where "a" is a constant that controls the balance between gain and complexity of the search process. In the example, "a" is equal to 5.

[0091] In one aspect, to speed up the template matching process, the search range of all search areas may be subsampled by a factor of 2. The reduced search range may result in a 4-fold reduction in template matching searches. After the best match is found, a further refinement process may be performed. Refinement may be accomplished by performing a second template matching search with a reduced range around the best match. The reduced range may be defined as min(BlkW, BlkH) / 2.

[0092] The intra template matching tool may be enabled for CUs with width and height less than or equal to 64. The maximum CU size for intra template matching may be configurable.

[0093] In one aspect, when decoder-side intra mode derivation (DIMD) is not used for the current CU, the intra template matching prediction mode may be signaled at the CU level through a dedicated flag.

[0094] The result of intraTMP for the current block can be a reference block. The reference block can be indicated by a single block vector (BV), indicating the location of the reference block related to the current coding block. In the current coding block, there may be two or more regions with different textures. It may be difficult to find a single BV to predict / match the current coding block. However, if the current coding block is divided into multiple sub-blocks, it may be easier to predict multiple sub-blocks based on two or more BVs.

[0095] Figure 5An example of sub-block based intraTMP is given, where two objects (504) and (506) are included in a current block (502). The two objects may each have a good match in a search area (508). For example, in the search area (508), object (504) may have a good match (510) and object (506) may have a good match (512). However, the large distance between the two objects may prevent the encoder from finding a single good reference block. In contrast, if the current block can be divided into two sub-blocks by a dotted line (514) in the middle of the current block (502), each sub-block may find a better match, thereby making a better prediction for the entire coding block (502).

[0096] Problems may arise when performing intraTMP prediction at the sub-block level. Fig. 6A An L-shaped reference template (602) of a current block (604) is shown. The L-shaped reference template (602) may be located in the top direction and the left direction of the current block (604). Samples in the L-shaped reference template area may be reconstructed samples that may be obtained from a previously coded block. If the L-shaped reference samples are applied to sub-blocks of the current block (604), some of the sub-blocks may not have neighboring samples that may be used as templates. Fig. 6A An example is shown when the current block (604) is divided into a 2×2 grid to generate 4 sub-blocks. These sub-blocks can be indicated by numbers 0 to 3. For sub-block 0, Figure 6B As shown in FIG. 6 , the L-shaped template of sub-block 0 may still be usable because the L-shaped template of sub-block 0 can apply samples in the L-shaped reference template (602). However, for sub-blocks 1 to 3, some samples may be missing in the L-shaped templates of sub-blocks 1 to 3. For example, Figure 6C As shown, for sub-block 1, the left part of the L-shaped template is missing. Fig.6D As shown, sub-block 2 may be missing the top portion of the L-shaped template. Fig. 6E As shown, sub-block 3 may be missing the top and left portions of the L-shaped template.

[0097] In one possible solution, the intermediate reconstruction results of a sub-block can be used to predict other sub-blocks. 7A to 7E An example of sequential intraTMP prediction is given. Fig. 7A As shown, the current block (704) can be partitioned into 4 sub-blocks 0 to 3. The current block (704) can have an L-shaped template (702). Figure 7B In the example, sub-block 0 can be reconstructed based on intraTMP via an L-shaped template (702). For sub-block 1, Figure 7CAs shown, sub-block 1 may be missing the left template. The missing left template can be derived from the reconstructed samples of sub-block 0 after predicting sub-block 0. For example, an inverse transform or other method for sub-block 0 may be required to derive the residual signal. Sub-block 0 can then be reconstructed using the prediction and the residual signal. Other sub-blocks can reuse intermediate reconstructed samples from previous sub-blocks in a similar manner. For sub-block 2, as shown in FIG. Fig.7D As shown, the missing top portion of the template of sub-block 2 can be derived from the reconstructed samples of sub-block 0 and sub-block 1. For sub-block 3, Fig. 7E As shown, the missing left portion and the missing top portion of the template of sub-block 3 can be derived according to sub-block 0, sub-block 1 and sub-block 2.

[0098] In another possible solution, prediction of sub-blocks of a coding block can be performed using available template information from a previously coded block. However, inverse transformation or other methods of deriving residual signals are postponed until the entire coding block is predicted. For example, Figure 7C As shown, for sub-block 1, the template of sub-block 1 may only include samples of the L-shaped template (702) located on the top side of sub-block 1. For sub-block 2, as shown Fig.7D As shown, the template of sub-block 2 may only include samples of the L-shaped template (702) located on the left side of sub-block 2. For sub-block 3, as Fig. 7E As shown, the BV of sub-block 1 and / or sub-block 2 can be reused to predict sub-block 3. After all four sub-blocks 0 to 3 are predicted, an inverse transform at the block level (or other methods of deriving a residual signal) can be performed. Based on the residual signal at the coding block level and the combined prediction signal from the sub-block level, the reconstructed samples of the entire block (e.g., (704)) can be obtained.

[0099] In one aspect, intraTMP may be performed at the sub-block level.

[0100] In one aspect, for each sub-block, a sub-block (eg, Figure 7B After performing a prediction on the sub-block 0 in the image to obtain a prediction, a corresponding inverse transform or other alternative method may be applied to derive a residual signal. For example, the residual signal may be the difference between the sub-block and the prediction. The prediction obtained at the sub-block level and the derived residual signal may be used to generate a reconstructed sample of the sub-block. The reconstructed sample of the sub-block may be used as the sample for other right sub-blocks and / or bottom sub-blocks (e.g., Fig.7D Therefore, if FIG. 6A to FIG. 6E or 7A to 7EAs shown, the right sub-block (e.g., sub-block 1) and / or the bottom sub-block (e.g., sub-block 3) may lack a part or all of the L-shaped template due to block partitioning, and the right sub-block and / or the bottom sub-block may still be encoded by intraTMP using the reconstructed samples of the adjacent sub-blocks. In other words, the sub-blocks (e.g., Figure 7B The sub-block 0) in can be a coding unit, a prediction unit and a transform unit at the same time.

[0101] Fig. 8A and Figure 8B An example of a template of a sub-block based on reconstructed samples of neighboring sub-blocks is shown. In the example, Fig. 8A As shown, sub-block 1 may include an L-shaped template (804). The L-shaped template (804) may include a reconstructed sample of sub-block 0 that was previously encoded and a portion of the L-shaped template (802) of the encoding block including sub-blocks 0 to 1. In an example, the reconstructed sample of sub-block 0 may be located at a first side (e.g., left side) of sub-block 1, and the portion of the L-shaped template (802) may be located at a second side (e.g., top side) of sub-block 1. In an example, as Figure 8B As shown, the L-shaped template (806) of sub-block 1 may include a portion of the reconstructed sample of sub-block 0 and a portion of the L-shaped template (802). In an example, a width W1 of the portion of the reconstructed sample of sub-block 0 and a width W2 of the portion of the L-shaped template (802) may be equal. In an example, the width W1 may be half the width of sub-block 0.

[0102] In an example, the size of the L-shaped template may be adaptively changed to the sub-block level. For example, the L-shaped template of the sub-block may only include samples at the left side, top side, and upper left side of the sub-block, and may not include samples at the upper right side and lower left side of the sub-block.

[0103] In one aspect, the template of the sub-block may be adaptively selected based on the previously encoded block, but the intermediate reconstructed samples of the previously encoded sub-block may not be reused. Therefore, the template of the sub-block may be derived using neighboring reconstructed samples (e.g., some or as many as possible) of the previously encoded block. For example, Figure 6CAs shown, although sub-block 1 may be missing left samples to have an L-shaped template, the top samples are available in the L-shaped template (602). Therefore, the top samples of sub-block 1 in the L-shaped template (602) can be selected as the template. Similar rules can be applied to sub-block 2, where only the left samples of sub-block 2 can be used as a template. For sub-block 3, sub-block 3 can use the derived BV of sub-block 1 and / or sub-block 2 to perform prediction. After all sub-blocks have been predicted, inverse transform or other methods of deriving residual signals can be performed at the coding block level (but not at the sub-block level). Based on the residual signal at the block level and the collective prediction signal at the sub-block level, the reconstructed samples of the coding block (e.g., (604)) can be derived. Therefore, the coding block (e.g., (604)) can be considered as a coding unit and a transform unit, but not a prediction unit. On the contrary, the coding block can be a collection of prediction units at the sub-block level.

[0104] In the example, for a sub-block that is missing a portion of the L-shaped template, only the immediate top sample / left sample can be considered as the template. Fig. 9A As shown, for sub-block 1, the upper left sample of sub-block 1 in the original L-shaped template (902) may not be included as the template of sub-block 1. Only the top sample (904) within the width of sub-block 1 in the original L-shaped template (902) may be defined as the template of sub-block 1. Fig. 9B For sub-block 2 in the original L-shaped template (902), the upper left sample of sub-block 2 in the original L-shaped template (902) may not be included as the template of sub-block 2. Only the remaining samples within the height of sub-block 2 in the original L-shaped template (902) may be defined as the template of sub-block 2.

[0105] In an example, for a sub-block of a coding block that lacks a portion of an L-shaped template, the top sample / left sample of the sub-block within the template of the coding block may be considered as the template of the sub-block. Fig. 10A For sub-block 1 in the coding block, the template (1004) of sub-block 1 may include all top samples and left samples in the original L-shaped template (1002) of the coding block including sub-blocks 0 to 1. Fig. 10B For sub-block 2 in the coding block, the template (1006) of sub-block 2 may include all top samples and left samples in the original L-shaped template (1002) of the coding block including sub-blocks 0 to 2.

[0106] In an aspect, a flag may be further signaled after the intraTMP flag to indicate whether a specific partitioning scheme is enabled for a coding block under consideration for intraTMP (or a coding block under consideration for intraTMP based on the indication of the intraTMP flag).

[0107] In the example, only one partitioning scheme may be allowed.

[0108] In an example, a set of partition schemes may be allowed. A combination of disable / enable flags may be used to indicate which partition scheme in the set of partition schemes to apply.

[0109] In an example, a set of partition schemes may be allowed. An index may be used to indicate which partition scheme in the set of partition schemes to use.

[0110] In one aspect, the search range of the coding block can be adapted to the sub-block. For example, for sub-block 1 in FIG. 6 , the maximum allowed sample distance from the left side (or the left boundary of the search range) can be reduced by the width of the adjacent sub-block 0, such as 4 (or 4 samples), because sub-block 1 may have been shifted 4 samples to the right (or the right side of the L-shaped template (602)). For sub-block 2, the maximum allowed sample distance from the top (or the top boundary of the search range) can be reduced by 4 (or 4 samples), because sub-block 2 may have been shifted 4 samples to the bottom (or the bottom side of the L-shaped template (602)).

[0111] In one aspect, the allowed number of sub-blocks in a coding block may be N×M, where N is the number of partitions (or sections) in the horizontal direction, and M is the number of partitions (or sections) in the vertical direction. In an example, as shown in FIG6 , N=2, and M=2. However, N and M may be any suitable numbers.

[0112] In an example, N=1, or M=1. In this case, such partitioning may be consistent with an intra sub-partitioning (ISP) coding tool, such as ISP in VVC. The sub-block partitioning of intraTMP may be considered as coordination with the ISP coding tool.

[0113] In an example, many sub-blocks can fit into the coding block size. For example, a fixed size of a sub-block can be defined as 8×8 or 4×4. In this case, a large coding block (e.g., greater than or equal to 32×32) can have more sub-blocks.

[0114] In one aspect, the partition shape may not be limited to a rectangular shape or a square shape. For example, the sub-block may be a partition result of other existing coding tools such as a geometric partition mode (GPM), where the partition of the coding block may be in Fig.11A and Fig. 11B In addition, GPM may not be limited to encoding blocks using inter-frame prediction. The encoding blocks may be encoded by a combination of inter-frame-intra-frame or intra-frame-intra-frame. For example, Fig.11A As shown, the block (1102) can be partitioned into partition P0 and partition P1 by partition line (1104). Partition P0 can be encoded by inter-frame prediction or intra-frame prediction. Partition P1 can be encoded by inter-frame prediction or intra-frame prediction. Fig. 11BAs shown, the block (1106) can be partitioned into partition P0 and partition P1 by partition line (1108). Partition P0 can be encoded by inter-frame prediction or intra-frame prediction. Partition P1 can be encoded by inter-frame prediction or intra-frame prediction.

[0115] In one aspect, the division (or size) of the sub-blocks may not be uniform. The number of samples in each sub-block may be different.

[0116] In one aspect, each sub-block may use multiple candidate reference blocks. A weighted average of the candidate blocks (also known as a fusion method) may be used to derive the final prediction value for each sub-block. For example, Figure 7B As shown, sub-block 0 may have multiple candidate blocks (or candidate prediction blocks) based on intraTMP within the search range. The prediction value of sub-block 0 may be determined based on a weighted average of the multiple candidate prediction blocks.

[0117] Fig.12 A flow chart outlining a process (1200) according to an aspect of the present disclosure is shown. The process (1200) may be used in a video decoder. In various aspects, the process (1200) is performed by a processing circuit system (such as a processing circuit system that performs the functions of a video decoder (110), a processing circuit system that performs the functions of a video decoder (210), etc.). In some aspects, the process (1200) is implemented in software instructions, so when the processing circuit system executes the software instructions, the processing circuit system performs the process (1200). The process starts at (S1201) and proceeds to (S1210).

[0118] At (S1210), a video code stream is received. The video code stream includes encoded information of a current block in a current picture and a template of the current block. The template of the current block includes samples adjacent to the current block, and the current block includes a first subblock and a second subblock adjacent to the first subblock.

[0119] At (S1220), a template of the second sub-block is determined based on at least one of (i) the template of the current block and (ii) the reconstructed sample of the first sub-block.

[0120] At (S1230), the second sub-block is reconstructed based on the determined template according to intraTMP.

[0121] In an example, the template of the second sub-block includes the reconstructed sample of the first sub-block located at a first side of the second sub-block and a portion of the template of the current block adjacent to a second side of the second sub-block.

[0122] In an example, the template of the second sub-block includes a portion of the reconstructed sample of the first sub-block adjacent to a first side of the second sub-block and a portion of the template of the current block adjacent to a second side of the second sub-block.

[0123] In an example, the template of the second sub-block includes a portion of the template of the current block adjacent to the first side of the second sub-block.

[0124] In an example, the template of the second sub-block includes a portion of the template of the current block having a width equal to a width of the first side of the second sub-block.

[0125] In an example, when the first side of the second sub-block is the left side of the second sub-block, the portion of the template of the current block includes: a first sample determined based on the width of the left side of the second sub-block, and a second sample located above the first sample. In an example, when the first side of the second sub-block is the top side of the second sub-block, the portion of the template of the current block includes: a third sample determined based on the width of the top side of the second sub-block, and a fourth sample located to the left of the third sample.

[0126] In an example, the current block is partitioned into a plurality of sub-blocks including a first sub-block and a second sub-block based on one of a plurality of candidate partition schemes, wherein one of the plurality of candidate partition schemes is indicated by an index included in the encoded information.

[0127] In an example, a plurality of candidate prediction subblocks are determined within a search range of the second subblock. A cost value between a template of the second subblock and a template of each of the plurality of candidate prediction subblocks is determined. One or more prediction subblocks are determined from the plurality of candidate prediction subblocks, the one or more prediction subblocks corresponding to one or more minimum cost values ​​among the cost values ​​between the template of the second subblock and the templates of the plurality of candidate prediction subblocks. The second subblock is reconstructed based on the one or more prediction subblocks.

[0128] In an example, when the first subblock is located at the left side of the second subblock, the left boundary of the search range is reduced by the width of the first subblock. In an example, when the first subblock is located at the top side of the second subblock, the top boundary of the search range is reduced by the height of the first subblock.

[0129] In an example, each of the plurality of sub-blocks includes one of a rectangular shape, a square shape, and a trapezoidal shape partitioned by a geometric partitioning mode (GPM).

[0130] In an example, the current block is partitioned into N parts in a horizontal direction and into M parts in a vertical direction, so that the number of the plurality of sub-blocks is equal to N×M.

[0131] In an example, the number of samples in the first sub-block is different from the number of samples in the second sub-block.

[0132] Then, the process proceeds to (S1299) and terminates.

[0133] The process (1200) may be adjusted appropriately. One or more steps in the process (1200) may be modified and / or omitted. Additional one or more steps may be added. Any suitable implementation order may be used.

[0134] Fig.13 A flow chart outlining a process (1300) according to an aspect of the present disclosure is shown. The process (1300) may be used in a video encoder. In various aspects, the process (1300) is performed by a processing circuit system (such as a processing circuit system that performs the functions of the video encoder (103), a processing circuit system that performs the functions of the video encoder (303), etc.). In some aspects, the process (1300) is implemented in software instructions, so when the processing circuit system executes the software instructions, the processing circuit system performs the process (1300). The process starts at (S1301) and proceeds to (S1310).

[0135] At (S1310), a current block of a current picture is partitioned into a plurality of sub-blocks, wherein the current block includes a template having an L shape and including samples adjacent to a top side and a left side of the current block.

[0136] At (S1320), a predicted subblock is determined for a first subblock among the plurality of subblocks to obtain a predicted subblock of the first subblock based on the intraTMP.

[0137] At (S1330), a template of a second subblock adjacent to the first subblock among the plurality of subblocks is determined based on at least one of (i) the template of the current block and (ii) the sample of the first subblock.

[0138] At (S1340), the second sub-block is encoded based on the determined template according to the intraTMP.

[0139] In an example, the template of the second sub-block includes samples of the first sub-block located at a first side of the second sub-block and a portion of the template of the current block adjacent to a second side of the second sub-block.

[0140] In an example, the template of the second sub-block includes a portion of the samples of the first sub-block adjacent to a first side of the second sub-block and a portion of the template of the current block adjacent to a second side of the second sub-block.

[0141] Then, the process proceeds to (S1399) and terminates.

[0142] The process (1300) may be adjusted appropriately. One or more steps in the process (1300) may be modified and / or omitted. Additional one or more steps may be added. Any suitable order of implementation may be used.

[0143] In the present disclosure, a code stream of visual media data including a current picture is processed according to a format rule. For example, the code stream may be a code stream for any of the decoding / encoding methods. The format rule may specify constraints on the code stream and / or a process to be performed by a decoder and / or an encoder.

[0144] In an example, a code stream includes encoded information of a current block in a current picture and a template of the current block. The template of the current block has an L shape and includes samples adjacent to the top and left sides of the current block. The format rule stipulates that the current block is partitioned into a plurality of sub-blocks based on a candidate partitioning scheme indicated by an index signaled in the encoded information. The format rule stipulates that a reconstructed sample of a first sub-block among a plurality of sub-blocks is determined based on an intraTMP, so that a predicted sub-block is determined based on a cost value between the template of the first sub-block and a template of a predicted sub-block of the first sub-block. The format rule stipulates that a template of a second sub-block adjacent to the first sub-block among a plurality of sub-blocks is determined based on at least one of (i) the template of the current block and (ii) the reconstructed sample of the first sub-block. The template of the second sub-block includes at least a portion of the template of the current block. At least a portion of the template of the current block included in the template of the second sub-block is determined based on which side of the second sub-block is adjacent to the template of the current block. The format rule stipulates that the second sub-block is processed based on the determined template according to the intraTMP.

[0145] The above techniques may be implemented as computer software using computer-readable instructions and physically stored in at least one computer-readable medium. For example, Fig.14 A computer system (1400) suitable for implementing certain aspects of the disclosed subject matter is shown.

[0146] Computer software may be encoded using any suitable machine code or computer language that may be assembled, compiled, linked, or similar mechanisms to create code comprising instructions that may be executed by one or more computer central processing units (CPUs), graphics processing units (GPUs), etc., either directly or through interpretation, microcode execution, etc.

[0147] The instructions may be executed on various types of computers or components thereof, including, for example, personal computers, tablet computers, servers, smart phones, gaming devices, Internet of Things devices, etc.

[0148] Fig.14 The components shown for the computer system (1400) are exemplary and are not intended to suggest any limitation on the scope of use or functionality of computer software implementing aspects of the present disclosure. Nor should the configuration of components be interpreted as having any dependency or requirement on any one component or combination of components illustrated in the exemplary aspects of the computer system (1400).

[0149] The computer system (1400) may include certain human interface input devices. Such human interface input devices may be responsive to input from one or more human users through, for example, tactile input (such as keystrokes, swipes, data glove movements), audio input (such as voice, tapping), visual input (such as gestures), olfactory input (not depicted). Human interface devices may also be used to capture certain media that are not necessarily directly related to conscious human input, such as audio (such as voice, music, ambient sounds), images (such as scanned images, photographic images obtained from a still image camera), videos (such as two-dimensional video, three-dimensional video including stereoscopic video).

[0150] Input human interface devices may include one or more of the following (only one of each is depicted): keyboard (1401), mouse (1402), touchpad (1403), touch screen (1410), data gloves (not shown), joystick (1405), microphone (1406), scanner (1407), camera (1408).

[0151] The computer system (1400) may also include certain human-computer interface output devices. Such human-computer interface output devices may stimulate one or more human user senses through, for example, tactile output, sound, light, and smell / taste. Such human-computer interface output devices may include tactile output devices (e.g., tactile feedback of a touch screen (1410), a data glove (not shown), or a joystick (1405), but there may also be tactile feedback devices that are not used as input devices), audio output devices (such as: speakers (1409), headphones (not depicted)), visual output devices (such as screens (1410), including CRT screens, LCD screens, plasma screens, OLED screens, each with or without touch screen input capabilities, each with or without tactile feedback capabilities - some of which are capable of outputting two-dimensional visual output or more than three-dimensional output through means such as stereo output; virtual reality glasses (not depicted), holographic displays, and smoke canisters (not depicted)), and printers (not depicted).

[0152] The computer system (1400) may also include human-accessible storage devices and their associated media, such as optical media including media (1421) such as CD / DVD ROM / RW (1420) with CD / DVD, thumb drives (1422), removable hard drives or solid-state drives (1423), traditional magnetic media such as tapes and floppy disks (not depicted), dedicated ROM / ASIC / PLD-based devices such as security dongles (not depicted), and the like.

[0153] Those skilled in the art should also understand that the term "computer-readable media" used in connection with the presently disclosed subject matter does not include transmission media, carrier waves, or other volatile signals.

[0154] The computer system (1400) may also include an interface (1454) to one or more communication networks (1455). The network may be, for example, wireless, wired, optical. The network may further be local, wide area, metropolitan, vehicular and industrial, real-time, delay tolerant, etc. Examples of networks include local area networks such as Ethernet, wireless LANs, cellular networks including GSM, 3G, 4G, 5G, LTE, etc., TV wired or wireless wide area digital networks including cable TV, satellite TV, and terrestrial broadcast TV, vehicular and industrial networks including CANBus, etc. Some networks typically require an external network interface adapter attached to some common data port or peripheral bus (1449) (such as, for example, the USB port (1400) of the computer system); other systems are typically integrated into the core of the computer system (1400) by attaching to a system bus as described below (e.g., Ethernet interface to a PC computer system or cellular network interface to a smartphone computer system). Using any of these networks, the computer system (1400) can communicate with other entities. Such communications may be one-way receive-only (e.g., broadcast TV), one-way send-only (e.g., CANbus to certain CANbus devices), or two-way, such as to other computer systems using a local area digital network or a wide area digital network. Certain protocols and protocol stacks may be used on each of those networks and network interfaces as described above.

[0155] The above-mentioned human interface devices, human-accessible storage devices, and network interfaces may be attached to the core (1440) of the computer system (1400).

[0156] The core (1440) may include one or more central processing units (CPUs) (1441), graphics processing units (GPUs) (1442), dedicated programmable processing units in the form of field programmable gate arrays (FPGAs) (1443), hardware accelerators (1444) for certain tasks, graphics adapters (1450), etc. These devices, along with read-only memory (ROM) (1445), random access memory (1446), internal mass storage (1447) such as internal non-user accessible hard drives, SSDs, etc., may be connected via a system bus (1448). In some computer systems, the system bus (1448) may be accessible in the form of one or more physical plugs to enable expansion by additional CPUs, GPUs, etc. Peripheral devices may be attached to the core's system bus (1448) directly or via a peripheral bus (1449). In an example, a screen (1410) may be connected to a graphics adapter (1450). The architecture of the peripheral bus includes PCI, USB, etc.

[0157] The CPU (1441), GPU (1442), FPGA (1443) and accelerator (1444) can execute certain instructions, and the combination of these instructions can constitute the above-mentioned computer code. The computer code can be stored in ROM (1445) or RAM (1446). Transition data can also be stored in RAM (1446), while permanent data can be stored in, for example, an internal mass storage device (1447). Fast storage and retrieval of any memory device can be enabled by using cache memory, which can be closely associated with one or more CPUs (1441), GPUs (1442), mass storage devices (1447), ROM (1445), RAM (1446), etc.

[0158] The computer readable medium may have computer code thereon for performing various computer-implemented operations. The medium and computer code may be those specially designed and constructed for the purposes of the present disclosure, or they may be of a type well known and available to those skilled in the art of computer software.

[0159] As an example and not limitation, a computer system (1400) having an architecture, and in particular a core (1440), can provide functionality as a result of one or more processors (including CPUs, GPUs, FPGAs, accelerators, etc.) executing software embodied in one or more tangible computer-readable media. Such computer-readable media can be media associated with a user-accessible mass storage as described above, as well as certain storage devices of the core (1440) having non-volatile properties, such as a core internal mass storage device (1447) or ROM (1445). Software implementing various aspects of the present disclosure can be stored in such devices and executed by the core (1440). Depending on specific needs, the computer-readable medium may include one or more memory devices or chips. The software can cause the core (1440) and in particular the processors therein (including CPUs, GPUs, FPGAs, etc.) to perform specific processes or specific parts of specific processes described herein, including defining data structures stored in RAM (1446) and modifying such data structures according to processes defined by the software. Additionally or alternatively, the computer system may provide functionality as a result of hard-wiring logic or otherwise embodied in circuitry (e.g., accelerator (1444)) that may replace software or operate in conjunction with software to perform specific processes or specific portions of specific processes described herein. Where appropriate, references to software may include logic and vice versa. Where appropriate, references to computer-readable media may include circuitry (such as an integrated circuit (IC)) storing software for execution, circuitry embodying logic for execution, or both. The present disclosure includes any suitable combination of hardware and software.

[0160] As used in this disclosure, "at least one" or "an" is intended to include any one or combination of the elements described. For example, reference to at least one of A, B, or C; at least one of A, B, and C; at least one of A, B, and / or C; and at least one of A through C is intended to include only A, only B, only C, or any combination thereof. Reference to one of A or B and one of A and B is intended to include A or B or (A and B). Where applicable, the use of "one of" does not exclude any combination of the elements described, such as when the elements are not mutually exclusive.

[0161] Although the present disclosure has described several exemplary aspects, there are changes, permutations, and various substitute equivalents that fall within the scope of the present disclosure. It will therefore be appreciated that those skilled in the art will be able to design many systems and methods that, although not explicitly shown or described herein, embody the principles of the present disclosure and are therefore within the spirit and scope of the present disclosure.

Claims

1. A method, characterized in that include: Processing a code stream of visual media data including the current picture according to the format rules, wherein: The code stream includes encoded information of a current block in the current picture and a template of the current block, wherein the template of the current block has an L shape and includes samples adjacent to a top side and a left side of the current block; and The formatting rules state: partitioning the current block into a plurality of sub-blocks based on a candidate partitioning scheme indicated by an index signaled in the encoded information, determining a reconstructed sample of a first sub-block among the plurality of sub-blocks based on intra-frame template matching prediction (intraTMP), so that the predicted sub-block is determined based on a cost value between a template of the first sub-block and a template of a predicted sub-block of the first sub-block, determining a template of a second subblock adjacent to the first subblock among the plurality of subblocks based on at least one of (i) the template of the current block and (ii) the reconstructed sample of the first subblock, wherein the template of the second subblock includes at least a portion of the template of the current block, and the at least a portion of the template of the current block included in the template of the second subblock is determined based on which side of the second subblock is adjacent to the template of the current block, and According to the intra-frame template matching prediction, the second sub-block is processed based on the determined template.

2. The method according to claim 1, characterized in that The template of the second sub-block includes: the reconstructed sample of the first sub-block located at a first side of the second sub-block and a portion of the template of the current block adjacent to a second side of the second sub-block, or a portion of the reconstructed sample of the first sub-block adjacent to the first side of the second sub-block and a portion of the template of the current block adjacent to the second side of the second sub-block.

3. The method according to claim 1, characterized in that The template of the second sub-block includes a portion of the template of the current block adjacent to a first side of the second sub-block, and a width of the portion of the template of the current block is equal to a width of the first side of the second sub-block.

4. A video encoding method, characterized in that: The method comprises: Partitioning a current block of a current picture into a plurality of sub-blocks, the current block including a template having an L shape and including samples adjacent to a top side and a left side of the current block; Determine a predicted subblock for a first subblock of the plurality of subblocks to obtain a predicted subblock of the first subblock based on intra template matching prediction (intraTMP); determining a template of a second subblock of the plurality of subblocks adjacent to the first subblock based on at least one of (i) the template of the current block and (ii) a sample of the first subblock; and According to the intra-frame template matching prediction, the second sub-block is encoded based on the determined template.

5. The method according to claim 4, characterized in that The template of the second sub-block includes: the sample of the first sub-block located at a first side of the second sub-block and a portion of the template of the current block adjacent to a second side of the second sub-block, or A portion of the samples of the first sub-block adjacent to the first side of the second sub-block and a portion of the template of the current block adjacent to the second side of the second sub-block.

6. A video decoding device, characterized in that: include: processing circuitry configured to: Receiving a video code stream including encoded information of a current block in a current picture and a template of the current block, wherein the template of the current block includes samples adjacent to the current block, and the current block includes a first subblock and a second subblock adjacent to the first subblock; determining a template of the second sub-block based on at least one of (i) the template of the current block and (ii) a reconstructed sample of the first sub-block; as well as According to intra template matching prediction (intraTMP), the second sub-block is reconstructed based on the determined template.

7. The device according to claim 6, characterized in that The template of the second sub-block includes: the reconstructed sample of the first sub-block located at a first side of the second sub-block, and a portion of the template of the current block adjacent to a second side of the second sub-block, or A portion of the reconstructed sample of the first sub-block adjacent to the first side of the second sub-block, and a portion of the template of the current block adjacent to the second side of the second sub-block.

8. The device according to claim 6, characterized in that The template of the second sub-block includes: a portion of the template of the current block adjacent to a first side of the second sub-block, a width of the portion of the template of the current block being equal to a width of the first side of the second sub-block.

9. The device according to claim 8, characterized in that: When the first side of the second sub-block is a left side of the second sub-block, the portion of the template of the current block includes: a first sample determined based on a width of the left side of the second sub-block, and a second sample located above the first sample, and When the first side of the second subblock is a top side of the second subblock, the portion of the template of the current block includes a third sample determined based on a width of the top side of the second subblock and a fourth sample located to the left of the third sample.

10. The device according to any one of claims 6 to 9, characterized in that The processing circuit system is configured to: Partitioning the current block into a plurality of sub-blocks including the first sub-block and the second sub-block based on one of a plurality of candidate partition schemes indicated by an index included in the encoded information.

11. The device according to any one of claims 6 to 10, characterized in that The processing circuit system is configured to: Determine a plurality of candidate prediction sub-blocks within a search range of the second sub-block; determining a cost value between the template of the second sub-block and a template of each of the plurality of candidate prediction sub-blocks; determining one or more prediction sub-blocks from the plurality of candidate prediction sub-blocks, the one or more prediction sub-blocks corresponding to one or more minimum cost values ​​among the cost values ​​between the template of the second sub-block and the templates of the plurality of candidate prediction sub-blocks; as well as The second sub-block is reconstructed based on the one or more predicted sub-blocks.

12. The device according to claim 11, characterized in that: When the first sub-block is located at the left side of the second sub-block, the left boundary of the search range is reduced by the width of the first sub-block, and When the first sub-block is located at a top side of the second sub-block, a top boundary of the search range is reduced by a height of the first sub-block.

13. The device according to claim 10, characterized in that Each of the plurality of sub-blocks includes one of a rectangular shape, a square shape, and a trapezoidal shape partitioned by a geometric partitioning mode (GPM).

14. The device according to claim 10 or 13, characterized in that The processing circuit system is configured to: The current block is partitioned into N parts in a horizontal direction and into M parts in a vertical direction, so that the number of the plurality of sub-blocks is equal to N×M.

15. The device according to any one of claims 6 to 14, characterized in that The number of samples in the first sub-block is different from the number of samples in the second sub-block.

Images