Video encoding and decoding method and device

Abstract

The present application provides a method and apparatus for video encoding and decoding. In a decoding method, encoded information about a coding unit (CU), a template region, and multiple reference regions is received. The encoded information includes a first syntax element indicating whether the CU is predicted according to a multiple reference line (MRL) intra-frame prediction mode based on template matching. In response to the first syntax element indicating that the CU is predicted according to the MRL intra-frame prediction mode based on template matching, multiple cost values ​​are determined between (i) each prediction sample of the template region determined based on samples in each reference region of the multiple reference regions and (ii) the reconstructed samples of the template region corresponding to the corresponding prediction samples. A reference region is determined from the multiple reference regions based on the multiple cost values. Samples of the CU are reconstructed based on the samples in the determined reference region.

Description

[0001] Incorporation by reference

[0002] This application claims priority to U.S. Patent Application No. 17 / 727,570, “Template Matching Based Intra Prediction,” filed on April 22, 2022, which claims priority to U.S. Provisional Application No. 63 / 179,891, “Template Matching Based Intra Prediction,” filed on April 26, 2021. The entire disclosure of the prior application is incorporated herein by reference. Technical Field

[0003] The embodiments described herein generally relate to video encoding and decoding. Background Art

[0004] The background description provided herein is intended to present the context of this application. The work of the named inventors, the work described in this background section, and the content within the scope of the embodiments of this specification may not be considered prior art at the time of filing and are not explicitly or implicitly admitted as prior art against this application.

[0005] Video encoding and decoding can be performed using inter-image prediction with motion compensation. Uncompressed digital video may consist of a sequence of images, each of which has a certain spatial dimension, for example, 1920x1080 luma samples and associated chroma samples. The image sequence may have a fixed or variable image rate (commonly known as frame rate), for example, 60 images per second or 60Hz. Uncompressed video requires a specific bit rate. For example, 1080p604:2:0 (a resolution of 1920x1080 luma samples at a frame rate of 60Hz) with 8 bits per sample requires a bandwidth of nearly 1.5Gbits / second. One hour of such video would require more than 600Gbytes of storage space.

[0006] One of the goals of video encoding and decoding is to reduce the redundancy of the input video signal through compression. In some cases, compression can reduce the bandwidth and / or memory requirements by at least two orders of magnitude. Lossless compression, lossy compression, or a combination thereof can be used. Lossless compression refers to a technique that can reconstruct an exact copy of the original signal from the compressed original signal. When lossy compression is used, the reconstructed signal may not be consistent with the original signal, but the distortion between the original signal and the reconstructed signal is small enough to enable the reconstructed signal to achieve the desired purpose. Lossy compression is widely used in the video field. The amount of distortion allowed depends on the application. For example, users of some consumer live broadcast applications can tolerate greater distortion than users of TV program applications. The achievable compression ratio can reflect that the greater the allowable / tolerable distortion, the higher the compression ratio that can be produced.

[0007] Video encoders and decoders may utilize several broad categories of techniques including, for example, motion compensation, transforms, quantization, and entropy coding.

[0008] Video coding techniques may include techniques known as intra-frame coding. In intra-frame coding, sample values ​​are represented without reference to samples or other data in a previously reconstructed reference image. In some video codecs, an image is spatially subdivided into blocks of samples. When all sample blocks are encoded in intra-frame mode, the image may be an intra-frame image. Intra-frame images and their derivatives (such as independent decoder refresh images) can be used to reset the decoder state and can thus be used as the first image in an encoded video stream and video session, or as a still image. The samples of an intra-frame block may be transformed, and the transform coefficients may be quantized before entropy coding. Intra-frame prediction may be a technique for minimizing sample values ​​in the pre-transform domain. In some cases, the smaller the DC value after transformation, the smaller the AC coefficient, and the fewer bits are required to represent the block with a given quantization step size after entropy coding.

[0009] Conventional intra-frame coding (e.g., as known from coding techniques such as MPEG-2) does not use intra-frame prediction. However, some newer video compression techniques include techniques that attempt to, for example, obtain surrounding sample data and / or metadata during the encoding and / or decoding of spatially adjacent and preceding blocks of data in decoding order. Such techniques are henceforth referred to as "intra-frame prediction" techniques. Note that, in at least some cases, intra-frame prediction uses reference data only from the current image being reconstructed (rather than a reference image).

[0010] There are many different forms of intra-frame prediction. When more than one such technique can be used in a given video coding technique, the technique used can be encoded in the intra-frame prediction mode. In some cases, a mode can have sub-modes and / or parameters, which can be encoded separately or included in the mode codeword. Which codeword is used for a given mode, sub-mode, and / or parameter combination can have an impact on the coding efficiency gain through intra-frame prediction, as can the entropy coding technique used to convert the codeword into the bitstream.

[0011] Certain intra-frame prediction modes were introduced with H.264, refined in H.265, and further refined in new coding techniques such as the Joint Exploration Model (JEM), Versatile Video Coding (VVC), and the Baseline Video Set (BMS). A prediction block can be formed using values ​​from neighboring samples, where the values ​​belong to already available samples. Sample values ​​from these neighboring samples are copied into the prediction block based on the direction. The direction used can be encoded in the bitstream or predicted.

[0012] See also Figure 1 , depicted in the lower right corner are a subset of nine known prediction directions from the 33 possible prediction directions for H.265 (corresponding to the 33 angular modes for the 35 intra modes). The point (101) where the arrows converge represents the sample being predicted. The arrows represent the direction of the sample being predicted. For example, arrow (102) indicates that the prediction direction for sample (101) is from one or more samples to the upper right corner, at a 45 degree angle to the horizontal. Similarly, arrow (103) indicates that the prediction direction for sample (101) is from one or more samples to the lower left of sample (101), at a 22.5 degree angle to the horizontal.

[0013] Still refer to Figure 1 , a square block (104) of 4×4 samples is shown in the upper left (indicated by the bold dashed line). The square block (104) includes 16 samples, each sample labeled with "S" to indicate its position in the Y dimension (e.g., row index) and its position in the X dimension (e.g., column index). For example, sample S21 is the second sample in the Y dimension (from the top) and the first sample in the X dimension (from the left). Similarly, sample S44 in block (104) is the fourth sample in both the Y and X dimensions. Since the size of the block is 4×4 samples, S44 is located in the lower right corner. Reference samples are also shown that follow a similar numbering scheme. The reference samples are labeled with R and their Y position (e.g., row index) and X position (column index) relative to the block (104). In H.264 and H.265, the prediction samples are adjacent to the block being reconstructed; therefore, there is no need to use negative values.

[0014] Intra-frame prediction works by copying reference sample values ​​from neighboring samples covered by a signaled prediction direction. For example, assume that the coded video bitstream includes signaling indicating that the prediction direction for the block is consistent with arrow (102)—that is, samples are predicted from one or more prediction samples to the upper right corner, at a 45-degree angle to the horizontal. In this case, samples S41, S32, S23, and S14 are predicted using the same reference sample R05. Sample S44 is then predicted using reference sample R08.

[0015] In some cases, the values ​​of multiple reference samples may be combined, for example by interpolation, to calculate the reference sample; in particular, when the direction is not divisible by 45 degrees.

[0016] As video coding technology has developed, the number of possible directions has increased. In H.264 (2003), nine different directions could be represented. This number was increased to 33 in H.265 (2013), and JEM / VVC / BMS can support up to 65 directions at the time of publication. Some experiments have been done to identify the most likely directions, and use certain techniques in entropy coding to represent those possible directions with a small number of bits, while taking the disadvantages of less likely directions. In addition, these directions themselves can sometimes be predicted from adjacent directions used by adjacent decoded blocks.

[0017] Figure 2 A schematic diagram (201) of JEM's 65 intra prediction directions is shown to illustrate the increasing number of prediction directions over time.

[0018] The mapping method of the intra prediction direction bits representing directions in the coded video bitstream can vary between different video coding techniques; it can range from, for example, a simple direct mapping of prediction directions to intra prediction modes or to codewords, to complex adaptive schemes involving most possible modes, and similar techniques. However, in all cases, there are likely to be some directions that are statistically less likely to occur in the video content than others. Since the goal of video compression is to reduce redundancy, in a well-performing video coding technique, less likely directions will be represented by more bits than more likely directions. Summary of the Invention

[0019] Various aspects of the present application provide methods and apparatus for video encoding / decoding. In some examples, a video decoding apparatus includes a receiving circuit and a processing circuit.

[0020] According to one aspect of the present application, a decoding method performed in a decoder is provided. In this method, encoded information about a coding unit (CU), a template region, and multiple reference regions can be received from an encoded video code stream. The encoded information includes a first syntax element, which indicates whether the CU is predicted according to a multiple reference line (MRL) intra-frame prediction mode based on template matching. The template region can be adjacent to the CU, and the multiple reference regions can be adjacent to the template region. In response to the first syntax element indicating that the CU is predicted according to the MRL intra-frame prediction mode based on template matching, multiple cost values ​​can be determined between (i) each predicted sample of the template region determined based on samples in each reference region of the multiple reference regions and (ii) the reconstructed samples of the template region corresponding to the corresponding predicted samples. A reference region can be determined from the multiple reference regions based on the multiple cost values. The samples of the CU can be reconstructed based on the samples in the determined reference regions.

[0021] In some embodiments, each of the plurality of cost values ​​is determined based on a difference between (i) a respective predicted sample of the template region determined based on a sample in a corresponding one of the plurality of reference regions and (ii) the reconstructed sample of the template region to which the respective predicted sample corresponds.

[0022] In some embodiments, the template region may further include one or a combination of: (i) top samples in a row above the top edge of the CU and (ii) side samples in a column adjacent to the left edge of the CU.

[0023] In some embodiments, the template region may further include a seam sample located above the side sample and adjacent to the top sample.

[0024] In some embodiments, the multiple reference areas may include: (i) a first reference area including a row segment located above the top edge of the template area and a column segment adjacent to the left edge of the template area; (ii) a second reference area including a row segment above the row segment of the first reference area and a column segment adjacent to the column segment of the first reference area; and (iii) a third reference area including a row segment above the row segment of the second reference area and a column segment adjacent to the column segment of the second reference area.

[0025] In some embodiments, the template area may further include one or a combination of the following: (i) top samples in a first row above the top edge of the CU and top samples in a second row above the first row, and (ii) side samples in a first column adjacent to the left edge of the CU and side samples in a second column next to the first column.

[0026] In some embodiments, the template region may further include a junction sample located above the side samples of the first and second columns and to the left of the top samples of the first and second rows.

[0027] In some embodiments, the multiple reference areas may include: a first reference area, which includes a row segment located above the top edge of the template area and a column segment adjacent to the left edge of the template area; and a second reference area, which includes a row segment located above the row segment of the first reference area and a column segment adjacent to the column segment of the first reference area.

[0028] In the method, a second syntax element may be further decoded from the encoded information, the second syntax element indicating whether the CU is intra-predicted according to a template-based intra mode derivation (TIMD) mode, the TIMD mode including a set of candidate intra prediction modes; in response to the first syntax element indicating that the CU is predicted according to the template matching-based MRL intra prediction mode and the second syntax element indicating that the CU is intra-predicted according to the TIMD mode, the corresponding prediction sample of the template area is determined based on the following information: (i) the sample located in a corresponding one of the multiple reference areas, and (ii) ) a corresponding candidate intra-frame prediction mode from the set of candidate intra-frame prediction modes; determining the multiple cost values, each cost value of the multiple cost values ​​being determined based on the sum of absolute transform differences between respective prediction samples of the template area and the reconstructed samples of the template area corresponding to the corresponding prediction samples; determining a pair of a reference area from the multiple reference areas and an intra-frame prediction mode from the set of candidate intra-frame prediction modes, the pair of reference area and intra-frame prediction mode being associated with a lowest cost value from the multiple cost values; and reconstructing the samples of the CU based on the samples in the determined pair of the reference area and the intra-frame prediction mode.

[0029] In some embodiments, in response to a second syntax element indicating that the CU is not intra-predicted according to the TIMD mode, a syntax element associated with another intra-coding mode is decoded, wherein the other intra-coding mode may include one of matrix-based intra prediction (MIP) and most probable mode (MPM).

[0030] According to another aspect of the present application, a device is provided. The device includes a processing circuit. The processing circuit is configured to execute the video encoding method of the present application.

[0031] Aspects of the present application also provide a non-transitory computer-readable medium storing instructions, which, when executed by a computer for video decoding, causes the computer to perform a method for video decoding. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] More features, properties and various advantages of the subject matter of the present application will become more apparent from the following detailed description and accompanying drawings, in which:

[0033] Figure 1 is a diagram illustrating an exemplary subset of intra prediction modes.

[0034] Figure 2 is a schematic diagram of exemplary intra prediction directions.

[0035] Figure 3 is a schematic diagram of a simplified block diagram of a communication system of an embodiment.

[0036] Figure 4 is a schematic diagram of a simplified block diagram of a communication system according to another embodiment.

[0037] Figure 5 is a schematic diagram of a simplified block diagram of a decoder of an embodiment.

[0038] Figure 6 is a schematic diagram of a simplified block diagram of an encoder of an embodiment.

[0039] Figure 7 A block diagram of an encoder according to another embodiment is shown.

[0040] Figure 8 A block diagram of a decoder according to another embodiment is shown.

[0041] Figure 9 FIG. 1 is a schematic diagram of multiple reference line (MRL) intra prediction according to an embodiment.

[0042] Figure 10 is a schematic diagram of template-based intra mode derivation (TIMD) according to an embodiment.

[0043] Figure 11A A first exemplary template of a template matching based MRL according to one embodiment is shown.

[0044] Figure 11B A second exemplary template of a template matching based MRL according to one embodiment is shown.

[0045] Figure 11C A third exemplary template of a template matching based MRL according to one embodiment is shown.

[0046] Figure 11D A fourth exemplary template of a template matching based MRL according to one embodiment is shown.

[0047] Figure 12 A flowchart illustrating an overview of an exemplary decoding process according to some embodiments of the present application is shown.

[0048] Figure 13 A flowchart illustrating an overview of an exemplary encoding process according to some embodiments of the present application is shown.

[0049] Figure 14 is a schematic diagram of a computer system according to an embodiment. DETAILED DESCRIPTION

[0050] Figure 3 3 is a simplified block diagram of a communication system (300) according to an embodiment disclosed herein. The communication system (300) includes a plurality of terminal devices that can communicate with each other via, for example, a network (350). For example, the communication system (300) includes a first terminal device (310) and a second terminal device (320) interconnected via the network (350). Figure 3 In an embodiment, the first terminal device (310) and the second terminal device (320) perform unidirectional data transmission. For example, the first terminal device (310) may encode video data (e.g., a video image stream captured by the terminal device (310)) for transmission to another second terminal device (320) via the network (350). The encoded video data is transmitted in the form of one or more encoded video streams. The second terminal device (320) may receive the encoded video data from the network (350), decode the encoded video data to restore the video data, and display the video image based on the restored video data. Unidirectional data transmission is more common in applications such as media services.

[0051] In another embodiment, a communication system (300) includes a third terminal device (330) and a fourth terminal device (340) that perform bidirectional transmission of encoded video data, which can occur, for example, during a video conference. For bidirectional data transmission, each of the third terminal device (330) and the fourth terminal device (340) can encode video data (e.g., a video image stream captured by the terminal device) for transmission to the other of the third terminal device (330) and the fourth terminal device (340) via a network (350). Each of the third terminal device (330) and the fourth terminal device (340) can also receive the encoded video data transmitted by the other of the third terminal device (330) and the fourth terminal device (340), and can decode the encoded video data to restore the video data, and can display the video image on an accessible display device based on the restored video data.

[0052] exist Figure 2 In an embodiment of the present invention, the first terminal device (310), the second terminal device (320), the third terminal device (330) and the fourth terminal device (340) may be servers, personal computers and smart phones, but the principles disclosed in this application may not be limited thereto. The embodiments disclosed in this application are applicable to laptop computers, tablet computers, media players and / or dedicated video conferencing equipment. The network (350) represents any number of networks that transmit encoded video data between the first terminal device (310), the second terminal device (320), the third terminal device (330) and the fourth terminal device (340), including, for example, wired (wired) and / or wireless communication networks. The communication network (350) can exchange data in circuit switching and / or packet switching channels. The network may include a telecommunications network, a local area network, a wide area network and / or the Internet. For the purposes of this discussion application, unless explained below, the architecture and topology of the network (350) may be irrelevant to the operations disclosed in this application.

[0053] As an example, Figure 4 The video encoder and video decoder are shown in a streaming environment. The subject matter disclosed in this application is equally applicable to other video-enabled applications, including, for example, video conferencing, digital TV, storing compressed video on digital media including CDs, DVDs, memory sticks, etc.

[0054] The streaming system may include an acquisition subsystem (413), which may include a video source (401) such as a digital camera, which creates an uncompressed video image stream (402). In an embodiment, the video image stream (402) includes samples captured by the digital camera. The video image stream (402) is depicted as a thick line to emphasize the high data volume of the video image stream compared to the encoded video data (404) (or encoded video code stream), and the video image stream (402) can be processed by an electronic device (420), which includes a video encoder (403) coupled to the video source (401). The video encoder (403) may include hardware, software, or a combination of hardware and software to implement or implement various aspects of the disclosed subject matter as described in more detail below. Compared to the video image stream (402), the encoded video data (404) (or the encoded video code stream (404)) is depicted as a thin line to emphasize the lower amount of data of the encoded video data (404) (or the encoded video code stream (404)), which can be stored on the streaming server (405) for future use. One or more streaming client subsystems, such as Figure 4 The client subsystem (406) and the client subsystem (408) in the streaming server (405) can access the streaming server (405) to retrieve the copy (407) and the copy (409) of the encoded video data (404). The client subsystem (406) can include, for example, a video decoder (410) in the electronic device (430). The video decoder (410) decodes the incoming copy (407) of the encoded video data and produces an output video image stream (411) that can be presented on a display (412) (e.g., a display screen) or another presentation device (not depicted). In some streaming systems, the encoded video data (404), the video data (407), and the video data (409) (e.g., a video bitstream) can be encoded according to certain video encoding / compression standards. Examples of such standards include ITU-T H.265. In an embodiment, the video coding standard under development is informally referred to as Versatile Video Coding (VVC), and the present application can be used in the context of the VVC standard.

[0055] It should be noted that the electronic device (420) and the electronic device (430) may include other components (not shown). For example, the electronic device (420) may include a video decoder (not shown), and the electronic device (430) may also include a video encoder (not shown).

[0056] Figure 51 is a block diagram of a video decoder (510) according to an embodiment disclosed in the present application. The video decoder (510) may be provided in an electronic device (530). The electronic device (530) may include a receiver (531) (e.g., a receiving circuit). The video decoder (510) may be used to replace Figure 3 A video decoder (410) of an embodiment.

[0057] A receiver (531) may receive one or more encoded video sequences to be decoded by a video decoder (510); in the same or another embodiment, one encoded video sequence is received at a time, wherein each encoded video sequence is decoded independently of the other encoded video sequences. The encoded video sequence may be received from a channel (501), which may be a hardware / software link to a storage device storing the encoded video data. The receiver (531) may receive the encoded video data as well as other data, such as encoded audio data and / or auxiliary data streams, which may be forwarded to their respective consuming entities (not shown). The receiver (531) may separate the encoded video sequence from the other data. To prevent network jitter, a buffer memory (515) may be coupled between the receiver (531) and the entropy decoder / parser (520) (hereinafter referred to as "parser (520)"). In some applications, the buffer memory (515) is part of the video decoder (510). In other cases, the buffer memory (515) may be provided external to the video decoder (510) (not shown). In other cases, a buffer memory (not shown) is provided external to the video decoder (510) to, for example, mitigate network jitter, and another buffer memory (515) may be provided internally to the video decoder (510) to, for example, handle broadcast timing. Furthermore, when the receiver (531) receives data from a store / forward device with sufficient bandwidth and controllability or from an isochronous network, the buffer memory (515) may not be required, or the buffer memory may be smaller. Of course, for use on a traffic packet network such as the Internet, the buffer memory (515) may also be required. The buffer memory may be relatively large and have an adaptive size, and may be at least partially implemented in an operating system or similar component (not shown) external to the video decoder (510).

[0058] The video decoder (510) may include a parser (520) for reconstructing symbols (521) from the encoded video sequence. The types of symbols include information for managing the operation of the video decoder (510) and potential information for controlling a display device such as a display device (512) (e.g., a display screen) that is not part of the electronic device (530) but can be coupled to the electronic device (530), such as Figure 5As shown in . The control information for the display device may be a parameter set fragment (not shown) of Supplemental Enhancement Information (SEI message) or Video Usability Information (VUI). The parser (520) may parse / entropy decode the received coded video sequence. The coding of the coded video sequence may be performed according to a video coding technique or standard and may follow various principles, including variable length coding, Huffman coding, arithmetic coding with or without context sensitivity, and the like. The parser (520) may extract a subgroup parameter set for at least one subgroup of the subgroups of pixels in the video decoder from the coded video sequence based on at least one parameter corresponding to the group. The subgroup may include a Group of Pictures (GOP), a picture, a tile, a slice, a macroblock, a Coding Unit (CU), a block, a Transform Unit (TU), a Prediction Unit (PU), and the like. The parser (420) may also extract information from the encoded video sequence, such as transform coefficients, quantizer parameter values, motion vectors, and so on.

[0059] The parser (520) may perform entropy decoding / parsing operations on the video sequence received from the buffer memory (515), thereby creating symbols (521).

[0060] Depending on the type of coded video picture or portion of a coded video picture (e.g., inter-frame and intra-frame pictures, inter-frame blocks and intra-frame blocks) and other factors, the reconstruction of the symbol (521) may involve multiple different units. Which units are involved and how they are involved can be controlled by subgroup control information parsed from the coded video sequence by the parser (520). For the sake of brevity, the flow of such subgroup control information between the parser (520) and the various units described below is not described.

[0061] In addition to the functional blocks already mentioned, the video decoder (510) can be conceptually broken down into several functional units as described below. In practical embodiments operating under commercial constraints, many of these units interact closely with each other and may be integrated with each other. However, for the purposes of describing the disclosed subject matter, the conceptual breakdown into the following functional units is appropriate.

[0062] The first unit is a scaler / inverse transformer (551). The scaler / inverse transformer (551) receives quantized transform coefficients as symbols (521) from the parser (520) along with control information, including which transform method to use, block size, quantization factor, quantization scaling matrix, etc. The scaler / inverse transformer (551) can output a block of sample values, which can be input to the aggregator (555).

[0063] In some cases, the output samples of the scaler / inverse transformer (551) may belong to an intra-coded block; that is, a block that does not use predictive information from a previously reconstructed image, but may use predictive information from a previously reconstructed portion of the current image. Such predictive information may be provided by an intra-image predictor (552). In some cases, the intra-image predictor (552) uses reconstructed information extracted from the current image buffer (558) to generate surrounding blocks of the same size and shape as the block being reconstructed. For example, the current image buffer (558) buffers a partially reconstructed current image and / or a fully reconstructed current image. In some cases, the aggregator (555) adds the prediction information generated by the intra-prediction unit (552) to the output sample information provided by the scaler / inverse transformer (551) on a per-sample basis.

[0064] In other cases, the output samples of the scaler / inverse transformer (551) may belong to an inter-frame coded and potentially motion compensated block. In this case, the motion compensated predictor (553) may access the reference picture memory (557) to extract samples for prediction. After the extracted samples are motion compensated according to the symbols (521), these samples may be added to the output of the scaler / inverse transformer (551) (in this case referred to as residual samples or residual signal) by the aggregator (555) to generate output sample information. The motion compensated predictor (553) may be controlled by the motion vector used by the motion compensated predictor (553) to obtain the predicted samples from the address in the reference picture memory (557), and the motion vector is provided to the motion compensated predictor (553) in the form of the symbols (521), for example, including X, Y and reference picture components. Motion compensation may also include interpolation of sample values ​​extracted from the reference picture memory (557) when using sub-sample accurate motion vectors, motion vector prediction mechanisms, etc.

[0065] The output samples of the aggregator (555) may be used in a loop filter (556) by various loop filtering techniques. The video compression techniques may include in-loop filtering techniques that are controlled by parameters included in the coded video sequence (also referred to as the coded video bitstream) and made available to the loop filter (556) as symbols (521) from the parser (520). However, in other embodiments, the video compression techniques may also be responsive to meta-information obtained during decoding of a previous (in decoding order) portion of the coded image or coded video sequence, as well as to previously reconstructed and loop-filtered sample values.

[0066] The output of the loop filter (556) may be a sample stream that may be output to a display device (512) and stored in a reference picture memory (557) for subsequent inter-picture prediction.

[0067] Once fully reconstructed, certain coded pictures can be used as reference pictures for future prediction. For example, once the coded picture corresponding to the current picture is fully reconstructed and the coded picture is identified as a reference picture (e.g., by the parser (520)), the current picture buffer (558) can become part of the reference picture memory (557) and a new current picture buffer can be reallocated before starting to reconstruct a subsequent coded picture.

[0068] The video decoder (510) may perform decoding operations according to a predetermined video compression technique, such as that in the ITU-T H.265 standard. A coded video sequence may conform to the syntax specified by the video compression technique or standard used, in the sense that the coded video sequence follows the syntax of the video compression technique or standard and a profile documented in the video compression technique or standard. Specifically, the profile may select certain tools from all the tools available in the video compression technique or standard as the only tools available for use under the profile. Compliance also requires that the complexity of the coded video sequence be within the limits defined by the hierarchy of the video compression technique or standard. In some cases, the hierarchy limits the maximum picture size, maximum frame rate, maximum reconstruction sampling rate (measured in, for example, megasamples per second), maximum reference picture size, etc. In some cases, the limits set by the hierarchy may be further defined by the Hypothetical Reference Decoder (HRD) specification and metadata about the HRD buffer management signaled in the coded video sequence.

[0069] In an embodiment, a receiver (531) may receive additional (redundant) data along with the encoded video. The additional data may be part of the encoded video sequence. The additional data may be used by the video decoder (510) to properly decode the data and / or more accurately reconstruct the original video data. The additional data may be in the form of, for example, temporal, spatial, or signal-to-noise ratio (SNR) enhancement layers, redundant slices, redundant pictures, forward error correction codes, and the like.

[0070] Figure 6 1 is a block diagram of a video encoder (603) according to an embodiment disclosed in the present application. The video encoder (603) is provided in an electronic device (620). The electronic device (620) includes a transmitter (640) (e.g., a transmission circuit). The video encoder (603) can be used to replace Figure 4 A video encoder (403) in an embodiment.

[0071] The video encoder (603) can be used to generate a video from a video source (601) (not Figure 5 In another embodiment, the video source (601) is a part of the electronic device (620) to receive video samples, and the video source can capture video images to be encoded by the video encoder (603). In another embodiment, the video source (601) is a part of the electronic device (620).

[0072] The video source (601) may provide a source video sequence in the form of a stream of digital video samples to be encoded by the video encoder (603), wherein the stream of digital video samples may have any suitable bit depth (e.g., 8-bit, 10-bit, 12-bit, ...), any color space (e.g., BT.601 Y CrCB, RGB, ...), and any suitable sampling structure (e.g., Y CrCb 4:2:0, Y CrCb 4:4:4). In a media serving system, the video source (601) may be a storage device storing previously prepared videos. In a video conferencing system, the video source (601) may be a camera that captures local image information as a video sequence. The video data may be provided as a plurality of individual images that are imparted with motion when viewed sequentially. The images themselves may be constructed as a spatial array of pixels, where each pixel may include one or more samples, depending on the sampling structure, color space, etc. used. The relationship between pixels and samples may be readily understood by those skilled in the art. The following description focuses on samples.

[0073] According to an embodiment, the video encoder (603) may encode and compress the images of the source video sequence into an encoded video sequence (643) in real time or under any other time constraints required by the application. Implementing an appropriate encoding speed is a function of the controller (650). In some embodiments, the controller (650) controls other functional units as described below and is functionally coupled to these units. For the sake of simplicity, the coupling is not shown in the figure. The parameters set by the controller (650) may include rate control related parameters (image skipping, quantizer, lambda value of rate distortion optimization technology, etc.), image size, group of pictures (GOP) layout, maximum motion vector search range, etc. The controller (650) can be used to have other suitable functions that are related to the video encoder (603) optimized for a certain system design.

[0074] In some embodiments, the video encoder (603) operates in a coding loop. As a simplified description, in some embodiments, the coding loop may include a source encoder (630) (e.g., responsible for creating symbols, such as a symbol stream, based on the input image to be encoded and a reference image) and a (local) decoder (633) embedded in the video encoder (603). The decoder (633) reconstructs the symbols to create sample data in a manner similar to how the (remote) decoder creates sample data (because in the video compression technology considered in this application, any compression between the symbols and the encoded video code stream is lossless). The reconstructed sample stream (sample data) is input to a reference image memory (634). Since the decoding of the symbol stream produces bit-accurate results regardless of the decoder location (local or remote), the contents of the reference image memory (634) are also bit-accurate between the local encoder and the remote encoder. In other words, the reference image samples "seen" by the prediction portion of the encoder are exactly the same sample values ​​that the decoder will "see" when using the prediction during decoding. This basic principle of reference image synchronicity (and the resulting drift when synchronicity cannot be maintained, eg due to channel errors) is also used in some related techniques.

[0075] The operation of the "local" decoder (633) can be combined with the operation of Figure 4 The "remote" decoder described in detail for the video decoder (510) is identical. However, additional brief reference is made to Figure 5 , when symbols are available and the entropy encoder (645) and parser (520) are capable of losslessly encoding / decoding the symbols into an encoded video sequence, the entropy decoding portion of the video decoder (510), including the buffer memory (515) and the parser (520), may not be fully implemented in the local decoder (633).

[0076] At this point, it can be observed that any decoder technology other than parsing / entropy decoding present in the decoder must also be present in a substantially identical functional form in the corresponding encoder. For this reason, this application focuses on the decoder operation. The description of the encoder technology can be simplified because the encoder technology is mutually inverse to the decoder technology described comprehensively. A more detailed description is only required in certain areas and is provided below.

[0077] During operation, in some embodiments, the source encoder (630) may perform motion-compensated predictive coding. The motion-compensated predictive coding predictively encodes an input image with reference to one or more previously encoded images from a video sequence designated as "reference images." In this manner, the encoding engine (632) encodes the differences between pixel blocks of the input image and pixel blocks of a reference image that may be selected as a prediction reference for the input image.

[0078] The local video decoder (633) may decode the encoded video data of a picture that may be designated as a reference picture based on the symbols created by the source encoder (630). The operation of the encoding engine (632) may be a lossy process. When the encoded video data is available at the video decoder ( Figure 6 When decoded at a remote location (not shown), the reconstructed video sequence may typically be a copy of the source video sequence with some errors. The local video decoder (633) replicates the decoding process that the video decoder may perform on the reference picture and may cause the reconstructed reference picture to be stored in the reference picture cache (634). In this way, the video encoder (603) may locally store a copy of the reconstructed reference picture that has common content (absent transmission errors) with the reconstructed reference picture that will be obtained by the remote video decoder.

[0079] The predictor (635) may perform a prediction search for the encoding engine (632). That is, for a new image to be encoded, the predictor (635) may search the reference image memory (634) for sample data (as candidate reference pixel blocks) or certain metadata, such as reference image motion vectors, block shapes, etc., that may serve as suitable prediction references for the new image. The predictor (635) may operate on a pixel block-by-pixel block basis to find a suitable prediction reference. In some cases, based on the search results obtained by the predictor (635), it may be determined that the input image may have prediction references taken from multiple reference images stored in the reference image memory (634).

[0080] The controller (650) can manage encoding operations of the source encoder (630), including, for example, setting parameters and subgroup parameters for encoding video data.

[0081] The outputs of all the above functional units may be entropy coded in an entropy encoder (645). The entropy encoder (645) losslessly compresses the symbols generated by the various functional units using techniques such as Huffman coding, variable length coding, arithmetic coding, etc., thereby converting the symbols into a coded video sequence.

[0082] The transmitter (640) can buffer one or more encoded video sequences created by the entropy encoder (645) in preparation for transmission over a communication channel (660), which can be a hardware / software link to a storage device where the encoded video data will be stored. The transmitter (640) can combine the encoded video data from the video encoder (603) with other data to be transmitted, such as encoded audio data and / or auxiliary data streams (source not shown).

[0083] The controller (650) can manage the operation of the video encoder (603). During encoding, the controller (650) can assign a certain coded picture type to each coded picture, but this may affect the encoding techniques that can be applied to the corresponding picture. For example, a picture can generally be assigned to any of the following picture types:

[0084] An intra picture (I picture) is a picture that can be encoded and decoded without using any other picture in the sequence as a prediction source. Some video codecs allow for different types of intra pictures, including, for example, Independent Decoder Refresh (IDR) pictures. Those skilled in the art are aware of the variations of I pictures and their corresponding applications and characteristics.

[0085] A predictive picture (P picture) may be a picture that can be encoded and decoded using intra prediction or inter prediction, which uses at most one motion vector and a reference index to predict sample values ​​for each block.

[0086] Bidirectionally predictive pictures (B pictures), which can be pictures that can be encoded and decoded using intra-frame prediction or inter-frame prediction, which uses up to two motion vectors and reference indices to predict the sample values ​​of each block. Similarly, multiple predictive pictures can use more than two reference pictures and associated metadata for reconstructing a single block.

[0087] A source picture is typically spatially subdivided into blocks of samples (e.g., blocks of 4×4, 8×8, 4×8, or 16×16 samples) and coded block by block. These blocks can be predictively coded with reference to other (already coded) blocks, determined according to the coding allocation applied to the block's corresponding picture. For example, blocks of an I picture can be non-predictively coded, or they can be predictively coded (spatial prediction or intra-frame prediction) with reference to already coded blocks of the same picture. Pixel blocks of a P picture can be predictively coded using spatial prediction with reference to a previously coded reference picture or using temporal prediction. Blocks of a B picture can be predictively coded using spatial prediction with reference to one or two previously coded reference pictures or using temporal prediction.

[0088] The video encoder (603) may perform encoding operations according to a predetermined video coding technique or standard, such as ITU-T Recommendation H.265. In operation, the video encoder (603) may perform various compression operations, including predictive coding operations that exploit temporal and spatial redundancy in the input video sequence. Thus, the encoded video data may conform to the syntax specified by the video coding technique or standard used.

[0089] In an embodiment, the transmitter (640) may transmit additional data along with the encoded video when transmitting the encoded video. The source encoder (630) may include such data as part of the encoded video sequence. The additional data may include temporal / spatial / SNR enhancement layers, redundant pictures and slices, and other forms of redundant data, SEI messages, VUI parameter set fragments, and the like.

[0090] The captured video may be presented as a temporal sequence of multiple source images (video images). Intra-frame image prediction (often shortened to intra-frame prediction) exploits spatial correlations within a given image, while inter-frame image prediction exploits correlations (temporal or otherwise) between images. In an embodiment, a particular image being encoded / decoded is divided into blocks, and the particular image being encoded / decoded is referred to as the current image. When a block in the current image is similar to a reference block in a previously encoded and still buffered reference image in the video, the block in the current image may be encoded using a vector called a motion vector. The motion vector points to a reference block in a reference image, and in the case of multiple reference images, the motion vector may have a third dimension that identifies the reference image.

[0091] In some embodiments, bidirectional prediction techniques can be used for inter-frame image prediction. According to bidirectional prediction techniques, two reference images are used, for example, a first reference image and a second reference image, both of which precede the current image in the video in decoding order (but may be in the past and future, respectively, in display order). A block in the current image can be encoded using a first motion vector pointing to a first reference block in the first reference image and a second motion vector pointing to a second reference block in the second reference image. Specifically, the block can be predicted using a combination of the first reference block and the second reference block.

[0092] In addition, merge mode technology can be used in inter-frame picture prediction to improve coding efficiency.

[0093] According to some embodiments disclosed in the present application, predictions such as inter-frame image prediction and intra-frame image prediction are performed in units of blocks. For example, according to the HEVC standard, images in a video image sequence are divided into coding tree units (CTUs) for compression, and the CTUs in the image have the same size, such as 64×64 pixels, 32×32 pixels, or 16×16 pixels. Generally speaking, a CTU includes three coding tree blocks (CTBs), which are a luminance CTB and two chrominance CTBs. Furthermore, each CTU can be split into one or more coding units (CUs) using a quadtree. For example, a 64×64 pixel CTU can be split into a 64×64 pixel CU, or four 32×32 pixel CUs, or sixteen 16×16 pixel CUs. In an embodiment, each CU is analyzed to determine the prediction type used for the CU, such as an inter prediction type or an intra prediction type. In addition, depending on temporal and / or spatial predictability, the CU is split into one or more prediction units (PUs). Typically, each PU includes a luma prediction block (PB) and two chroma PBs. In an embodiment, the prediction operation in encoding (encoding / decoding) is performed in units of prediction blocks. Taking the luma prediction block as an example, the prediction block includes a matrix of pixel values ​​(e.g., luma values), such as 8×8 pixels, 16×16 pixels, 8×16 pixels, 16×8 pixels, and the like.

[0094] Figure 7 FIG2 is a diagram of a video encoder (703) according to another embodiment disclosed herein. The video encoder (703) is configured to receive a processed block (e.g., a prediction block) of sample values ​​within a current video image in a video image sequence and to encode the processed block into an encoded image that is part of an encoded video sequence. In this embodiment, the video encoder (703) is configured to replace Figure 4A video encoder (403) in an embodiment.

[0095] In an HEVC embodiment, a video encoder (703) receives a matrix of sample values ​​for a processing block, such as a prediction block of 8×8 samples. The video encoder (703) uses, for example, rate-distortion (RD) optimization to determine whether to use intra mode, inter mode, or bi-prediction mode to encode the processing block. When encoding the processing block in intra mode, the video encoder (703) may use intra prediction techniques to encode the processing block into the encoded image; and when encoding the processing block in inter mode or bi-prediction mode, the video encoder (703) may use inter prediction or bi-prediction techniques, respectively, to encode the processing block into the encoded image. In some video coding techniques, the merge mode may be an inter-picture prediction submode, in which a motion vector is derived from one or more motion vector predictors without resorting to coded motion vector components external to the predictor. In some other video coding techniques, there may be a motion vector component applicable to the subject block. In an embodiment, the video encoder (703) includes other components, such as a mode decision module (not shown) for determining the processing block mode.

[0096] exist Figure 7 In an embodiment of the present invention, the video encoder (703) includes Figure 7 Shown are an inter-frame encoder (730), an intra-frame encoder (722), a residual calculator (723), a switch (726), a residual encoder (724), a general controller (721), and an entropy encoder (725) coupled together.

[0097] The inter-frame encoder (730) is configured to receive samples of a current block (e.g., a processing block), compare the current block with one or more reference blocks in a reference image (e.g., blocks in a previous image and a subsequent image), generate inter-frame prediction information (e.g., redundant information description according to an inter-frame coding technique, motion vectors, merge mode information), and calculate an inter-frame prediction result (e.g., a predicted block) based on the inter-frame prediction information using any suitable technique. In some embodiments, the reference image is a decoded reference image decoded based on the encoded video information.

[0098] The intra-frame encoder (722) is configured to receive samples of a current block (e.g., a processing block), compare the block with previously encoded blocks in the same image in some cases, generate quantized coefficients after transformation, and in some cases also generate intra-frame prediction information (e.g., intra-frame prediction direction information based on one or more intra-frame coding techniques). In an embodiment, the intra-frame encoder (722) further calculates an intra-frame prediction result (e.g., a predicted block) based on the intra-frame prediction information and a reference block in the same image.

[0099] The general controller (721) is used to determine general control data and control other components of the video encoder (703) based on the general control data. In an embodiment, the general controller (721) determines the mode of the block and provides a control signal to the switch (726) based on the mode. For example, when the mode is intra mode, the general controller (721) controls the switch (726) to select the intra mode result for use by the residual calculator (723) and controls the entropy encoder (725) to select intra prediction information and add the intra prediction information to the bitstream; and when the mode is inter mode, the general controller (721) controls the switch (726) to select the inter prediction result for use by the residual calculator (723) and controls the entropy encoder (725) to select inter prediction information and add the inter prediction information to the bitstream.

[0100] The residual calculator (723) is used to calculate the difference (residual data) between the received block and the prediction result selected from the intra-frame encoder (722) or the inter-frame encoder (730). The residual encoder (724) is used to operate based on the residual data to encode the residual data to generate transform coefficients. In an embodiment, the residual encoder (724) is used to convert the residual data from the time domain to the frequency domain and generate transform coefficients. The transform coefficients are then quantized to obtain quantized transform coefficients. In various embodiments, the video encoder (703) also includes a residual decoder (728). The residual decoder (728) is used to perform an inverse transform and generate decoded residual data. The decoded residual data can be used appropriately by the intra-frame encoder (722) and the inter-frame encoder (730). For example, the inter-frame encoder (730) can generate a decoded block based on the decoded residual data and inter-frame prediction information, and the intra-frame encoder (722) can generate a decoded block based on the decoded residual data and intra-frame prediction information. The decoded blocks are appropriately processed to generate a decoded picture, and in some embodiments, the decoded picture may be buffered in a memory circuit (not shown) and used as a reference picture.

[0101] The entropy encoder (725) is used to format the codestream to produce encoded blocks. The entropy encoder (725) generates various information according to a suitable standard, such as the HEVC standard. In an embodiment, the entropy encoder (725) is used to obtain general control data, selected prediction information (e.g., intra-frame prediction information or inter-frame prediction information), residual information, and other suitable information from the codestream. It should be noted that according to the disclosed subject matter, when encoding a block in inter-frame mode or the merge sub-mode of bidirectional prediction mode, there is no residual information.

[0102] Figure 8FIG is a diagram of a video decoder (810) according to another embodiment disclosed herein. The video decoder (810) is configured to receive an encoded image as part of an encoded video sequence and decode the encoded image to generate a reconstructed image. In an embodiment, the video decoder (810) is configured to replace Figure 4 A video decoder (410) of an embodiment.

[0103] exist Figure 8 In one embodiment, the video decoder (810) includes Figure 7 , an entropy decoder (871), an inter-frame decoder (880), a residual decoder (873), a reconstruction module (874), and an intra-frame decoder (872) coupled together are shown.

[0104] The entropy decoder (871) is operable to reconstruct certain symbols from the coded image, these symbols representing syntax elements constituting the coded image. Such symbols may include, for example, the mode used to encode the block (e.g., intra mode, inter mode, bidirectional prediction mode, a combined submode of the latter two, or another submode), prediction information (e.g., intra prediction information or inter prediction information) that can identify certain samples or metadata for prediction by the intra decoder (872) or the inter decoder (880), respectively, residual information in the form of, for example, quantized transform coefficients, and the like. In an embodiment, when the prediction mode is inter or bidirectional prediction mode, the inter prediction information is provided to the inter decoder (880); and when the prediction type is intra prediction type, the intra prediction information is provided to the intra decoder (872). The residual information may be inverse quantized and provided to the residual decoder (873).

[0105] The inter-frame decoder (880) is configured to receive inter-frame prediction information and generate an inter-frame prediction result based on the inter-frame prediction information.

[0106] The intra-frame decoder (872) is configured to receive intra-frame prediction information and generate a prediction result based on the intra-frame prediction information.

[0107] The residual decoder (873) is used to perform inverse quantization to extract dequantized transform coefficients and process the dequantized transform coefficients to convert the residual from the frequency domain to the spatial domain. The residual decoder (873) may also require certain control information (to obtain the quantizer parameter QP), and this information can be provided by the entropy decoder (871) (the data path is not shown because this is only low-level control information).

[0108] The reconstruction module (874) is used to combine the residual output by the residual decoder (873) with the prediction result (which may be output by the inter-frame prediction module or the intra-frame prediction module) in the spatial domain to form a reconstructed block. The reconstructed block can be part of a reconstructed image, which in turn can be part of a reconstructed video. It should be noted that other suitable operations such as deblocking operations can be performed to improve visual quality.

[0109] It should be noted that the video encoder (403), video encoder (603), and video encoder (703), as well as the video decoder (410), video decoder (510), and video decoder (810) may be implemented using any suitable technology. In one embodiment, the video encoder (403), video encoder (603), and video encoder (703), as well as the video decoder (410), video decoder (510), and video decoder (810) may be implemented using one or more integrated circuits. In another embodiment, the video encoder (403), video encoder (603), and video encoder (703), as well as the video decoder (410), video decoder (510), and video decoder (810) may be implemented using one or more processors executing software instructions.

[0110] This application includes improvements to intra prediction based on template matching.

[0111] ITU-T VCEG (Q6 / 16) and ISO / IEC MPEG (JTC 1 / SC 29 / WG 11) announced the H.265 / HEVC (High Efficiency Video Coding) standard in 2013 (version 1), 2014 (version 2), 2015 (version 3), and 2016 (version 4). In 2015, the two standards organizations jointly formed the JVET (Joint Video Exploration Team) to explore the possibility of developing the next video coding standard after HEVC. In April 2018, JVET officially launched the standardization process for the next generation of video coding after HEVC. The new standard was named Versatile Video Coding (VVC), and JVET was renamed the Joint Video Expert Team. In July 2020, H.266 / VVC version 1 was finalized. In January 2021, an ad hoc group was established to study enhanced compression beyond the capabilities of VVC.

[0112] In the decoder-side intra-frame mode acquisition, the intra-frame mode can be derived using relevant syntax elements signaled in the codestream, or the intra-frame mode can be derived at the decoder-side without using relevant syntax elements signaled in the codestream. There are many methods for deriving the intra-frame mode at the decoder-side, and "decoder-side intra-frame mode acquisition" is not limited to the exemplary methods described in this application.

[0113] Multiple reference line (MRL) intra prediction can use more reference lines (or reference regions) for intra prediction. Figure 9 As shown in FIG, an example of four reference lines is depicted, where the samples of segments A and F are not extracted from the reconstructed adjacent samples, but are padded (or filled) with the nearest neighboring samples in segments B and E, respectively. HEVC intra prediction uses the nearest neighboring reference line (e.g., reference line 0). In MRL, two additional lines (e.g., reference line 1 and reference line 3) can be used. Therefore, the samples in these two additional lines can be used for intra prediction of the block unit (902).

[0114] The index of the selected reference row (e.g., mrl_idx) can be signaled and used to generate an intra predictor for the block unit (902). For reference row indices greater than 0, only additional reference row modes can be included in the most probable mode (MPM) list, and only the MPM index, not the remaining modes (e.g., intra prediction modes not included in the MPM list), can be signaled. The reference row index can be signaled before the intra prediction mode, and planar mode and DC mode can be excluded from the intra prediction mode if a non-zero reference row index is signaled.

[0115] Template-based intra mode derivation (TIMD) can use the reference samples of the current CU as a template and select the best intra mode from a set of candidate intra prediction modes associated with TIMD. Figure 10As shown in , the neighboring reconstructed samples of the current CU (1002) can be used as a template (1004). The reconstructed samples in the template (1004) can be compared with the predicted samples of the template (1004). The predicted samples can be generated using the reference samples (1006) of the template (1004). The reference samples (1006) can be the neighboring reconstructed samples around the template (1004). The cost function can be used to calculate the cost (or distortion) between the predicted samples and the reconstructed samples in the template (1004) based on a corresponding one of a set of candidate intra-frame prediction modes. The intra-frame prediction mode with the minimum cost (or distortion) can be selected as the best intra-frame prediction mode to perform inter-frame prediction on the current CU (1002).

[0116] Table 1 shows an exemplary encoding process associated with TIMD. As shown in Table 1, when the decoder-side intra mode derivation (DIMD) flag (e.g., DIMD_flag) is not 1 (or not true), a TIMD flag (e.g., TIMD_flag) can be signaled. When DIMD_flag is 1, the current CU / PU is using DIMD, and the ISP flag (e.g., ISP_flag) can be parsed to see if ISP is used for the current CU / PU. When DIMD_flag is not 1, TIMD_flag is parsed. When TIMD_flag is 1, TIMD can be applied to the current CU / PU without applying other intra-coding tools (e.g., ISP is not allowed when TIMD is used). When TIMD_flag is not 1, other intra-coding tools such as associated syntax elements (MIP, MRL, MPM, etc.) can be parsed in the decoder.

[0117] Table 1: Pseudocode for TIMD signaling

[0118]

[0119] In this application, MRL based on decoder-side template matching can be applied. MRL based on template matching can use a template to find a reference line (e.g., the best reference line) among the candidate reference lines of the current CU / PU, and the reference line can be used for intra prediction of the current CU / PU. The reference index of the current CU can be obtained using a template matching-based method instead of being signaled in the bitstream. Examples of templates can be found in, but are not limited to, 11A to 11D .

[0120] In one embodiment, the template may include only one column and / or one row, e.g. Figure 11A and Figure 11BAs shown in . Accordingly, the candidate reference rows may include reference rows 1, 2 and 3. For example, in Figure 11A In

[110] , the current CU (1102) may have a template that may include one or a combination of: (i) top samples (1104A) in a row above the top edge of the current CU (1102), and (ii) side samples (1104B) in a column adjacent to the left edge of the current CU (1102). Samples in reference rows 1, 2, and 3 (1106) may be applied to generate prediction samples for the template. Figure 11B , the template of the current CU ( 1102 ) may further include a joint sample ( 1104C) located above the side sample 1104B and adjacent to the top sample 1104A.

[0121] exist Figure 11C , the template of the current CU (1102) may include one of or a combination of top samples in multiple rows and side samples in multiple columns. For example, (i) the top sample (1104A) in the first row above the top edge of the current CU (1102) and the top sample (1104D) in the second row above the first row, and (ii) one of or a combination of the side sample (1104B) in the first column adjacent to the left edge of the current CU (1102) and the side sample (1104E) in the second column next to the first column. Accordingly, the candidate reference rows may be reference row 2 and reference row 3. The samples (1106) in reference row 2 and reference row 3 may be applied to generate the predicted samples of the template. In Figure 11D , the template may also include a joint sample (1104C) that is located above or on top of the side samples (1104B) and (1104E) in the first and second columns and to the left of the top samples (1104A) and (1104D) in the first and second rows.

[0122] For each candidate reference row, a prediction sample of the template can be generated. A cost function can be used to calculate the cost between the prediction sample of the template and the reconstructed sample of the template corresponding to the prediction sample of the template. For example, the sum of absolute difference (SAD) or the sum of absolute transformed difference (SATD) between the prediction sample of the template and the reconstructed sample of the template corresponding to the prediction sample of the template can be calculated. The reference row with the smallest cost (for example, the smallest SAD value or the smallest SATD value) can be selected as the reference row or the best reference row for the current CU / PU. Accordingly, the samples in the reference row can be used for intra-frame prediction of the current CU / PU.

[0123] The application of template matching-based MRL can be indicated by template matching-based MRL information, such as a template matching-based MRL flag (e.g., MRL_flag). For example, when the value of the template matching-based MRL flag is 1, it indicates that the current CU / PU uses template matching-based MRL. Otherwise, when the value of MRL_flag is not 1, template matching-based MRL is not used.

[0124] In this application, the template matching based MRL can be combined with another template based mode. For example, Figure 10 The exemplary pseudo code for the combination of MRL and TIMD based on template matching described in .

[0125] Table 2: Pseudocode for the combination of MRL and TIMD based on template matching

[0126]

[0127] As shown in Table 2, a TIMD flag (e.g., TIMD_flag) may be signaled. When TIMD_flag is 1, it indicates that the current CU / PU is using TIMD. Furthermore, a template matching-based MRL flag (e.g., MRL_flag) may be parsed to determine whether template matching-based MRL is used for the current CU / PU. When TIMD_flag is not 1, syntax elements related to other intra-coding tools (e.g., MIP, MPM, etc.) may be parsed in the decoder.

[0128] When both TIMD_flag and MRL_flag are equal to 1, the same template (e.g., 11A to 11D ) to determine the intra mode (e.g., the best intra mode) and the reference line (e.g., the best reference line). Therefore, a pair of selectedIntraMode (e.g., bestIntraMode) and selectedRefereneLine (e.g., bestReferenceLine) can be obtained. In order to determine the intra mode and reference line to be used, a set of candidate intra modes and candidate reference lines (e.g., 11A to 11DThe search process is performed using reference lines 1-3 in the set of candidate intra modes. For each candidate intra mode in the set of candidate intra modes and each candidate reference line in the set of candidate reference lines, the SATD or SAD between the predicted samples of the template and the reconstructed samples of the template can be calculated. Thus, multiple pairs of intra modes and reference lines can be obtained. A pair of intra mode and reference line with the minimum cost can be selected as the intra mode and reference line of the current CU / PU. Thereafter, intra prediction can be performed using the selected intra mode and the selected reference line.

[0129] Figure 12 A flowchart illustrating an overview of an exemplary decoding process (1200) according to some embodiments of the present application is shown. Figure 13 A flowchart outlining an exemplary encoding process (1300) for some embodiments of the present application is shown. The proposed methods can be used individually or in combination in any order. Furthermore, each of the methods (or embodiments), encoders, and decoders can be implemented by processing circuitry (e.g., one or more processors or one or more integrated circuits). In one example, one or more processors execute a program stored in a non-volatile computer-readable medium.

[0130] In various embodiments, any operations of the processes (e.g., (1200) and (1300)) may be combined or arranged in any number or order as needed. In various embodiments, two or more operations of the processes (e.g., (1200) and (1300)) may be performed in parallel.

[0131] The processes (e.g., (1200) and (1300)) can be used for reconstruction and / or encoding of a block to generate a prediction block for the block being reconstructed. In various embodiments, the processes (e.g., (1200) and (1300)) are performed by processing circuitry, such as processing circuitry in terminal devices (210), (220), (230), and (240), processing circuitry that performs the functions of a video encoder (303), processing circuitry that performs the functions of a video decoder (310), processing circuitry that performs the functions of a video decoder (410), and processing circuitry that performs the functions of a video encoder (503). In some embodiments, the processes (e.g., (1200) and (1300)) are implemented in software instructions, so that when the processing circuitry executes the software instructions, the processing circuitry performs the processes (e.g., (1200) and (1300)).

[0132] like Figure 12As shown, the process (1200) may start from (S1201) and then proceed to (S1210). At (S1210), encoded information about a coding unit (CU), a template region, and a plurality of reference regions may be received from an encoded video stream. The encoded information may include a first syntax element indicating whether the CU is predicted according to a multiple reference line (MRL) intra prediction mode based on template matching. The template region may be adjacent to the CU, and the plurality of reference regions may be adjacent to the template region.

[0133] At (S1220), in response to the first syntax element indicating that the CU is predicted according to the MRL intra-frame prediction mode based on template matching, multiple cost values ​​between (i) each predicted sample of the template area obtained based on the samples in each reference area of ​​the multiple reference areas and (ii) the reconstructed samples of the template area corresponding to the corresponding predicted samples can be determined.

[0134] At (S1230), a reference region may be determined from among the plurality of reference regions based on the plurality of cost values.

[0135] At (S1240), each sample of the CU may be reconstructed based on the samples in the determined reference region.

[0136] In some embodiments, each of the multiple cost values ​​can be determined based on the difference between (i) a respective predicted sample of the template region determined based on a sample in a corresponding one of the multiple reference regions and (ii) a reconstructed sample of the template region corresponding to the respective predicted sample.

[0137] In some embodiments, the template area may further include one or a combination of the following: (i) top samples in a row above the top edge of the CU and (ii) side samples in a column adjacent to the left edge of the CU.

[0138] In some embodiments, the template region may further include a junction sample located above the side sample and adjacent to the top sample.

[0139] In some embodiments, the multiple reference regions may include: (i) a first reference region including a row segment located above the top edge of the template region and a column segment adjacent to the left edge of the template region; (ii) a second reference region including a row segment above the row segment of the first reference region and a column segment adjacent to the column segment of the first reference region; and (iii) a third reference region including a row segment above the row segment of the second reference region and a column segment adjacent to the column segment of the second reference region.

[0140] In some embodiments, the template area may further include one or a combination of the following: (i) top samples in the first row above the top edge of the CU, and (ii) side samples in the first column adjacent to the left edge of the CU. The template area may also include (i) top samples in the second row above the first row, and (ii) side samples in the second column next to the first column.

[0141] In some embodiments, the template region may further include a junction sample located above the side samples in the first and second columns and to the left of the top samples in the first and second rows.

[0142] In some embodiments, the multiple reference regions may include: a first reference region, which includes a row segment located above the top edge of the template region and a column segment adjacent to the left edge of the template region; and a second reference region, which includes a row segment above the row segment of the first reference region and a column segment adjacent to the column segment of the first reference region.

[0143] In the process (1200), a second syntax element may be further decoded from the encoded information, the second syntax element indicating whether the CU is intra-predicted according to a template-based intra mode derivation (TIMD) mode. The TIMD mode may include a set of candidate intra prediction modes. In response to the first syntax element indicating that the CU is predicted according to the template matching-based MRL intra prediction mode and the second syntax element indicating that the CU is intra-predicted based on the TIMD mode, each prediction sample of the template region may be determined based on the following information: (i) samples in each reference region of a plurality of reference regions, and (ii) each candidate intra prediction mode in the set of candidate intra prediction modes. A plurality of cost values ​​may be determined. Each cost value in the plurality of cost values ​​may be determined based on a sum of absolute transform differences between each prediction sample of the template region and a reconstructed sample of the template region corresponding to the corresponding prediction sample. A pair of a reference region from the plurality of reference regions and an intra prediction mode from the set of candidate intra prediction modes may be determined, the pair of reference region and intra prediction mode being associated with a lowest cost value among the plurality of cost values. The samples of the CU may be reconstructed based on the determined pair of reference regions and samples in the intra prediction mode.

[0144] In some embodiments, in response to a second syntax element indicating that intra prediction of the CU is not performed based on the TIMD mode, a syntax element associated with another intra coding mode can be decoded, where the other intra coding mode can include one of matrix-based intra prediction (MIP) and MPM.

[0145] like Figure 13As shown, process (1300) may start at (S1301) and then proceed to (S1310). At (S1310), a plurality of cost values ​​may be determined between (i) respective predicted samples of a template region of a coding unit (CU) in an image obtained based on samples in each of a plurality of reference regions of the CU and (ii) reconstructed samples of the template region corresponding to the respective predicted samples. The template region may be adjacent to the CU, and the plurality of reference regions may be adjacent to the template region.

[0146] For example, each of the plurality of cost values ​​may be determined according to a difference between each predicted sample of the template region determined based on samples in each of the plurality of reference regions and a reconstructed sample of the template region corresponding to the corresponding predicted sample.

[0147] At (S1320), the reference region may be determined from the plurality of reference regions based on the plurality of cost values. For example, the determined reference region may be associated with a lowest cost value among the plurality of cost values.

[0148] At (S1330), the CU may be predicted based on the determined reference region and the template region according to a multiple reference line (MRL) intra prediction mode based on template matching.

[0149] At (S1340), a first syntax element may be generated, the first syntax element indicating whether to perform intra prediction on the CU according to the MRL intra prediction mode based on template matching.

[0150] The techniques described above may be implemented as computer software using computer-readable instructions and physically stored in one or more computer-readable media. For example, Figure 14 A computer system (1400) suitable for implementing certain embodiments of the disclosed subject matter is shown.

[0151] Computer software may be encoded using any suitable machine code or computer language that may be assembled, compiled, linked, or similar mechanisms to create code comprising instructions that may be executed directly by one or more computer central processing units (CPUs), graphics processing units (GPUs), etc., or through interpretation, microcode execution, etc.

[0152] These instructions may be executed on various types of computers or components thereof, including, for example, personal computers, tablet computers, servers, smart phones, gaming devices, IoT devices, and the like.

[0153] Figure 14The components shown for the computer system (1400) are exemplary in nature and are not intended to limit the scope of use or functionality of the computer software implementing the embodiments of the application. The configuration of components should not be interpreted as having any dependency or requirement on any one component or combination of components shown in the exemplary embodiment of the computer system (1400).

[0154] The computer system (1400) may include certain human interface input devices. Such human interface input devices may be responsive to input from one or more human users through, for example, tactile input (e.g., keystrokes, swipes, data glove motions), audio input (e.g., voice, taps), visual input (e.g., gestures), and olfactory input (not depicted). The human interface devices may also be used to capture certain media that is not necessarily directly related to conscious human input, such as audio (e.g., voice, music, ambient sounds), images (e.g., scanned images, photographic images obtained from a still camera), and video (e.g., two-dimensional video, three-dimensional video including stereoscopic video).

[0155] Input human interface devices may include one or more of the following (only one of each is depicted): keyboard (1401), mouse (1402), touchpad (1403), touch screen (1410), data gloves (not shown), joystick (1405), microphone (1406), scanner (1407), camera (1408).

[0156] The computer system (1400) may also include certain human interface output devices. Such human interface output devices may stimulate one or more of the human user's senses through, for example, tactile output, sound, light, and smell / taste. Such human interface output devices may include tactile output devices (e.g., tactile feedback through a screen (1410), a data glove (not shown), or a joystick (1405), although there may also be tactile feedback devices that do not function as input devices), audio output devices (e.g., speakers (1409), headphones (not depicted)), visual output devices (e.g., for screens (1410) including CRT screens, LCD screens, plasma screens, OLED screens, each with or without touch screen input capabilities, each with or without tactile feedback capabilities, some of which are capable of outputting two-dimensional visual output or more than three-dimensional output through, for example, stereoscopic output; virtual reality glasses (not depicted), holographic displays, and smoke canisters (not depicted), and printers (not depicted).

[0157] The computer system (1400) may also include human-accessible storage devices and their associated media, such as optical media including media (1421) such as CD / DVD ROM / RW (1420) with CD / DVD, thumb drives (1422), removable hard drives or solid-state drives (1423), traditional magnetic media such as tapes and floppy disks (not depicted), dedicated ROM / ASIC / PLD-based devices such as security dongles (not depicted), and the like.

[0158] Those skilled in the art will also understand that the term "computer-readable media" used in connection with the presently disclosed subject matter does not include transmission media, carrier waves, or other volatile signals.

[0159] The computer system (1400) may also include an interface (1454) to one or more communication networks (1455). The network may be, for example, wireless, wired, or optical. The network may further be local, wide-area, metropolitan, vehicular, industrial, real-time, delay-tolerant, and the like. Examples of networks include local area networks (e.g., Ethernet), wireless LANs, cellular networks (including GSM, 3G, 4G, 5G, LTE, etc.), television wired or wireless wide-area digital networks (including cable, satellite, and terrestrial broadcast television), vehicle and industrial networks (including CANBus), and the like. Some networks typically require an external network interface adapter attached to some common data port or peripheral bus (1449) (e.g., a USB port of the computer system (1400)); other networks are typically integrated into the kernel of the computer system (1400) by attaching to a system bus as described below (e.g., an Ethernet interface integrated into a PC computer system or a cellular network interface integrated into a smartphone computer system). Using any of these networks, the computer system (1400) can communicate with other entities. Such communication can be one-way, receive-only (e.g., broadcast TV), one-way send-only (e.g., CANbus to certain CANbus devices), or two-way (e.g., to other computer systems using a local area digital network or a wide area digital network). Certain protocols and protocol stacks can be used on each of the networks and network interfaces as those described above.

[0160] The above-mentioned human interface devices, human-accessible storage devices, and network interfaces may be attached to the kernel (1440) of the computer system (1400).

[0161] The core (1440) may include one or more central processing units (CPUs) (1441), graphics processing units (GPUs) (1442), specialized programmable processing units in the form of field programmable gate arrays (FPGAs) (1443), hardware accelerators for certain tasks (1444), a graphics adapter (1450), and the like. These devices, along with read-only memory (ROM) (1445), random access memory (1446), and internal mass storage devices (1447) such as internal non-user accessible hard drives and solid-state drives (SSDs), may be connected via a system bus (1448). In some computer systems, the system bus (1448) may be accessible in the form of one or more physical plugs to enable expansion by additional CPUs, GPUs, and the like. Peripheral devices may be attached directly to the core's system bus (1448) or to the system bus (1448) via a peripheral bus (1449). In one example, a screen (1410) may be connected to a graphics adapter (1450). Peripheral bus architectures include PCI, USB, and the like.

[0162] The CPU (1441), GPU (1442), FPGA (1443), and accelerator (1444) can execute certain instructions, the combination of which can constitute the aforementioned computer code. The computer code can be stored in ROM (1445) or RAM (1446). Transient data can also be stored in RAM (1446), while permanent data can be stored in, for example, an internal mass storage device (1447). Fast storage and retrieval of any memory device can be enabled by using a cache memory, which can be closely associated with one or more CPUs (1441), GPUs (1442), mass storage devices (1447), ROM (1445), RAM (1446), etc.

[0163] The computer readable medium may have computer code thereon for performing various computer-implemented operations. The media and computer code may be those specially designed and constructed for the purposes of this application, or they may be of a type well known and available to those skilled in the art of computer software.

[0164] As an example and not by way of limitation, a computer system having the architecture (1400) and, in particular, the kernel (1440) can provide functionality as a result of one or more processors (including CPUs, GPUs, FPGAs, accelerators, etc.) executing software implemented in one or more tangible computer-readable media. Such computer-readable media can be media associated with user-accessible mass storage devices as described above, as well as certain storage devices of the kernel (1440) having non-volatile properties (e.g., kernel-internal mass storage devices (1447) or ROM (1445)). Software implementing various embodiments of the present application can be stored in such devices and executed by the kernel (1440). Depending on specific needs, the computer-readable medium can include one or more memory devices or chips. The software can cause the kernel (1440) and, in particular, the processors therein (including CPUs, GPUs, FPGAs, etc.) to perform specific processes or specific portions of specific processes described herein, including defining data structures stored in RAM (1446) and modifying such data structures according to software-defined processes. In addition or as an alternative, the computer system may provide functionality as a result of logic hardwired or otherwise implemented in circuitry (e.g., accelerator (1444)) that may operate in place of or in conjunction with software to perform a particular process or a particular portion of a particular process described herein. Where appropriate, references to software may include logic and vice versa. Where appropriate, references to computer-readable media may include circuitry (e.g., an integrated circuit (IC)) storing software for execution, circuitry including logic for execution, or both. The present application encompasses any suitable combination of hardware and software.

[0165] Appendix A: Acronyms

[0166] JEM: joint exploration model

[0167] VVC: universal video coding

[0168] BMS: benchmark set

[0169] MV: motion vector

[0170] HEVC: High Efficiency Video Coding

[0171] SEI: Supplementary Enhancement Information

[0172] VUI: Video Usability Information

[0173] GOP:Groups of Pictures

[0174] TU: Transform Unit

[0175] PU: Prediction Unit

[0176] CTU: Coding Tree Unit

[0177] CTB: Coding Tree Block

[0178] PB: Prediction Block

[0179] HRD: Hypothetical Reference Decoder

[0180] SNR:Signal Noise Ratio

[0181] CPU:Central Processing Unit

[0182] GPU: Graphics Processing Unit

[0183] CRT: Cathode Ray Tube

[0184] LCD: Liquid-Crystal Display

[0185] OLED: Organic Light-Emitting Diode

[0186] CD: Compact Disc

[0187] DVD: Digital Video Disc

[0188] ROM:Read-Only Memory

[0189] RAM: Random Access Memory

[0190] ASIC: Application-Specific Integrated Circuit

[0191] PLD:Programmable Logic Device

[0192] LAN:Local Area Network

[0193] GSM:Global System for Mobile communicationGlobal System for Mobile Communications

[0194] LTE: Long-Term Evolution

[0195] CANBus:Controller Area Network Bus

[0196] USB: Universal Serial Bus

[0197] PCI: Peripheral Component Interconnect

[0198] FPGA: Field Programmable Gate Array field programmable gate area

[0199] SSD: solid-state drive

[0200] IC:Integrated Circuit

[0201] CU: Coding Unit

[0202] Although this application has described a number of exemplary embodiments, various modifications, permutations, and substitutions of the embodiments are within the scope of this application. Therefore, it should be understood that those skilled in the art will be able to design various systems and methods that, although not explicitly shown or described herein, embody the principles of this application and are therefore within the spirit and scope of this application.

Claims

1. A decoding method, characterized in that: include: Receiving encoded information about a coding unit (CU), a template region, and a plurality of reference regions from an encoded video bitstream, the encoded information comprising a first syntax element and a second syntax element, the first syntax element indicating whether the CU is predicted according to a template matching-based multiple reference line (MRL) intra prediction mode, the second syntax element indicating whether the CU is intra predicted according to a template based intra mode derivation (TIMD) mode, the TIMD mode including a set of candidate intra prediction modes, the template region being adjacent to the CU, and the plurality of reference regions being adjacent to the template region; If the first syntax element indicates that the CU is predicted according to the template matching based MRL intra prediction mode and the second syntax element indicates that the CU is not intra predicted according to the TIMD mode, then decoding a syntax element associated with another intra coding mode, the another intra coding mode comprising one of a matrix-based intra prediction (MIP) and a most probable mode (MPM); determining a plurality of cost values ​​between (i) respective predicted samples of the template region determined based on samples in each of the plurality of reference regions and (ii) reconstructed samples of the template region corresponding to the respective predicted samples; determining a reference region from among the plurality of reference regions based on the plurality of cost values; and Reconstructing samples of the CU based on the determined samples in the reference area; If the first syntax element indicates that the CU is predicted according to the MRL intra prediction mode based on template matching and the second syntax element indicates that the CU is intra predicted according to the TIMD mode, then determining the corresponding prediction sample of the template region based on: (i) the sample located in one of the reference regions, and (ii) a corresponding candidate intra prediction mode in the set of candidate intra prediction modes; determining a plurality of cost values, each of the plurality of cost values ​​being determined according to a sum of absolute transformation differences between respective prediction samples of the template region and the reconstructed samples of the template region corresponding to the corresponding prediction samples; determining a reference region from the plurality of reference regions and an intra prediction mode from the set of candidate intra prediction modes, the determined reference region and the determined intra prediction mode being associated with a lowest cost value from among the plurality of cost values; and The samples of the CU are reconstructed based on the determined reference region and samples in the intra prediction mode.

2. The method according to claim 1, characterized in that: Each of the plurality of cost values ​​is determined based on a difference between (i) a respective predicted sample of the template region determined based on a sample in one of the plurality of reference regions and (ii) the reconstructed sample of the template region to which the respective predicted sample corresponds.

3. The method according to claim 1, characterized in that The template area includes: One or a combination of the following: (i) top samples in a row above the top edge of the CU and (ii) side samples in a column adjacent to the left edge of the CU.

4. The method according to claim 3, characterized in that The template area further includes: A seam sample located above the side sample and adjacent to the top sample.

5. The method according to claim 4, characterized in that The plurality of reference areas include: a first reference region including a row segment located above a top edge of the template region and a column segment adjacent to a left edge of the template region; a second reference region including a row segment above the row segment of the first reference region and a column segment adjacent to the column segment of the first reference region; and The third reference region includes a row segment above the row segment of the second reference region and a column segment adjacent to the column segment of the second reference region.

6. The method according to claim 1, characterized in that The template area further includes: One or a combination of the following: (i) top samples in a first row above a top edge of the CU and top samples in a second row above the first row, and (ii) side samples in a first column adjacent to a left edge of the CU and side samples in a second column next to the first column.

7. The method according to claim 6, characterized in that The template area further includes: A seam sample located above the side samples of the first and second columns and to the left of the top samples of the first and second rows.

8. The method according to claim 7, characterized in that The plurality of reference areas include: a first reference region including a row segment located above a top edge of the template region and a column segment adjacent to a left edge of the template region; and The second reference region includes a row segment located above the row segment of the first reference region and a column segment adjacent to the column segment of the first reference region.

9. A coding method, characterized in that: include: If it is determined that a coding unit (CU) is predicted according to a template matching based multiple reference line (MRL) intra prediction mode, and it is determined that the CU is not intra predicted according to a template based intra mode derivation (TIMD) mode, then Determine a syntax element associated with a first syntax element, a second syntax element, and another intra-frame coding mode, wherein the first syntax element indicates that the CU is predicted according to the MRL intra-frame prediction mode based on template matching, and the second syntax element indicates that the CU is not intra-frame predicted according to the TIMD mode, and the other intra-frame coding mode includes one of a matrix-based intra-frame prediction (MIP) and a most probable mode (MPM); Determine a plurality of cost values ​​between (i) respective prediction samples of a template region determined based on samples in each of a plurality of reference regions and (ii) reconstructed samples of the template region corresponding to the respective prediction samples, the template region being adjacent to the CU, and the plurality of reference regions being adjacent to the template region; determining a reference region from among the plurality of reference regions based on the plurality of cost values; Encoding the samples of the CU based on the determined samples in the reference area to obtain encoded information of the CU, the template area, and the multiple reference areas; Adding the first syntax element, the second syntax element and the syntax element associated with the another intra coding mode to the encoded information; If it is determined that the CU is predicted according to a multiple reference line (MRL) intra prediction mode based on template matching, and it is confirmed that the CU is intra predicted according to the TIMD mode, then Determine a first syntax element and a second syntax element, wherein the first syntax element indicates that the CU is predicted according to the MRL intra prediction mode based on template matching, and the second syntax element indicates that the CU is intra predicted according to the TIMD mode; Determine a prediction sample of a template region based on the following information: (i) the sample located in one of the multiple reference regions, and (ii) a corresponding candidate intra prediction mode in a set of candidate intra prediction modes included in the TIMD mode, the template region being adjacent to the CU, and the multiple reference regions being adjacent to the template region; determining a plurality of cost values, each of the plurality of cost values ​​being determined according to a sum of absolute transformation differences between respective predicted samples of the template region and reconstructed samples of the template region corresponding to the corresponding predicted samples; determining a reference region from the plurality of reference regions and an intra prediction mode from the set of candidate intra prediction modes, the determined reference region and the determined intra prediction mode being associated with a lowest cost value from among the plurality of cost values; Encoding samples of the CU based on the determined reference area and samples in the intra prediction mode to obtain encoded information of the CU, the template area, and the multiple reference areas; Adding the first syntax element and the second syntax element to the encoded information; and The encoded information of the CU, the template region and the multiple reference regions is added to a bitstream.

10. A device for video decoding, characterized in that: include: A first module is configured to receive encoded information about a coding unit (CU), a template region, and a plurality of reference regions from an encoded video bitstream, the encoded information comprising a first syntax element and a second syntax element, the first syntax element indicating whether the CU is predicted according to a template matching-based multiple reference line (MRL) intra prediction mode, the second syntax element indicating whether the CU is intra predicted according to a template-based intra mode derivation (TIMD) mode, the TIMD mode comprising a set of candidate intra prediction modes, the template region being adjacent to the CU, and the plurality of reference regions being adjacent to the template region; The second module is configured to, when the first syntax element indicates that the CU is predicted according to the MRL intra prediction mode based on template matching and the second syntax element indicates that the CU is not intra predicted according to the TIMD mode, decoding a syntax element associated with another intra coding mode, the another intra coding mode comprising one of a matrix-based intra prediction (MIP) and a most probable mode (MPM); determining a plurality of cost values ​​between (i) respective predicted samples of the template region determined based on samples in each of the plurality of reference regions and (ii) reconstructed samples of the template region corresponding to the respective predicted samples; determining a reference region from among the plurality of reference regions based on the plurality of cost values; and Reconstructing samples of the CU based on the determined samples in the reference area; The third module is configured to, when the first syntax element indicates that the CU is predicted according to the MRL intra prediction mode based on template matching and the second syntax element indicates that the CU is intra predicted according to the TIMD mode, determining the corresponding prediction sample of the template region based on: (i) the sample located in one of the reference regions, and (ii) a corresponding candidate intra prediction mode in the set of candidate intra prediction modes; determining a plurality of cost values, each of the plurality of cost values ​​being determined according to a sum of absolute transformation differences between respective prediction samples of the template region and the reconstructed samples of the template region corresponding to the corresponding prediction samples; determining a reference region from the plurality of reference regions and an intra prediction mode from the set of candidate intra prediction modes, the determined reference region and the determined intra prediction mode being associated with a lowest cost value from among the plurality of cost values; and The samples of the CU are reconstructed based on the determined reference region and samples in the intra prediction mode.

11. A computing device, characterized in that: The method comprises a processor and a memory, wherein the memory stores instructions, and when the instructions are executed by the processor, the processor executes the method according to any one of claims 1 to 9.

12. A non-volatile computer-readable storage medium storing instructions, characterized in that: The instructions, when executed by at least one processor, cause the at least one processor to perform the method according to any one of claims 1-9.

13. A method for storing a video code stream, characterized in that: include: Generate a video code stream, wherein the video code stream is generated by the encoding method according to claim 9; The video code stream is stored in a computer-readable storage medium.

Images