Video decoding processing method and apparatus, video encoding processing method and apparatus, and device

Abstract

The present application relates to the technical field of computers, and discloses a video decoding processing method and apparatus, a video encoding processing method and apparatus, and a device. The video decoding processing method in embodiments of the present application comprises: on the basis of multi-reference line information of a first target block, a decoding end acquires a reconstructed reference sample corresponding to the first target block; the decoding end determines a predicted value of the first target block on the basis of the reconstructed reference sample; and the decoding end performs decoding processing on the basis of the predicted value.

Description

Video decoding processing methods, video encoding processing methods, devices and equipment

[0001] Cross-reference to related applications

[0002] This application claims priority to Chinese Patent Application No. 202411486763.1, filed in China on October 23, 2024, the entire contents of which are incorporated herein by reference. Technical Field

[0003] This application belongs to the field of computer technology, and specifically relates to a video decoding processing method, a video encoding processing method, an apparatus, and a device. Background Technology

[0004] The video encoder employs a block-based hybrid coding framework. The coding process includes: block partitioning, intra-frame prediction, inter-frame prediction, transform, quantization, loop filtering, and entropy coding. The encoder first divides the image into non-overlapping Coding Tree Units (CTUs), further subdividing them into different Coding Units (CUs) according to a quadtree structure. It then traverses various tree types for partitioning, ultimately determining the final partitioning result based on rate-distortion cost. The encoder uses intra-frame prediction technology based on image texture correlation. Using the reconstructed samples in the row above and column to the left adjacent to the current Coding Unit as references, it traverses DC mode, Planar mode, and angular prediction mode, utilizing rate-distortion cost to select the optimal intra-frame prediction mode to remove spatial redundancy within the block.

[0005] In related technologies, matrix-based intra prediction (PDP) uses reconstructed reference samples adjacent to the current block and obtains the predicted value through operations with the weight matrix. However, in natural images, the current block may have low correlation with adjacent reconstructed reference samples, and directly using reconstructed reference samples adjacent to the current block for intra prediction will result in poor performance in removing spatial redundancy of the block. Summary of the Invention

[0006] This application provides a video decoding processing method, a video encoding processing method, an apparatus, and a device that can solve the problem of poor spatial redundancy removal of blocks.

[0007] Firstly, a video decoding processing method is provided, the method comprising:

[0008] The decoding end obtains the reconstructed reference sample corresponding to the first target block based on the multi-reference row information of the first target block;

[0009] The decoding end determines the predicted value of the first target block based on the reconstructed reference sample;

[0010] The decoding end performs decoding processing based on the predicted value.

[0011] Secondly, a video encoding processing method is provided, including:

[0012] The encoding end obtains the reconstructed reference sample corresponding to the first target block based on the multi-reference row information of the first target block;

[0013] The encoding end determines the predicted value of the first target block based on the reconstructed reference sample;

[0014] The encoding end performs encoding processing based on the predicted value.

[0015] Thirdly, a video decoding processing apparatus is provided, comprising:

[0016] The first acquisition module is used to acquire the reconstruction reference sample corresponding to the first target block based on the multi-reference row information of the first target block;

[0017] The first determining module is used to determine the predicted value of the first target block based on the reconstructed reference sample;

[0018] The processing module is used to perform decoding processing based on the predicted value.

[0019] Fourthly, a video encoding processing apparatus is provided, comprising:

[0020] The first acquisition module is used to acquire the reconstruction reference sample corresponding to the first target block based on the multi-reference row information of the first target block;

[0021] The first determining module is used to determine the predicted value of the first target block based on the reconstructed reference sample;

[0022] The processing module is used to perform encoding processing based on the predicted value.

[0023] Fifthly, an apparatus for video encoding and decoding processing is provided, the apparatus being configured to perform the steps of the method described in the first aspect, or to implement the steps of the method described in the second aspect.

[0024] In a sixth aspect, an electronic device is provided, comprising a processor and a memory, the memory storing a program or instructions executable on the processor, the program or instructions, when executed by the processor, implementing the steps of the method as described in the first aspect, or implementing the steps of the method as described in the second aspect.

[0025] In a seventh aspect, an electronic device is provided, including a processor and a communication interface, wherein the processor is used for:

[0026] Based on the multi-reference row information of the first target block, obtain the reconstruction reference sample corresponding to the first target block;

[0027] The predicted value of the first target block is determined based on the reconstructed reference sample;

[0028] Decoding is performed based on the predicted values.

[0029] Eighthly, an electronic device is provided, comprising: a memory configured to store video data, and processing circuitry configured to implement the steps of the method described in the first aspect, or the steps of the method described in the second aspect.

[0030] A ninth aspect provides a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect, or implement the steps of the method described in the second aspect.

[0031] In a tenth aspect, a coding / decoding system is provided, comprising: a decoding end device and an encoding end device, wherein the decoding end device is configured to perform the steps of the method described in the first aspect, and the encoding end device is configured to perform the steps of the method described in the second aspect.

[0032] Eleventhly, a chip is provided, the chip including a processor and a communication interface coupled to the processor, the processor being configured to run a program or instructions to implement the steps of the method described in the first aspect, or to implement the steps of the method described in the second aspect.

[0033] In a twelfth aspect, a computer program / program product is provided, which is stored in a storage medium and is executed by at least one processor to implement the steps of the method as described in the first aspect, or to implement the steps of the method as described in the second aspect.

[0034] In this embodiment, the decoding end obtains the reconstructed reference sample corresponding to the first target block based on the multi-reference row information of the first target block; the decoding end determines the predicted value of the first target block based on the reconstructed reference sample; and the decoding end performs decoding processing based on the predicted value. Thus, obtaining the reconstructed reference sample corresponding to the first target block based on the multi-reference row information of the first target block for prediction and decoding is more flexible than directly using the reconstructed reference sample adjacent to the current block (i.e., the first target block) for intra-frame prediction, and can improve the effect of removing spatial redundancy of blocks. Attached Figure Description

[0035] Figure 1 is a schematic diagram of one of the encoding / decoding systems provided in an embodiment of this application;

[0036] Figure 2 is a second schematic diagram of the encoding and decoding system provided in an embodiment of this application;

[0037] Figure 3 is a third schematic diagram of the encoding and decoding system provided in the embodiment of this application;

[0038] Figure 4 is a schematic diagram of the angle prediction mode in related technologies;

[0039] Figure 5 is a schematic diagram of multiple reference rows in related technologies;

[0040] Figure 6 is a schematic diagram of the location of adjacent prediction units in related technologies;

[0041] Figure 7 is one of the schematic diagrams of templates in related technologies;

[0042] Figure 8 is a second schematic diagram of a template in related technologies;

[0043] Figure 9 is a flowchart illustrating a video decoding processing method provided in an embodiment of this application;

[0044] Figure 10 is a schematic diagram of a template provided in an embodiment of this application;

[0045] Figure 11 is a flowchart illustrating a video encoding processing method provided in an embodiment of this application;

[0046] Figure 12 is a schematic diagram of a video decoding processing device provided in an embodiment of this application;

[0047] Figure 13 is a schematic diagram of a video encoding processing device provided in an embodiment of this application;

[0048] Figure 14 is a structural diagram of an electronic device provided in an embodiment of this application;

[0049] Figure 15 is a structural diagram of a terminal provided in an embodiment of this application. Detailed Implementation

[0050] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0051] The terms "first," "second," etc., used in this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first" and "second" are generally of the same class, not limited in number; for example, the first object can be one or more. Furthermore, "or" in this application indicates at least one of the connected objects. For example, the scope of protection for "A or B" covers at least three scenarios: Scenario 1: including A but not B; Scenario 2: including B but not A; Scenario 3: including both A and B. In addition, the terms "A and / or B," "at least one of A and B," and "at least one of A or B" also cover at least the above three scenarios. The character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0052] Figure 1 is a schematic diagram of the encoding / decoding system 10 provided in an embodiment of this application. The technical solution of this application embodiment relates to encoding and decoding (CODEC) video data (including encoding or decoding). The video data includes original unencoded video, encoded video, decoded (e.g., reconstructed) video, or syntax elements, etc.

[0053] As shown in Figure 1, the encoding / decoding system 10 includes a source device 100, which provides encoded video data to be decoded and displayed by the destination device 110. Specifically, the source device 100 provides video data to the destination device 110 via a communication medium 120. The source device 100 and the destination device 110 may include any one or more of the following: desktop computer, laptop computer, tablet computer, set-top box, mobile phone, wearable device (e.g., smartwatch or wearable camera), television, camera, display device, in-vehicle device, virtual reality (VR) device, augmented reality (AR) device, mixed reality (MR) device, digital media player, video game console, video conferencing equipment, video streaming equipment, broadcast receiver equipment, broadcast transmitter equipment, spacecraft, aircraft, robot, satellite, etc.

[0054] In the example of Figure 1, source device 100 includes a data source 101, memory 102, encoder 200, and output interface 104. Destination device 110 includes an input interface 111, decoder 300, memory 113, and display device 114. Source device 100 represents an example of a video encoding device, while destination device 110 represents an example of a video decoding device. In other examples, source device 100 and destination device 110 may not include some of the components shown in Figure 1, or they may include components other than those shown in Figure 1. For example, source device 100 may receive video data from an external data source (such as an external camera). Similarly, destination device 110 may interface with an external display device instead of including an integrated display device. As another example, memory 102 and memory 113 may be external memories.

[0055] Although Figure 1 illustrates the source device 100 and the destination device 110 as separate devices, in some examples, they may be integrated into a single device. In such embodiments, the same hardware or software, separate hardware or software, or any combination thereof may be used to implement the functionality corresponding to the source device 100 and the functionality corresponding to the destination device 110.

[0056] In some examples, source device 100 and destination device 110 can perform unidirectional or bidirectional video transmission. If it is bidirectional video transmission, source device 100 and destination device 110 can operate in a substantially symmetrical manner, that is, each of source device 100 and destination device 110 includes an encoder and a decoder.

[0057] Data source 101 represents the source of video data (i.e., raw, unencoded video data) and provides encoder 200 with a series of images containing video data, which encoder 200 encodes. Data source 101 of source device 100 may include a video acquisition device (such as a video camera), a video archive containing previously acquired raw video, or a video feed interface for receiving video from a video content provider. Alternatively, data source 101 may generate computer graphics-based data as source video, or combine live video, archived video, and computer-generated video. In these cases, encoder 200 encodes the acquired, pre-acquired, or computer-generated video data. Encoder 200 may rearrange the images from the received order (sometimes referred to as the "display order") according to the encoding order. Encoder 200 may generate a bitstream including the encoded video data. Source device 100 may then output the encoded video data to communication medium 120 via output interface 104 for reception or retrieval, for example, by input interface 111 of destination device 110.

[0058] The memory 102 of the source device 100 and the memory 113 of the destination device 110 represent general-purpose memory. In some examples, memory 102 may store raw video data from data source 101, and memory 113 may store decoded video data from decoder 300. Additionally or alternatively, memories 102 and 113 may respectively store software instructions executable by, for example, encoder 200 and decoder 300. Although memories 102 and 113 are shown separately from encoder 200 and decoder 300 in this example, it should be understood that encoder 200 and decoder 300 may also include internal memory for functionally similar or equivalent purposes. If encoder 200 and decoder 300 are deployed on the same hardware device, memories 102 and 113 may be the same memory. Furthermore, memories 102 and 113 may store, for example, encoded video data output from encoder 200 and input to decoder 300. In some examples, portions of memories 102 and 113 may be allocated as one or more video buffers, for example, to store raw, decoded, or encoded video data.

[0059] In some examples, source device 100 can output encoded data from output interface 104 to memory 113. Similarly, destination device 110 can access encoded data from memory 113 via input interface 111. Memory 113 or memory 102 can include any of a variety of distributed or locally accessed data storage media, such as hard drives, Blu-ray discs, digital versatile discs (DVDs), compact disc read-only memory (CD-ROMs), flash memory, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded video data.

[0060] Output interface 104 may include any type of medium or device capable of transmitting encoded video data from source device 100 to destination device 110. For example, output interface 104 may include a transmitter or transceiver, such as an antenna, configured to transmit encoded video data directly from source device 100 to destination device 110 in real time. The encoded video data may be modulated according to the communication standards of a wireless communication protocol and transmitted to destination device 110.

[0061] Communication medium 120 may include transient media, such as wireless broadcasting or wired network transmission. For example, communication medium 120 may include radio frequency (RF) spectrum or one or more physical transmission lines (e.g., cables). Communication medium 120 may form part of a packet-based network (such as a local area network, a wide area network, or a global network such as the Internet). Communication medium 120 may also take the form of a storage medium (e.g., a non-transitory storage medium), such as a hard disk, flash drive, compact disc, digital video disc, Blu-ray disc, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded video data.

[0062] In some implementations, the communication medium 120 may include a router, switch, base station, or any other device that can be used to facilitate communication from source device 100 to destination device 110. For example, a server (not shown) may receive encoded video from source device 100 and provide the encoded video data to destination device 110, for example, via network transmission. The server may include (e.g., a web server for a website), a server configured to provide file transfer protocol services (such as File Transfer Protocol (FTP) or File Delivery Over Unidirectional Transport (FLUTE) protocol), a content delivery network (CDN) device, a Hypertext Transfer Protocol (HTTP) server, a Multimedia Broadcast Multicast Services (MBMS) or Evolved Multimedia Broadcast Multicast Service (eMBMS) server, or a Network-attached storage (NAS) device, etc. The server can implement one or more HTTP streaming protocols, such as MPEG Media Transport (MMT), Dynamic Adaptive Streaming over HTTP (DASH), HTTP Live Streaming (HLS), or Real Time Streaming Protocol (RTSP).

[0063] Destination device 110 can access encoded video data from a server, for example, via a wireless channel (e.g., Wi-Fi connection) or a wired connection (e.g., digital subscriber line (DSL), cable modem, etc.) for accessing encoded video data stored on the server.

[0064] Output interface 104 and input interface 111 can represent a wireless transmitter / receiver, a modem, a wired networking component (e.g., an Ethernet card), a wireless communication component operating according to the IEEE 802.11 or IEEE 802.15 standard (e.g., ZigBee™), Bluetooth standard, or other physical components. In an example where output interface 104 and input interface 111 include wireless components, output interface 104 and input interface 111 can be configured to transmit data, such as encoded video data, via Wi-Fi, Ethernet, cellular networks (such as 4G, Long Term Evolution (LTE), Advanced LTE, 5G, 6G, etc.).

[0065] The technology provided in this application can be applied to support video encoding and decoding in one or more multimedia applications such as video conferencing, over-the-air television broadcasting, cable television transmission, satellite television transmission, internet streaming video transmission, digital video encoded onto a data storage medium, decoding of digital video stored on a data storage medium, or other applications.

[0066] The input interface 111 of the destination device 110 receives an encoded video bitstream from the communication medium 120. The encoded video bitstream may include syntax elements and encoded data units (e.g., sequences, image groups, images, slices, blocks, etc.), where the syntax elements are used to decode the encoded data units to obtain decoded video data. The display device 114 displays the decoded video data to the user. The display device 114 may include a cathode ray tube (CRT), a liquid crystal display (LCD), a plasma display, an organic light-emitting diode (OLED) display, or other types of display devices.

[0067] The encoder 200 and decoder 300 can be implemented as one or more of various processing circuits, which may include microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, hardware, or any combination thereof. When the technology is implemented wholly or partially in software, the device may store instructions for the software in a suitable non-transitory computer-readable storage medium and use one or more processors to execute the instructions in hardware to perform the technology provided in the embodiments of this application.

[0068] The encoder 200 and decoder 300 can process based on the following video codec standards: H.263, H.264, H.265 (also known as High Efficiency Video Coding, HEVC), H.266 (also known as Versatile Video Coding, VVC), Moving Picture Experts Group 2 (MPEG-2), MPEG-4, VP8, VP9, ​​Alliance for Open Media Video 1 (AV1), Audio Video Coding Standard 1 (AVS1), AVS2, AVS3, or next-generation video standard protocols. This application embodiment does not specifically limit the specific implementation.

[0069] Typically, encoder 200 and decoder 300 can perform block-based encoding and decoding of images. The term "block" generally refers to a structure that includes data to be processed (e.g., encoded, decoded, or otherwise used during encoding or decoding). For example, a block can include a two-dimensional matrix of samples of luminance or chrominance data. For example, encoder 200 and decoder 300 can encode and decode video data represented in YUV format.

[0070] Referring to Figure 2, which is a schematic diagram of the encoder 200 provided in an embodiment of this application, the encoder 200 can be the encoder 200 in Figure 1. In the example of Figure 2, the encoder 200 includes a memory 201, an encoding parameter determination unit 210, a residual generation unit 202, a transform processing unit 203, a quantization unit 204, an inverse quantization unit 205, an inverse transform processing unit 206, a reconstruction unit 207, a filter unit 208, a decoded picture buffer (DPB) 209, and an entropy encoding unit 220.

[0071] The memory 201 can store video data to be encoded. For example, the encoder 200 can receive and store video data from the data source 104 shown in Figure 1. In some examples, the memory 201 can be on the same chip as other components of the encoder 200 (as shown in Figure 2), or it can be on a separate chip from those components.

[0072] The coding parameter determination unit 210 includes a mode selection unit 211, an inter-frame prediction unit 212, and an intra-frame prediction unit 213. The inter-frame prediction unit 212 is used to obtain a first prediction block for the current block using an inter-frame prediction mode. The intra-frame prediction unit 213 is used to obtain a second prediction block for the current block using an intra-frame prediction mode. The mode selection unit 211 is used to obtain a target prediction block based on the first and second prediction blocks and determine the final prediction mode. Furthermore, the coding parameter determination unit 210 may also include other functional units, such as functional units for determining the partitioning method of coding units (CUs), functional units for determining the transformation type of the residual data of the CUs, or functional units for determining the quantization parameters of the residual data of the CUs.

[0073] For ease of description and understanding, in the embodiments of this application, the CU to be processed in the current image is referred to as the current CU, and the image block to be processed in the current CU is referred to as the current block or the image block to be processed. For example, in encoding, it refers to the block currently being encoded; in decoding, it refers to the block currently being decoded.

[0074] Inter-frame prediction unit 212 may include a motion estimation unit and a motion compensation unit. For inter-frame prediction of the current block, the motion estimation unit may perform a motion search to identify one or more matching reference blocks in one or more reference pictures (e.g., one or more previously encoded / decoded pictures stored in DPB 209).

[0075] The motion estimation unit can generate one or more motion vectors (MVs) representing the position of a reference block in a reference image relative to the position of the current block in the current image. The motion compensation unit can then use interpolation to obtain a predicted value with the precision indicated by the motion vectors.

[0076] The encoding parameter determination unit 210 can provide the target prediction block to the residual generation unit 202. The residual generation unit 202 receives the raw uncoded video data of the current block from the memory 201 and calculates the residual between the current block and the target prediction block to obtain the residual block. In some examples, the function of the residual generation unit 202 can be implemented using one or more subtractor circuits that perform binary subtraction.

[0077] As an example, the encoding parameter determination unit 210 can provide the entropy encoding unit 220 with syntax elements representing encoding parameters for encoding. The encoding parameters include one or more of the following: the partitioning method of the CU, the final prediction mode, the transformation type of the residual data of the CU, or the quantization parameters of the residual data of the CU.

[0078] The transformation processing unit 203 transforms the residual block output by the residual generation unit 202 to obtain a transform coefficient block. This transformation may include Discrete Cosine Transform (DCT), integer transformation, direction transformation, or Karhunen-Loeve transformation, etc. In some examples, the encoder 200 may not include the transformation processing unit 203.

[0079] Quantization unit 204 can quantize the transform coefficients in the transform coefficient block according to the quantization parameter (QP) value associated with the current block to generate a quantized transform coefficient block.

[0080] The inverse quantization unit 205 and the inverse transform processing unit 206 can perform inverse quantization and inverse transform on the transform coefficient block, respectively, to obtain the reconstructed residual block. The reconstruction unit 207 can generate a reconstructed block corresponding to the current block based on the reconstructed residual block and the target prediction block generated by the coding parameter determination unit 210.

[0081] Filter unit 208 can perform one or more filter operations on the reconstructed block. For example, filter unit 208 can be a deblocking filter (DBF), an adaptive loop filter (ALF), a sample adaptive offset (SAO) filter, etc. In some examples, encoder 200 may not include filter unit 208.

[0082] Encoder 200 stores the reconstructed image obtained from the reconstructed blocks in DPB 209. For example, in an example where the operation of filter unit 208 is not required, reconstruction unit 207 can store the reconstructed blocks in DPB 209. In an example where the operation of filter unit 208 is required, filter unit 208 can store the filtered reconstructed blocks in DPB 209. Inter-frame prediction unit 212 retrieves the reconstructed image from DPB 209 to perform inter-frame prediction on blocks of subsequent images to be encoded. In some examples, DPB 209 can be replaced with other types of memory.

[0083] Entropy coding unit 220 can entropy code the syntax elements of other components in encoder 200 to output encoded video data. For example, entropy coding unit 220 can entropy code the quantized transform coefficient block from quantization unit 204. As another example, entropy coding unit 220 can entropy code the syntax elements (e.g., motion information for inter-frame prediction or intra-frame mode information for intra-frame prediction) from coding parameter determination unit 210.

[0084] It is understood that the composition of the encoder 200 shown in Figure 2 is only illustrative and does not constitute a limitation on the embodiments of this application.

[0085] Figure 3 is a schematic diagram of the structure of the decoder 300 provided in an embodiment of this application. The decoder 300 can be the decoder 300 described in Figure 1. In the example of Figure 3, the decoder 300 includes a coded picture buffer (CPB) 301, an entropy decoding unit 302, a prediction processing unit 310, an inverse quantization unit 303, an inverse transform processing unit 304, a reconstruction unit 305, a filter unit 306, and a DPB 307.

[0086] The entropy decoding unit 302 can receive encoded video data from the CPB 301 and perform entropy decoding on the video data to obtain syntax elements. The syntax elements indicate encoding parameters, including one or more of the following: CU partitioning method, final prediction mode, transformation type of CU residual data, or quantization parameters of CU residual data.

[0087] When the syntax element includes the final prediction mode, the prediction processing unit 310 obtains the final prediction mode. If the final prediction mode is an inter-frame prediction mode, the prediction block of the current CU can be obtained through the inter-frame prediction unit 311 of the prediction processing unit 310; if the final prediction mode is an intra-frame prediction mode, the prediction block of the current CU can be obtained through the intra-frame prediction unit 312 of the prediction processing unit 310. In some examples, the prediction processing unit 310 may also include a unit for performing prediction functions according to other prediction modes.

[0088] CPB 301 can acquire and store encoded video data from the communication medium 120 shown in Figure 1. DPB 307 is used to store decoded images. Optionally, CPB 301 and DPB 307 can be replaced with other types of memory, which are not specifically limited in this application. In some examples, CPB 301 can be on the same chip as other components of decoder 300 (as shown in the figure), or it can be on a separate chip from those components.

[0089] Decoder 300 can perform reconstruction operations on each block individually. Entropy decoding unit 302 can entropy decode the syntax elements and transform information (e.g., QP or transform mode indication) of the quantized transform coefficients to obtain the quantized transform coefficients. Dequantization unit 303 dequantizes the quantized transform coefficients to obtain a transform coefficient block including the transform coefficients. Inverse transform processing unit 304 performs an inverse transform on the transform coefficient block to generate a residual block corresponding to the current block; this inverse transform is the reverse operation of the above transform.

[0090] Reconstruction unit 305 can reconstruct the current block based on the prediction block and the residual block. For example, reconstruction unit 305 can add samples from the residual block to the corresponding samples from the prediction block to reconstruct the current block.

[0091] Filter unit 306 can perform one or more filter operations on the reconstructed block. For example, the type of filter unit 306 can be referenced to the type of filter unit 208, and will not be described again here. In some examples, the operations of filter unit 306 can be skipped.

[0092] Decoder 300 can store the reconstructed image obtained from the reconstructed blocks in DPB 307. For example, in an example where filter unit 306 is not operated, reconstruction unit 305 can store the reconstructed blocks in DPB 307. In an example where filter unit 306 is operated, filter unit 306 can store the filtered reconstructed blocks in DPB 307. Decoder 300 can output the decoded image (e.g., decoded video) from DPB 307 for subsequent rendering on a display device (such as display device 114 of FIG. 1).

[0093] The video encoder employs a block-based hybrid coding framework. The coding process includes: block partitioning, intra-frame prediction, inter-frame prediction, transform, quantization, loop filtering, and entropy coding. The encoder first divides the image into non-overlapping Coding Tree Units (CTUs), further subdividing them into different Coding Units (CUs) according to a quadtree structure. It then traverses various tree types for partitioning, ultimately determining the final partitioning result based on rate-distortion cost. The encoder uses intra-frame prediction technology based on image texture correlation. Using the reconstructed samples in the row above and column to the left adjacent to the current Coding Unit as references, it traverses Direct Current (DC) mode, Planar mode, and angle prediction mode, utilizing rate-distortion cost to select the optimal intra-frame prediction mode to remove spatial redundancy within the block.

[0094] The latest video compression standard, VVC, includes 65 angular prediction modes, Planar mode, and DC mode for intra-frame prediction, as shown in Figure 4. VVC supports multiple reference rows. In addition to the adjacent top row and left column of reconstructed samples, it also includes a total of 6 reference rows with index values ​​{1, 3, 5, 7, 12}, as shown in Figure 5. The index values ​​indicate the distance of the reference row from the current coding unit.

[0095] The MPEG-WG5 JVET established the Exploration Experiment on Enhanced Compression beyond VVC capability (EE2), which introduces new intra-prediction methods into the existing reference model, Enhanced Compression Model (ECM), such as template-based multiple reference line intra-prediction (TMRL) and matrix-based intra-prediction (PDP).

[0096] Technology 1: TMRL

[0097] The steps for encoding are as follows:

[0098] (1) Establish an intra-prediction mode candidate list, which includes: the available intra-prediction modes of the five adjacent prediction units of the current coding unit as shown in Figure 6, DC mode, decoder side intra-mode derivation (DIMD) mode, and angle prediction modes already in the list with offsets from ±1 to ±4, until 10 candidate modes are filled.

[0099] (2) Combine 10 intra-frame prediction modes and 5 reference lines {1,3,5,7,12} to create up to 50 combinations;

[0100] (3) Take the top row and left column of the current coding unit as the template (as shown in Figure 7), use 50 combinations to obtain the predicted value of the template, and calculate the sum of absolute differences between the predicted value and the reconstructed value (Sum of Absolute Differences, SAD).

[0101] (4) Sort the 50 combinations in descending order of SAD value to obtain the top 20 combinations;

[0102] (5) Compare the obtained 20 combinations with other prediction modes using the sum of absolute transformed differences (SATD) and / or rate distortion cost. If the optimal prediction mode of the current coding unit is one of these 20 combinations, then encode the TMRL identifier (value 1) and encode the index value of the selected combination to transmit to the decoding end.

[0103] After the decoder determines that the current coding unit to be decoded is in TMRL mode, it parses the index value of the combination. Following the same method as the encoder, it builds an intra-prediction mode candidate list, which is then combined with 5 multi-reference rows to form 50 combinations. Using a template-based method, the 50 combinations are sorted in descending order of SAD values ​​to obtain the top 20 combinations. The combination for the current coding unit is derived from the parsed index value, and this combination is used to obtain the prediction value for the current coding unit. The residual information of the current coding unit is parsed to obtain the residual value, which is then added to the prediction value to obtain the reconstructed value for the current coding unit.

[0104] Technique 2: Matrix-based intra prediction (PDP)

[0105] Using the reconstructed reference sample r of the top row and left column adjacent to the current block (W×H) as a template, the corresponding weight matrix F(x,y) is selected according to the shape of the current block (i.e., width W and height H) and the intra-frame prediction mode, and the predicted value P(x,y) of the current block is calculated.

[0106] Specifically, an L-shaped template as shown in Figure 8 is used. The template consists of the upper (TW+T2)×T1 reconstruction reference sample adjacent to the current block and the adjacent left T2×(TH+T1) reconstruction reference sample.

[0107] For blocks with width and height both less than or equal to 16, T1 and T2 are equal to 2, and are only used when the intra prediction mode (ipm) index value is 0, 1, or (2+2×k), where k is a natural number;

[0108] For other block sizes, T1 and T2 are equal to 1, and are only used when ipm is 0, 1, and (2+4×k), where k is a natural number;

[0109] For IPM greater than or equal to 19 and less than or equal to 49, TW = W, TH = H;

[0110] When IPM is less than 19 or greater than 49, TW = 2 × W, TH = 2 × H.

[0111] After determining the template size and available IPM values, the corresponding weight matrix F(x,y) is obtained based on the current block shape (i.e., width W and height H) and the intra-frame prediction mode. The predicted value P(x,y) of the current block is then calculated using the following formula, where k is the index of the reference sample in the template:

[0112] The video encoding and decoding methods provided in this application are described below with reference to the accompanying drawings. The video encoding method provided in this application can be executed by an encoding end, such as the encoder 200 shown in Figure 1 or Figure 2. The video decoding method provided in this application can be executed by a decoding end, such as the decoder 300 shown in Figure 1 or Figure 3. The encoding end and decoding end can be implemented by software, hardware, or a combination thereof. When implemented by hardware, the encoding end can be referred to as an encoding end device or a video encoding device, and the decoding end can be referred to as a decoding end device or a video decoding device.

[0113] Referring to Figure 9, which is a flowchart of a video decoding processing method provided in an embodiment of this application, the method includes the following steps:

[0114] Step 11: The decoding end obtains the reconstructed reference sample corresponding to the first target block based on the multi-reference row information of the first target block;

[0115] Step 12: The decoding end determines the predicted value of the first target block based on the reconstructed reference sample;

[0116] Step 13: The decoding end performs decoding processing based on the predicted value.

[0117] The multi-reference row information may include a multi-reference index (multiRefIdx). The multi-reference row information can be a value from the multi-reference row candidate list {1,3,5,7,12}.

[0118] The first target block can be understood or replaced as the current block, or the image block to be processed, or the CU to be processed in the current image.

[0119] The step of obtaining the reconstructed reference sample corresponding to the first target block based on the multi-reference row information of the first target block may include: determining a second target block based on the multi-reference row information of the first target block and the shape information of the first target block, and obtaining the reconstructed reference sample corresponding to the first target block based on the second target block; or, the reconstructed reference sample corresponding to the first target block is a reconstructed reference sample of a template region, which consists of a reconstructed reference sample of a preset size that is a distance of multiRefIdx from the top row of the first target block, and a reconstructed reference sample of a preset size that is a distance of multiRefIdx from the leftmost column of the first target block, where multiRefIdx is the multi-reference row index.

[0120] The step of determining the predicted value of the first target block based on the reconstructed reference sample may include: determining a weight matrix, and determining the predicted value of the first target block based on the weight matrix and the reconstructed reference sample; for example, the weight matrix may be determined based on the shape information of the second target block and the intra-frame prediction mode of the first target block, and the predicted value of the first target block may be determined based on the weight matrix and the reconstructed reference sample.

[0121] The decoding process performed by the decoding end based on the predicted value can refer to the decoding end performing intra-frame prediction decoding based on the predicted value.

[0122] The reconstruction reference sample corresponding to the first target block may include reconstruction reference samples of regions that are not adjacent to the first target block.

[0123] It should be noted that the multi-reference row information of the first target block can be the multi-reference row information associated with the intra-prediction mode that minimizes the rate-distortion cost. The encoder can calculate the prediction value of the first target block for each combination in the TMRL combination list, and then calculate the rate-distortion cost of the first target block for each combination. The rate-distortion cost of the first target block for each combination is compared with the rate-distortion cost of the first target block obtained using the intra-prediction modes of the adjacent templates of the first target block. The intra-prediction mode with the minimum rate-distortion cost is selected as the optimal intra-prediction mode for the first target block. The multi-reference row information associated with this optimal intra-prediction mode is then determined as the multi-reference row information of the first target block. This allows the selection of reference samples with higher relevance to the current block to obtain prediction values, thereby improving prediction accuracy and ultimately increasing coding efficiency.

[0124] In related techniques, PDP uses reconstructed reference samples adjacent to the current block and obtains the predicted value through operations with the weight matrix. However, in natural images, the current block may have a greater correlation with non-adjacent regions than with adjacent regions. Therefore, the predicted value obtained by PDP in related techniques is not necessarily accurate.

[0125] This application's embodiments overcome the limitations of related technologies that only use adjacent templates by obtaining templates composed of reconstructed reference sample points from non-adjacent reference rows and columns. When using PDP mode for the current block, predicted values ​​can be obtained by selecting reference samples with higher correlation to the current block from both adjacent and non-adjacent reference samples, thereby improving prediction accuracy and thus enhancing coding efficiency.

[0126] In this embodiment, the decoding end obtains the reconstructed reference sample corresponding to the first target block based on the multi-reference row information of the first target block; the decoding end determines the predicted value of the first target block based on the reconstructed reference sample; and the decoding end performs decoding processing based on the predicted value. Thus, obtaining the reconstructed reference sample corresponding to the first target block based on the multi-reference row information of the first target block for prediction and decoding is more flexible than directly using the reconstructed reference sample adjacent to the current block (i.e., the first target block) for intra-frame prediction, and can improve the effect of removing spatial redundancy of blocks.

[0127] Optionally, obtaining the reconstructed reference sample corresponding to the first target block based on the multi-reference row information of the first target block includes:

[0128] The second target block is determined based on the multi-reference row information of the first target block and the shape information of the first target block;

[0129] Based on the second target block, obtain the reconstruction reference sample corresponding to the first target block.

[0130] The shape information of the first target block may include its width and height values. For example, the shape information of the first target block may be one of the following lists: {4×4, 4×8, 8×4, 8×8, 4×16, 16×4, 8×16, 16×8, 16×16, 16×32, 32×16, 32×32}. This list is a W×H list, where W is the width value and H is the height value.

[0131] In one embodiment, the region where the second target block is located includes the region where the first target block is located and is larger than the region where the first target block is located, so that the determined reconstruction reference sample includes reconstruction reference samples of regions that are not adjacent to the first target block, which can improve prediction accuracy and thus improve encoding and decoding efficiency.

[0132] In one implementation, the shape information (W'×H') of the second target block satisfies: W′≥W+multiRefIdx, H′≥H+multiRefIdx, and W′,H′∈{4,8,16,32}. The shape information of the first target block is W×H, and the multi-reference row information of the first target block is multiRefIdx.

[0133] It should be noted that the shape information of the second target block can be obtained according to the mapping relationship in Table 1; or the shape information of the second target block can be obtained through other mapping relationships; or the shape information of the second target block can be obtained by calculation, for example, W′=W+multiRefIdx, H′=H+multiRefIdx; this embodiment does not limit the specific implementation method of obtaining the shape information of the second target block.

[0134] Table 1

[0135] The step of obtaining the reconstruction reference sample corresponding to the first target block based on the second target block may include: determining a template region based on the shape information of the second target block and the intra-frame prediction mode of the first target block, wherein the reconstruction reference sample corresponding to the first target block is the reconstruction reference sample of the template region. The template region may include at least one of a first region, a second region, and a third region. For example, the template region may include a first region, a second region, and a third region; or, the template region may include a first region and a second region. The first region is adjacent to the second target block and located in a first orientation of the second target block; the second region is adjacent to the second target block and located in a second orientation of the second target block; and the third region is adjacent to both the first region and the second region and located in a third orientation of the second target block.

[0136] In this embodiment, a second target block is determined based on the multi-reference row information of the first target block and the shape information of the first target block. A reconstructed reference sample corresponding to the first target block is obtained based on the second target block. Compared to directly using a reconstructed reference sample adjacent to the current block (i.e., the first target block) for intra-frame prediction, this method offers greater flexibility, improves the removal of spatial redundancy in blocks, and thus enhances encoding / decoding efficiency. Furthermore, the shape information of the first target block is combined to determine the second target block used to obtain the reconstructed reference sample. This facilitates the decoding and encoding ends in locating the second target block based on the first target block, ensuring that the second target block determined during decoding is consistent with the second target block used during encoding, thereby improving the accuracy of intra-frame prediction decoding.

[0137] Optionally, determining the predicted value of the first target block based on the reconstructed reference sample includes:

[0138] The weight matrix is ​​determined based on the shape information of the second target block and the intra-frame prediction mode of the first target block;

[0139] The predicted value of the first target block is determined based on the weight matrix and the reconstruction reference sample.

[0140] Specifically, based on the shape information (W'×H') of the second target block, the sizeKey (sizeKey = (W′<<8)+H′) is calculated. The sizeIdx corresponding to the sizeKey is then obtained by looking up the table. Next, the modeIdx is obtained by looking up the table based on the intra-frame prediction mode. Finally, the weight matrix F = LUT[modeIdx][sizeIdx] is obtained based on sizeIdx and modeIdx. The method for determining the weight matrix can refer to existing techniques and will not be elaborated here.

[0141] In one implementation, the predicted value P(x,y) of the first target block can be determined using the following formula:

[0142] Where F(x′,y′) is the weight matrix, r(k) is the reconstruction reference sample, (x,y) is the sample point position information in the first target block, and k is the index of the reconstruction reference sample corresponding to the first target block. The shape information of the first target block is W×H, and the multi-reference row information of the first target block is multiRefIdx.

[0143] The range of x′ is: Sx~Ex; optional, Sx=multiRefIdx, Ex=multiRefIdx+W;

[0144] The range of y′ is Sy to Ey; optional, Sy = multiRefIdx, Ey = multiRefIdx + H;

[0145] The range of r is: Sr ~ Er; optional, Sr = multiRefIdx, Er = multiRefIdx + W, or Er = multiRefIdx + H; or Sr = 0, Er = n, where n is the number of template samples.

[0146] In this embodiment, a weight matrix is ​​determined based on the shape information of the second target block and the intra-frame prediction mode of the first target block. The predicted value of the first target block is then determined based on the weight matrix and the reconstructed reference sample. By determining the weight matrix based on the shape information of the second target block, the determined weight matrix corresponds to the reconstructed reference sample determined based on the second target block, thereby improving the accuracy of the obtained predicted value.

[0147] Optionally, obtaining the reconstruction reference sample corresponding to the first target block based on the second target block includes:

[0148] Based on the shape information of the second target block and the intra-frame prediction mode of the first target block, a template region is determined. The template region includes at least one of a first region, a second region, and a third region. The first region is adjacent to the second target block and located in a first position of the second target block. The second region is adjacent to the second target block and located in a second position of the second target block. The third region is adjacent to both the first region and the second region and located in a third position of the second target block.

[0149] Wherein, the reconstruction reference sample corresponding to the first target block is the reconstruction reference sample of the template region.

[0150] Specifically, the size of the template region can be determined based on the shape information (W'×H') of the second target block and the intra-frame prediction mode (ipm) of the first target block, and the value r of the reconstructed reference sample can be filled into the template region.

[0151] In one embodiment, as shown in FIG10, the first orientation is above, and the first region is a region with a size of TW×T1 that is adjacent to and above the second target block.

[0152] In one embodiment, as shown in FIG10, the second orientation is to the left, and the second region is a region with a size of T2×TH that is adjacent to the second target block and located to the left of the second target block.

[0153] In one embodiment, as shown in FIG10, the third region is the upper left corner, the third region is adjacent to both the first region and the second region, and is located at the upper left corner of the second target block, with a size of T2×T1.

[0154] In one implementation, the first region, the second region, and the third region constitute an L-shaped region.

[0155] For example, as shown in Figure 10, the template region consists of a reconstructed reference sample with a size of (TW+T2)×T1 and a distance of multiRefIdx from the top row of the first target block, and a reconstructed reference sample with a size of T2×(TH+T1) and a distance of multiRefIdx from the leftmost column of the first target block.

[0156] In this embodiment, a template region is determined based on the shape information of the second target block and the intra-frame prediction mode of the first target block. The template region includes at least one of a first region, a second region, and a third region. The reconstruction reference sample corresponding to the first target block is the reconstruction reference sample of the template region. In this way, the template region is determined by the shape information of the second target block, instead of directly using the adjacent regions of the first target block to combine the template region. This allows the template region to be composed of reconstruction reference samples that are not adjacent to the current block (i.e., the first target block). Using the reconstruction reference samples of the non-adjacent regions of the current block (i.e., the first target block) for intra-frame prediction can support the encoder to consider the non-adjacent regions of the current block (i.e., the first target block) as template regions for intra-frame prediction, thereby improving prediction accuracy and thus improving encoding and decoding efficiency.

[0157] Optionally, the template area is an L-shaped area formed by the first area, the second area, and the third area.

[0158] In this way, by using the L-shaped region formed by the first region, the second region, and the third region as the template region, a larger range of template regions can be obtained. By using a larger number of reconstruction reference samples for intra-frame prediction, the prediction accuracy can be improved.

[0159] Optionally, if the width and height values ​​of the second target block are both less than or equal to the first size threshold, the height value of the first region and the width value of the second region are both equal to the first size value, and the width and height values ​​of the third region are both equal to the first size value;

[0160] If either the width or height of the second target block is greater than the first size threshold, the height of the first region and the width of the second region are both equal to the second size value, and the width and height of the third region are both equal to the second size value.

[0161] When the index value of the intra-prediction mode is greater than or equal to the first mode index threshold and less than or equal to the second mode index threshold, the width value of the first region is the same as the width value of the second target block, the height value of the second region is the same as the height value of the second target block, and the first mode index threshold is less than the second mode index threshold.

[0162] If the index value of the intra-prediction mode is less than the first mode index threshold or greater than the second mode index threshold, the width of the first region is twice the width of the second target block, and the height of the second region is twice the height of the second target block.

[0163] The shape information of the second target block includes its width and height.

[0164] In this embodiment, the first size threshold can be a preset value, for example, 16, 18, or 20, etc., and this embodiment does not limit this. The first size value can be a preset value, for example, 2, 3, or 4, etc., and this embodiment does not limit this. The second size value can be a preset value, for example, 1, 2, or 3, etc., and this embodiment does not limit this. The second size value can be smaller than the first size value. The first mode index threshold can be a preset value, for example, 18, 19, or 20, etc., and this embodiment does not limit this. The second mode index threshold can be a preset value, for example, 48, 49, or 50, etc., and this embodiment does not limit this.

[0165] In one embodiment, the template region satisfies at least one of the following conditions:

[0166] When the width and height of the second target block are both less than or equal to 16, the height of the first region and the width of the second region are both equal to 2, and the width and height of the third region are both equal to 2.

[0167] If either the width or height of the second target block is greater than 16, the height of the first region and the width of the second region are both equal to 1, and the width and height of the third region are both equal to 1.

[0168] When the index value of the intra-prediction mode is greater than or equal to 19 and less than or equal to 49, the width value of the first region is the same as the width value of the second target block, and the height value of the second region is the same as the height value of the second target block.

[0169] When the index value of the intra-prediction mode is less than 19 or greater than 49, the width of the first region is twice the width of the second target block, and the height of the second region is twice the height of the second target block.

[0170] The shape information of the second target block includes its width and height.

[0171] In one implementation, when the width and height values ​​of the second target block are both less than or equal to 16, the height value of the first region and the width value of the second region are both equal to 2, and the width and height values ​​of the third region are both equal to 2, and this is only used when the intra-prediction mode (ipm) index value is 0, 1, or (2+2×k), where k is a natural number.

[0172] In one implementation, if either the width or height of the second target block is greater than 16, the height of the first region and the width of the second region are both equal to 1, and the width and height of the third region are both equal to 1. This is used only when the intra-frame prediction mode IPM is 0, 1, or (2+4×k), where k is a natural number.

[0173] In one embodiment, when the index value of the intra-frame prediction mode is greater than or equal to 0 and less than 19, or greater than 49 and less than or equal to 66, the width of the first region is twice the width of the second target block, and the height of the second region is twice the height of the second target block.

[0174] It should be noted that "equal to" in the above conditions can be the same as "greater than" or "less than", and this embodiment does not limit this. For example, if one condition is that the width and height of the second target block are both less than or equal to 16, then the other condition is that the width and height of the second target block are both greater than 16; or, if one condition is that the width and height of the second target block are both less than 16, then the other condition is that the width and height of the second target block are both greater than or equal to 16.

[0175] In this embodiment, the template region can be determined by the above conditions, thereby enabling the decoding end and the encoding end to determine the template region by consistent conditions. Intra-frame prediction can be performed by reconstructing reference samples of the determined template region, which can improve the prediction accuracy.

[0176] Optionally, the method further includes:

[0177] Obtain template-based multi-reference row intra-prediction mode (TMRL) combination information corresponding to the first target block. The TMRL combination information includes multi-reference row information and intra-prediction mode.

[0178] The multi-reference line information of the first target block and the intra-frame prediction mode of the first target block are determined based on the TMRL combination information corresponding to the first target block.

[0179] The step of obtaining the TMRL combination information corresponding to the first target block may include: parsing to obtain the index of the TMRL combination information used by the first target block, and obtaining the TMRL combination information corresponding to the first target block based on the index of the TMRL combination information. The decoding end can obtain a TMRL combination list, which includes multiple TMRL combination information. The TMRL combination information corresponding to the first target block can be determined from these multiple TMRL combination information based on the index of the TMRL combination information used by the first target block.

[0180] It should be noted that the TMRL combination list consists of multiple TMRL combination information, and this embodiment does not limit the specific form of the TMRL combination list. For example, the combination in the TMRL combination list is: {intra-prediction mode, upper multi-reference line information (multiRefIdxAbove), left multi-reference line information (multiRefIdxLeft)}, where the upper multi-reference line information is not equal to the left multi-reference line information (i.e., multiRefIdxLeft and multiRefIdxAbove are not equal); or, the combination in the TMRL combination list is: {intra-prediction mode, multi-reference line information}; or, the combination in the TMRL combination list is: {ipm+offset, multi-reference line information}, {ipm-offset, multi-reference line information}, where ipm is the intra-prediction mode index value corresponding to the first N PDP modes in the TMRL combination list obtained by sorting the adjacent templates of the current block according to the relevant technology, and the offset value offset can be a positive integer such as 2 or 1, which is not limited in this embodiment.

[0181] In one embodiment, the multi-reference row information of the first target block is the multi-reference row information in the TMRL combination information corresponding to the first target block.

[0182] In one embodiment, when the multi-reference row information in the TMRL combination information includes the index value of the first orientation and the index value of the second orientation, and the index value of the first orientation and the index value of the second orientation are different, the multi-reference row information of the first target block is the maximum or minimum value of the index value of the first orientation and the index value of the second orientation.

[0183] In one embodiment, the intra-prediction mode of the first target block is the intra-prediction mode in the TMRL combination information corresponding to the first target block.

[0184] In one implementation, the intra-prediction mode in the TMRL combination information corresponding to the first target block can be converted into a low-precision intra-prediction mode, and the intra-prediction mode of the first target block is the low-precision intra-prediction mode.

[0185] For example, if IPM H <2, ipm=ipm H ,otherwise, ipm H is the intra-prediction mode in the TMRL combination information corresponding to the first target block, and ipm is the low-precision intra-prediction mode obtained by conversion.

[0186] In this embodiment, TMRL combination information corresponding to the first target block is obtained. The TMRL combination information includes multiple reference line information and intra-prediction mode. Based on the TMRL combination information corresponding to the first target block, the multiple reference line information and intra-prediction mode of the first target block are determined. Thus, by determining the multiple reference line information and intra-prediction mode of the first target block through TMRL combination information, and combining TMRL and PDP technologies for intra-prediction, prediction accuracy can be improved.

[0187] Optionally, if the multi-reference row information in the TMRL combination information includes the index value of the first orientation and the index value of the second orientation, and the index value of the first orientation and the index value of the second orientation are different, the multi-reference row information of the first target block is the maximum value of the index value of the first orientation and the index value of the second orientation.

[0188] The first orientation can be upward, and the index value of the first orientation can be considered as the multi-reference row information above (e.g., multiRefIdxAbove). The second orientation can be left, and the index value of the second orientation can be considered as the multi-reference row information to the left (e.g., multiRefIdxLeft).

[0189] This implementation provides a method for determining multi-reference row information of a first target block when using asymmetric multi-reference rows.

[0190] In this embodiment, when the multi-reference row information in the TMRL combination information includes the index value of the first orientation and the index value of the second orientation, and the index value of the first orientation and the index value of the second orientation are different, the multi-reference row information of the first target block is the maximum value of the index value of the first orientation and the index value of the second orientation. The first target block is extended with the maximum value as the multi-reference row information of the first target block to obtain the second target block. The template region is determined based on the adjacent region of the second target block. This allows for the acquisition of reconstructed reference samples of the template region farther away from the first target block for intra-frame prediction, thereby improving the intra-frame prediction effect.

[0191] This application proposes a method for increasing candidate templates, namely, using reconstructed reference samples from non-adjacent regions of the current block as templates, in addition to existing adjacent templates, and a corresponding weight matrix selection method. This application enables the PDP to select the template most relevant to the current block, thereby obtaining more accurate predictions and improving coding efficiency.

[0192] The following examples will provide further explanation:

[0193] The methods in Examples 1, 2, and 3 can be executed by either the encoding end or the decoding end.

[0194] Example 1:

[0195] Example 1 illustrates how to select a weight matrix based on information from multiple reference rows.

[0196] Step (1): Obtain the multi-reference index (multiRefIdx) used by the current block (i.e., the first target block) and the shape information (W×H) of the current block;

[0197] Step (2): Obtain the shape information (W'×H') of the mapped block (i.e., the second target block) based on the multi-reference row information and the shape information of the current block;

[0198] The multiref row information multiRefIdx is a value in the multiref row candidate list {1,3,5,7,12}, and the shape information of the current block is a value in {4×4,4×8,8×4,8×8,4×16,16×4,8×16,16×8,16×16,16×32,32×16,32×32}.

[0199] Optionally, one implementation method is to obtain the shape information of the mapping block according to Table 1 above. Alternatively, the shape information of the mapping block can be obtained through other mapping relationships, which is not limited here. The shape information of the mapping block needs to satisfy: W′≥W+multiRefIdx, H′≥H+multiRefIdx, and W′,H′∈{4,8,16,32}.

[0200] Step (3): Select the corresponding weight matrix F based on the shape of the current block and the intra-prediction mode. The method for selecting the weight matrix can be the same as the existing technology. First, calculate the sizeKey (sizeKey = (W′<<8) + H′) based on the width and height of the mapped block, look up the sizeIdx corresponding to the sizeKey in the table, then look up the modeIdx in the table based on the intra-prediction mode, and finally obtain the weight matrix F = LUT[modeIdx][sizeIdx] based on sizeIdx and modeIdx.

[0201] Example 2:

[0202] This example describes how to obtain the current block prediction value based on multiple reference row information.

[0203] Step (1): Obtain the multi-reference row information (multiRefIdx) used by the current block and the shape information (W×H) of the current block;

[0204] Step (2): Obtain the mapped block shape information (W'×H') based on the reference row information and the current block shape information;

[0205] Step (3): Select the corresponding weight matrix F according to the shape of the current block and the intra-prediction mode (ipm); the method for obtaining the weight matrix can be referred to Example 1, and will not be repeated here.

[0206] Step (4): Obtain the reference sample value of the template based on the multi-reference row information used by the current block and the shape information of the mapped block.

[0207] Based on the shape information of the mapped block (width W' and height H') and the intra-frame prediction mode information (ipm) of the current block, the template size can be obtained and filled with the reconstructed sample value r. As shown in Figure 10, the template consists of a reconstructed reference sample of size (TW+T2)×T1 at a distance of multiRefIdx from the top row of the current block, and a reconstructed reference sample of size T2×(TH+T1) at a distance of multiRefIdx from the leftmost column of the current block.

[0208] It should be noted that (TW+T2)×T1 and T2×(TH+T1) are the dimensions of the template, which determine the range of values ​​for the reconstructed reference sample r(k).

[0209] For blocks with width W' and height H' both less than or equal to 16, T1 and T2 are equal to 2, and are used only when the intra prediction mode (ipm) index value is 0, 1, and (2+2×k), where k is a natural number;

[0210] For other block sizes, T1 and T2 are equal to 1, and are only used when ipm is 0, 1, and (2+4×k), where k is a natural number;

[0211] For IPM greater than or equal to 19 and less than or equal to 49, TW = W', TH = H';

[0212] When IPM is less than 19 or greater than 49, TW = 2 × W' and TH = 2 × H'.

[0213] Step (5): Based on the multi-reference row information (multiRefIdx) used in the current block, use the obtained weight matrix F to calculate the predicted value of the current block according to the following formula (1), where (x,y) is the sample point location information in the current block, and k is the reference sample index in the template.

[0214] Where x′ takes values ​​from Sx to Ex; optionally, sx = multiRefIdx, Ex = multiRefIdx + W;

[0215] The range of y′ is Sy to Ey; optional, Sy = multiRefIdx, Ey = multiRefIdx + H;

[0216] The range of r is: Sr ~ Er; optional, Sr = multiRefIdx, Er = multiRefIdx + W, or Er = multiRefIdx + H; or Sr = 0, Er = n, where n is the number of template samples.

[0217] Example 3:

[0218] Example 3 describes the implementation schemes of Examples 1 and 2 in TMRL.

[0219] Step (1): Obtain the TMRL combination list, where the combination in the TMRL combination list is a combination of intra-prediction mode information and multi-reference line information, and the list includes N combinations; parse to obtain the TMRL combination index information used by the current block, and obtain the multi-reference line information and intra-prediction mode information used by the current block based on the index information; combine the intra-prediction mode information (ipm) with the TMRL combination index information used by the current block. H ) is converted into low-precision intra-prediction mode information (IPM), for example, if IPM H <2, ipm=ipm H ,otherwise,

[0220] For example, the method for constructing a TMRL combinatorial list is as follows:

[0221] Optionally, the combination in the TMRL combination list is: {intra-frame prediction mode, upper multi-reference line information (multiRefIdxAbove), left multi-reference line information (multiRefIdxLeft)}, where the upper multi-reference line information is not equal to the left multi-reference line information (i.e., multiRefIdxLeft and multiRefIdxAbove are not equal); in this case, the range of values ​​for the parameters in the above formula (1) is modified accordingly:

[0222] The range of x′ is: Sx~Ex; optional, Sx=multiRefIdxLeft, Ex=multiRefIdxLeft+W;

[0223] The range of y′ is Sy to Ey; optional, Sy = multiRefIdxAbove, Ey = multiRefIdxAbove + h.

[0224] Optionally, the combination in the TMRL combination list is: {ipm+offset, multiple reference line information}, {ipm-offset, multiple reference line information}, where ipm is the intra-prediction mode index value corresponding to the first N PDP modes in the TMRL combination list obtained by sorting the adjacent templates of the current block according to the relevant technology, and the offset value offset can be a positive integer such as 2 or 1, which is not restricted in this example.

[0225] It should be noted that the intra-prediction mode in the TMRL combination is high-precision and needs to be converted to low-precision intra-prediction mode before using PDP.

[0226] Step (2): Obtain the multi-reference row information used by the current block (the larger of multiRefIdxLeft and multiRefIdxAbove), and the shape information (W×H) of the current block;

[0227] Step (3): Obtain the mapped block shape information (W'×H') based on the reference row information and the current block shape information;

[0228] Step (4): Select the corresponding weight matrix F based on the current block's mapping block shape and the intra-prediction mode (ipm);

[0229] Step (5): Based on the multi-reference row information used by the current block and the shape information of the mapped block, obtain the reference sample value of the template;

[0230] Step (6): Based on the multi-reference row information used in the current block (the larger of multiRefIdxLeft and multiRefIdxAbove), use the obtained weight matrix F to calculate the predicted value of the current block according to formula (1), where (x,y) is the sample point location information in the current block and k is the reference sample index in the template.

[0231] Example 4:

[0232] Example 4 describes how the encoder selects the optimal multi-reference line PDP mode.

[0233] The encoder can calculate the current block prediction value for each combination in the TMRL combination list, following the method in Example 3, and then calculate the rate-distortion cost of the current block for each combination. This cost is compared with the rate-distortion cost obtained using the PDP mode of the adjacent templates of the current block in related techniques. The PDP mode with the lowest rate-distortion cost is selected as the optimal PDP mode for the current block. This allows for the selection of reference samples with higher relevance to the current block to obtain prediction values, thereby improving prediction accuracy and ultimately increasing coding efficiency.

[0234] This application's embodiments overcome the limitations of related technologies that only use adjacent templates of the current block by obtaining reconstructed reference sample points from non-adjacent reference rows and columns to form a template. When using PDP mode for the current block, the predicted value can be obtained by selecting reference samples with higher correlation to the current block from adjacent and non-adjacent reference samples, thereby improving prediction accuracy and thus enhancing coding efficiency.

[0235] Referring to Figure 11, which is a flowchart of a video encoding processing method provided in an embodiment of this application, the method includes the following steps:

[0236] Step 21: The encoding end obtains the reconstructed reference sample corresponding to the first target block based on the multi-reference row information of the first target block;

[0237] Step 22: The encoding end determines the predicted value of the first target block based on the reconstructed reference sample;

[0238] Step 23: The encoding end performs encoding processing based on the predicted value.

[0239] Optionally, obtaining the reconstructed reference sample corresponding to the first target block based on the multi-reference row information of the first target block includes:

[0240] The second target block is determined based on the multi-reference row information of the first target block and the shape information of the first target block;

[0241] Based on the second target block, obtain the reconstruction reference sample corresponding to the first target block.

[0242] Optionally, determining the predicted value of the first target block based on the reconstructed reference sample includes:

[0243] The weight matrix is ​​determined based on the shape information of the second target block and the intra-frame prediction mode of the first target block;

[0244] The predicted value of the first target block is determined based on the weight matrix and the reconstruction reference sample.

[0245] Optionally, obtaining the reconstruction reference sample corresponding to the first target block based on the second target block includes:

[0246] Based on the shape information of the second target block and the intra-frame prediction mode of the first target block, a template region is determined. The template region includes at least one of a first region, a second region, and a third region. The first region is adjacent to the second target block and located in a first position of the second target block. The second region is adjacent to the second target block and located in a second position of the second target block. The third region is adjacent to both the first region and the second region and located in a third position of the second target block.

[0247] Wherein, the reconstruction reference sample corresponding to the first target block is the reconstruction reference sample of the template region.

[0248] Optionally, the template area is an L-shaped area formed by the first area, the second area, and the third area.

[0249] Optionally, the template region satisfies at least one of the following conditions:

[0250] When the width and height values ​​of the second target block are both less than or equal to the first size threshold, the height value of the first region and the width value of the second region are both equal to the first size value, and the width and height values ​​of the third region are both equal to the first size value;

[0251] If either the width or height of the second target block is greater than the first size threshold, the height of the first region and the width of the second region are both equal to the second size value, and the width and height of the third region are both equal to the second size value.

[0252] When the index value of the intra-prediction mode is greater than or equal to the first mode index threshold and less than or equal to the second mode index threshold, the width value of the first region is the same as the width value of the second target block, the height value of the second region is the same as the height value of the second target block, and the first mode index threshold is less than the second mode index threshold.

[0253] If the index value of the intra-prediction mode is less than the first mode index threshold or greater than the second mode index threshold, the width of the first region is twice the width of the second target block, and the height of the second region is twice the height of the second target block.

[0254] The shape information of the second target block includes its width and height.

[0255] Optionally, the method further includes:

[0256] Obtain the TMRL combination information corresponding to the first target block, wherein the TMRL combination information includes multiple reference line information and intra-frame prediction mode;

[0257] The multi-reference line information of the first target block and the intra-frame prediction mode of the first target block are determined based on the TMRL combination information corresponding to the first target block.

[0258] Optionally, if the multi-reference row information in the TMRL combination information includes the index value of the first orientation and the index value of the second orientation, and the index value of the first orientation and the index value of the second orientation are different, the multi-reference row information of the first target block is the maximum value of the index value of the first orientation and the index value of the second orientation.

[0259] Optionally, the method further includes:

[0260] The predicted value of the first target block is calculated using multiple TMRL combination information, and the predicted value corresponding to each TMRL combination information is obtained.

[0261] Calculate the rate-distortion cost corresponding to each TMRL combination information based on the predicted value corresponding to each TMRL combination information;

[0262] The optimal intra-frame prediction mode for the first target block is determined based on the rate-distortion cost corresponding to each TMRL combination information.

[0263] The step of determining the optimal intra-prediction mode of the first target block based on the rate-distortion cost corresponding to each TMRL combination information may include: comparing the rate-distortion cost corresponding to the multiple TMRL combination information with the rate-distortion cost of the first target block obtained by using the intra-prediction mode of the adjacent template of the first target block (i.e., the intra-prediction mode determined based on the template in Figure 7), and selecting the intra-prediction mode with the minimum rate-distortion cost as the optimal intra-prediction mode of the first target block.

[0264] It should be noted that the encoder can calculate the prediction value of the first target block of each combination in multiple TMRL combination information, and then calculate the rate-distortion cost of the first target block of each combination. The rate-distortion cost of the first target block of each combination is compared with the rate-distortion cost of the first target block obtained by using the intra prediction mode of the adjacent template of the first target block. The intra prediction mode with the smallest rate-distortion cost is selected as the optimal intra prediction mode of the first target block, which can improve the prediction accuracy and thus improve the coding efficiency.

[0265] It should be noted that this embodiment is an implementation method of the encoding side corresponding to the embodiment shown in FIG9. For the specific implementation method, please refer to the relevant description of the embodiment shown in FIG9. In order to avoid repeated description, this embodiment will not be repeated, and the same beneficial effects can be achieved.

[0266] The video decoding processing method provided in this application can be executed by a video decoding processing device. As an example, the device can be an electronic device or a component within an electronic device, such as a chip or circuit. This application uses a video decoding processing device executing the video decoding processing method as an example to illustrate the video decoding processing device provided in this application.

[0267] Please refer to Figure 12, which is a structural diagram of a video decoding processing apparatus provided in an embodiment of this application. As shown in Figure 12, the video decoding processing apparatus 400 includes:

[0268] The first acquisition module 401 is used to acquire the reconstruction reference sample corresponding to the first target block based on the multi-reference row information of the first target block;

[0269] The first determining module 402 is used to determine the predicted value of the first target block based on the reconstructed reference sample;

[0270] Processing module 403 is used to perform decoding processing based on the predicted value.

[0271] Optionally, the first acquisition module 401 includes:

[0272] The determining unit is configured to determine the second target block based on the multi-reference row information of the first target block and the shape information of the first target block;

[0273] The acquisition unit is used to acquire the reconstruction reference sample corresponding to the first target block based on the second target block.

[0274] Optionally, the first determining module 402 is specifically used for:

[0275] The weight matrix is ​​determined based on the shape information of the second target block and the intra-frame prediction mode of the first target block;

[0276] The predicted value of the first target block is determined based on the weight matrix and the reconstruction reference sample.

[0277] Optionally, the acquisition unit is specifically used for:

[0278] Based on the shape information of the second target block and the intra-frame prediction mode of the first target block, a template region is determined. The template region includes at least one of a first region, a second region, and a third region. The first region is adjacent to the second target block and located in a first position of the second target block. The second region is adjacent to the second target block and located in a second position of the second target block. The third region is adjacent to both the first region and the second region and located in a third position of the second target block.

[0279] Wherein, the reconstruction reference sample corresponding to the first target block is the reconstruction reference sample of the template region.

[0280] Optionally, the template area is an L-shaped area formed by the first area, the second area, and the third area.

[0281] Optionally, the template region satisfies at least one of the following conditions:

[0282] When the width and height values ​​of the second target block are both less than or equal to the first size threshold, the height value of the first region and the width value of the second region are both equal to the first size value, and the width and height values ​​of the third region are both equal to the first size value;

[0283] If either the width or height of the second target block is greater than the first size threshold, the height of the first region and the width of the second region are both equal to the second size value, and the width and height of the third region are both equal to the second size value.

[0284] When the index value of the intra-prediction mode is greater than or equal to the first mode index threshold and less than or equal to the second mode index threshold, the width value of the first region is the same as the width value of the second target block, the height value of the second region is the same as the height value of the second target block, and the first mode index threshold is less than the second mode index threshold.

[0285] If the index value of the intra-prediction mode is less than the first mode index threshold or greater than the second mode index threshold, the width of the first region is twice the width of the second target block, and the height of the second region is twice the height of the second target block.

[0286] The shape information of the second target block includes its width and height.

[0287] Optionally, the device further includes:

[0288] The second acquisition module is used to acquire template-based multi-reference row intra-frame prediction mode (TMRL) combination information corresponding to the first target block. The TMRL combination information includes multi-reference row information and intra-frame prediction mode.

[0289] The second determining module is used to determine the multi-reference row information of the first target block and the intra-frame prediction mode of the first target block based on the TMRL combination information corresponding to the first target block.

[0290] Optionally, if the multi-reference row information in the TMRL combination information includes the index value of the first orientation and the index value of the second orientation, and the index value of the first orientation and the index value of the second orientation are different, the multi-reference row information of the first target block is the maximum value of the index value of the first orientation and the index value of the second orientation.

[0291] The embodiments of this application can improve the effect of removing spatial redundancy of blocks.

[0292] The video decoding processing apparatus provided in this application embodiment can implement the various processes implemented in the method embodiment of FIG9 and achieve the same technical effect. To avoid repetition, it will not be described again here.

[0293] The video encoding processing method provided in this application can be executed by a video encoding processing device. As an example, the device can be an electronic device or a component within an electronic device, such as a chip or circuit. This application uses a video encoding processing device executing the video encoding processing method as an example to illustrate the video encoding processing device provided in this application.

[0294] Please refer to Figure 13, which is a structural diagram of a video encoding processing apparatus provided in an embodiment of this application. As shown in Figure 13, the video encoding processing apparatus 500 includes:

[0295] The first acquisition module 501 is used to acquire the reconstruction reference sample corresponding to the first target block based on the multi-reference row information of the first target block;

[0296] The first determining module 502 is used to determine the predicted value of the first target block based on the reconstructed reference sample;

[0297] The processing module 503 is used to perform encoding processing based on the predicted value.

[0298] Optionally, the first acquisition module 501 includes:

[0299] The determining unit is configured to determine the second target block based on the multi-reference row information of the first target block and the shape information of the first target block;

[0300] The acquisition unit is used to acquire the reconstruction reference sample corresponding to the first target block based on the second target block.

[0301] Optionally, the first determining module 502 is specifically used for:

[0302] The weight matrix is ​​determined based on the shape information of the second target block and the intra-frame prediction mode of the first target block;

[0303] The predicted value of the first target block is determined based on the weight matrix and the reconstruction reference sample.

[0304] Optionally, the acquisition unit is specifically used for:

[0305] Based on the shape information of the second target block and the intra-frame prediction mode of the first target block, a template region is determined. The template region includes at least one of a first region, a second region, and a third region. The first region is adjacent to the second target block and located in a first position of the second target block. The second region is adjacent to the second target block and located in a second position of the second target block. The third region is adjacent to both the first region and the second region and located in a third position of the second target block.

[0306] Wherein, the reconstruction reference sample corresponding to the first target block is the reconstruction reference sample of the template region.

[0307] Optionally, the template area is an L-shaped area formed by the first area, the second area, and the third area.

[0308] Optionally, the template region satisfies at least one of the following conditions:

[0309] When the width and height values ​​of the second target block are both less than or equal to the first size threshold, the height value of the first region and the width value of the second region are both equal to the first size value, and the width and height values ​​of the third region are both equal to the first size value;

[0310] If either the width or height of the second target block is greater than the first size threshold, the height of the first region and the width of the second region are both equal to the second size value, and the width and height of the third region are both equal to the second size value.

[0311] When the index value of the intra-prediction mode is greater than or equal to the first mode index threshold and less than or equal to the second mode index threshold, the width value of the first region is the same as the width value of the second target block, the height value of the second region is the same as the height value of the second target block, and the first mode index threshold is less than the second mode index threshold.

[0312] If the index value of the intra-prediction mode is less than the first mode index threshold or greater than the second mode index threshold, the width of the first region is twice the width of the second target block, and the height of the second region is twice the height of the second target block.

[0313] The shape information of the second target block includes its width and height.

[0314] Optionally, the device further includes:

[0315] The second acquisition module is used to acquire TMRL combination information corresponding to the first target block, wherein the TMRL combination information includes multiple reference line information and intra-frame prediction mode.

[0316] The second determining module is used to determine the multi-reference row information of the first target block and the intra-frame prediction mode of the first target block based on the TMRL combination information corresponding to the first target block.

[0317] Optionally, if the multi-reference row information in the TMRL combination information includes the index value of the first orientation and the index value of the second orientation, and the index value of the first orientation and the index value of the second orientation are different, the multi-reference row information of the first target block is the maximum value of the index value of the first orientation and the index value of the second orientation.

[0318] Optionally, the device further includes:

[0319] The first calculation module is used to calculate the predicted value of the first target block using multiple TMRL combination information respectively, and obtain the predicted value corresponding to each TMRL combination information.

[0320] The second calculation module is used to calculate the rate-distortion cost corresponding to each TMRL combination information based on the predicted value corresponding to each TMRL combination information.

[0321] The third determining module is used to determine the optimal intra-frame prediction mode of the first target block based on the rate-distortion cost corresponding to each TMRL combination information.

[0322] The embodiments of this application can improve the effect of removing spatial redundancy of blocks.

[0323] The video encoding processing apparatus provided in this application embodiment can implement the various processes implemented in the method embodiment of FIG11 and achieve the same technical effect. To avoid repetition, it will not be described again here.

[0324] As shown in Figure 14, this application embodiment also provides an electronic device 600, including a processor 601 and a memory 602. The memory 602 stores programs or instructions that can run on the processor 601. For example, when the electronic device 600 is an encoding device, the program or instructions executed by the processor 601 implement the various steps of the above-described video decoding processing method embodiment and achieve the same technical effect. When the electronic device 600 is a decoding device, the program or instructions executed by the processor 601 implement the various steps of the above-described video decoding processing method embodiment and achieve the same technical effect. To avoid repetition, this will not be repeated here. Optionally, the memory 602 can be the memory 102 or memory 113 in the embodiment shown in Figure 1, and the processor 601 can implement the functions of the encoder 200 or decoder 300 in the embodiments shown in Figures 1-3.

[0325] This application also provides an electronic device, including: a memory configured to store video data; and a processing circuit configured to implement the various steps of the video decoding processing method or video encoding processing method embodiments described above. Optionally, the memory may be memory 102 or memory 113 in the embodiment shown in FIG1, and the processing circuit may implement the functions of encoder 200 or decoder 300 in the embodiments shown in FIG1-3.

[0326] This application also provides an electronic device, including a processor and a communication interface. The communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the steps in the method embodiments shown in FIG9 or FIG11. This device embodiment corresponds to the above method embodiment, and all implementation processes and methods of the above method embodiments can be applied to this terminal embodiment and can achieve the same technical effect.

[0327] The processor or processing circuit in this application embodiment may include general-purpose processors, special-purpose processors, etc., such as central processing units (CPUs), microprocessors, digital signal processors (DSPs), artificial intelligence (AI) processors, graphics processing units (GPUs), application-specific integrated circuits (ASICs), network processors (NPs), field-programmable gate arrays (FPGAs), or other programmable logic devices, gate circuits, transistors, discrete hardware components, etc. The communication interface in this application embodiment may include transceivers, pins, circuits, buses, etc.

[0328] The aforementioned electronic devices can be terminals or other devices besides terminals, such as servers, network attached storage (NAS), etc.

[0329] The terminal can be a mobile phone, tablet computer, laptop computer, notebook computer, personal digital assistant (PDA), handheld computer, netbook, ultra-mobile personal computer (UMPC), mobile internet device (MID), augmented reality (AR), virtual reality (VR) device, mixed reality (MR) device, robot, wearable device, flight vehicle, vehicle user equipment (VUE), shipboard equipment, pedestrian user equipment (PUE), smart home (home devices with wireless communication capabilities, such as refrigerators, televisions, washing machines, or furniture), game console, personal computer (PC), ATM or self-service machine, etc. Wearable devices include: smartwatches, smart bracelets, smart earphones, smart glasses, smart jewelry (smart bracelets, smart chains, smart rings, smart necklaces, smart anklets, smart anklets, etc.), smart wristbands, smart clothing, etc. Among these, in-vehicle devices can also be referred to as in-vehicle terminals, in-vehicle controllers, in-vehicle modules, in-vehicle components, in-vehicle chips, or in-vehicle units, etc. It should be noted that the embodiments in this application do not limit the specific type of terminal.

[0330] A server can be a standalone physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server. A cloud server can provide cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), or cloud computing services based on big data and artificial intelligence platforms.

[0331] For example, the aforementioned electronic device may include, but is not limited to, the type of source device 100 or destination device 110 shown in FIG1.

[0332] Taking an electronic device as an example, Figure 15 is a schematic diagram of the hardware structure of a terminal implementing an embodiment of this application.

[0333] The terminal 700 includes, but is not limited to, at least some of the following components: radio frequency unit 701, network module 702, audio output unit 703, input unit 704, sensor 705, display unit 706, user input unit 707, interface unit 708, memory 709, and processor 710.

[0334] Those skilled in the art will understand that the terminal 700 may also include a power supply (such as a battery) for powering various components. The power supply can be logically connected to the processor 710 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The terminal structure shown in Figure 15 does not constitute a limitation on the terminal. The terminal may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0335] It should be understood that, in this embodiment, the input unit 704 may include a graphics processor 7041 and a microphone 7042. The graphics processor 7041 processes image data of still images or videos obtained by an image acquisition device (such as a camera) in video acquisition mode or image acquisition mode, or it may process the obtained point cloud data. The display unit 706 may include a display panel 7061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, etc. The user input unit 707 includes at least one of a touch panel 7071 and other input devices 7072. The touch panel 7071 is also called a touch screen. The touch panel 7071 may include two parts: a touch detection device and a touch controller. Other input devices 7072 may include, but are not limited to, a physical keyboard, function keys (such as volume control buttons, power buttons, etc.), a trackball, a mouse, and a joystick, which will not be described in detail here.

[0336] In this embodiment, after receiving downlink data from the network-side device, the radio frequency unit 701 can transmit it to the processor 710 for processing; in addition, the radio frequency unit 701 can send uplink data to the network-side device. Typically, the radio frequency unit 701 includes, but is not limited to, antennas, amplifiers, transceivers, couplers, low-noise amplifiers, duplexers, etc.

[0337] The memory 709 can be used to store software programs or instructions, as well as various data. The memory 709 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 709 may include volatile memory or non-volatile memory. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 709 in the embodiments of this application includes, but is not limited to, these and any other suitable types of memory.

[0338] Processor 710 may include one or more processing units; optionally, processor 710 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into processor 710.

[0339] When the terminal is a decoding device:

[0340] Processor 710, used for:

[0341] Based on the multi-reference row information of the first target block, obtain the reconstruction reference sample corresponding to the first target block;

[0342] The predicted value of the first target block is determined based on the reconstructed reference sample;

[0343] Decoding is performed based on the predicted values.

[0344] Optionally, the processor 710 is specifically used for:

[0345] The second target block is determined based on the multi-reference row information of the first target block and the shape information of the first target block;

[0346] Based on the second target block, obtain the reconstruction reference sample corresponding to the first target block.

[0347] Optionally, the processor 710 is specifically used for:

[0348] The weight matrix is ​​determined based on the shape information of the second target block and the intra-frame prediction mode of the first target block;

[0349] The predicted value of the first target block is determined based on the weight matrix and the reconstruction reference sample.

[0350] Optionally, the processor 710 is specifically used for:

[0351] Based on the shape information of the second target block and the intra-frame prediction mode of the first target block, a template region is determined. The template region includes at least one of a first region, a second region, and a third region. The first region is adjacent to the second target block and located in a first position of the second target block. The second region is adjacent to the second target block and located in a second position of the second target block. The third region is adjacent to both the first region and the second region and located in a third position of the second target block.

[0352] Wherein, the reconstruction reference sample corresponding to the first target block is the reconstruction reference sample of the template region.

[0353] Optionally, the template area is an L-shaped area formed by the first area, the second area, and the third area.

[0354] Optionally, the template region satisfies at least one of the following conditions:

[0355] When the width and height values ​​of the second target block are both less than or equal to the first size threshold, the height value of the first region and the width value of the second region are both equal to the first size value, and the width and height values ​​of the third region are both equal to the first size value;

[0356] If either the width or height of the second target block is greater than the first size threshold, the height of the first region and the width of the second region are both equal to the second size value, and the width and height of the third region are both equal to the second size value.

[0357] When the index value of the intra-prediction mode is greater than or equal to the first mode index threshold and less than or equal to the second mode index threshold, the width value of the first region is the same as the width value of the second target block, the height value of the second region is the same as the height value of the second target block, and the first mode index threshold is less than the second mode index threshold.

[0358] If the index value of the intra-prediction mode is less than the first mode index threshold or greater than the second mode index threshold, the width of the first region is twice the width of the second target block, and the height of the second region is twice the height of the second target block.

[0359] The shape information of the second target block includes its width and height.

[0360] Optionally, the processor 710 is further configured to:

[0361] Obtain template-based multi-reference row intra-prediction mode (TMRL) combination information corresponding to the first target block. The TMRL combination information includes multi-reference row information and intra-prediction mode.

[0362] The multi-reference line information of the first target block and the intra-frame prediction mode of the first target block are determined based on the TMRL combination information corresponding to the first target block.

[0363] Optionally, if the multi-reference row information in the TMRL combination information includes the index value of the first orientation and the index value of the second orientation, and the index value of the first orientation and the index value of the second orientation are different, the multi-reference row information of the first target block is the maximum value of the index value of the first orientation and the index value of the second orientation.

[0364] In the case where the terminal is an encoding device:

[0365] Processor 710, used for:

[0366] Based on the multi-reference row information of the first target block, obtain the reconstruction reference sample corresponding to the first target block;

[0367] The predicted value of the first target block is determined based on the reconstructed reference sample;

[0368] Encoding is performed based on the predicted values.

[0369] Optionally, the processor 710 is specifically used for:

[0370] The second target block is determined based on the multi-reference row information of the first target block and the shape information of the first target block;

[0371] Based on the second target block, obtain the reconstruction reference sample corresponding to the first target block.

[0372] Optionally, the processor 710 is specifically used for:

[0373] The weight matrix is ​​determined based on the shape information of the second target block and the intra-frame prediction mode of the first target block;

[0374] The predicted value of the first target block is determined based on the weight matrix and the reconstruction reference sample.

[0375] Optionally, the processor 710 is specifically used for:

[0376] Based on the shape information of the second target block and the intra-frame prediction mode of the first target block, a template region is determined. The template region includes at least one of a first region, a second region, and a third region. The first region is adjacent to the second target block and located in a first position of the second target block. The second region is adjacent to the second target block and located in a second position of the second target block. The third region is adjacent to both the first region and the second region and located in a third position of the second target block.

[0377] Wherein, the reconstruction reference sample corresponding to the first target block is the reconstruction reference sample of the template region.

[0378] Optionally, the template area is an L-shaped area formed by the first area, the second area, and the third area.

[0379] Optionally, the template region satisfies at least one of the following conditions:

[0380] When the width and height values ​​of the second target block are both less than or equal to the first size threshold, the height value of the first region and the width value of the second region are both equal to the first size value, and the width and height values ​​of the third region are both equal to the first size value;

[0381] If either the width or height of the second target block is greater than the first size threshold, the height of the first region and the width of the second region are both equal to the second size value, and the width and height of the third region are both equal to the second size value.

[0382] When the index value of the intra-prediction mode is greater than or equal to the first mode index threshold and less than or equal to the second mode index threshold, the width value of the first region is the same as the width value of the second target block, the height value of the second region is the same as the height value of the second target block, and the first mode index threshold is less than the second mode index threshold.

[0383] If the index value of the intra-prediction mode is less than the first mode index threshold or greater than the second mode index threshold, the width of the first region is twice the width of the second target block, and the height of the second region is twice the height of the second target block.

[0384] The shape information of the second target block includes its width and height.

[0385] Optionally, the processor 710 is further configured to:

[0386] Obtain the TMRL combination information corresponding to the first target block, wherein the TMRL combination information includes multiple reference line information and intra-frame prediction mode;

[0387] The multi-reference line information of the first target block and the intra-frame prediction mode of the first target block are determined based on the TMRL combination information corresponding to the first target block.

[0388] Optionally, if the multi-reference row information in the TMRL combination information includes the index value of the first orientation and the index value of the second orientation, and the index value of the first orientation and the index value of the second orientation are different, the multi-reference row information of the first target block is the maximum value of the index value of the first orientation and the index value of the second orientation.

[0389] Optionally, the processor 710 is further configured to:

[0390] The predicted value of the first target block is calculated using multiple TMRL combination information, and the predicted value corresponding to each TMRL combination information is obtained.

[0391] Calculate the rate-distortion cost corresponding to each TMRL combination information based on the predicted value corresponding to each TMRL combination information;

[0392] The optimal intra-frame prediction mode for the first target block is determined based on the rate-distortion cost corresponding to each TMRL combination information.

[0393] The embodiments of this application can improve the effect of removing spatial redundancy of blocks.

[0394] It is understood that the implementation process of each implementation method mentioned in this embodiment can refer to the relevant description in Figure 9 or Figure 11 of the method embodiment, and achieve the same or corresponding technical effects. To avoid repetition, it will not be described again here.

[0395] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described video decoding or video encoding methods and achieve the same technical effect. To avoid repetition, these will not be described again here.

[0396] The processor mentioned above is the processor in the terminal described in the above embodiments. The readable storage medium includes computer-readable storage media, such as ROM, RAM, magnetic disk, or optical disk. In some examples, the readable storage medium may be a non-transient readable storage medium.

[0397] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described video decoding processing method or video encoding processing method embodiments, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0398] It should be understood that the chips mentioned in the embodiments of this application may include system-on-a-chip (also known as system chip, chip system, or system-on-a-chip) or discrete display chips, etc.

[0399] This application also provides a computer program / program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of the above-described video decoding processing method or video encoding processing method embodiments, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0400] This application also provides an encoding / decoding system, including: an encoding end device and a decoding end device. The encoding end device can be used to perform the steps of the video encoding processing method described above, and the decoding end device can be used to perform the steps of the video decoding processing method described above.

[0401] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0402] From the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of computer software products plus necessary general-purpose hardware platforms, and of course, they can also be implemented by hardware. The computer software product is stored in a storage medium (such as ROM, RAM, magnetic disk, optical disk, etc.) and includes several instructions to cause the terminal or network-side device to execute the methods described in the various embodiments of this application.

[0403] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other implementations under the guidance of this application without departing from the spirit and scope of the claims. All of these implementations are within the protection scope of this application.

Claims

1. A method for video decoding, comprising: obtaining, by a decoding end, reconstructed reference samples corresponding to a first target block based on multi-reference line information of the first target block; determining, by the decoding end, a prediction value of the first target block based on the reconstructed reference samples; performing, by the decoding end, decoding processing based on the prediction value.

2. The method of claim 1, wherein, The obtaining, by the decoding end, the reconstructed reference samples corresponding to the first target block based on the multi-reference line information of the first target block comprises: determining a second target block according to the multi-reference line information of the first target block and shape information of the first target block; obtaining the reconstructed reference samples corresponding to the first target block based on the second target block.

3. The method of claim 2, wherein, The determining, by the decoding end, the prediction value of the first target block based on the reconstructed reference samples comprises: determining a weight matrix according to the shape information of the second target block and an intra prediction mode of the first target block; determining the prediction value of the first target block based on the weight matrix and the reconstructed reference samples.

4. The method of claim 2 or 3, wherein, The obtaining, by the decoding end, the reconstructed reference samples corresponding to the first target block based on the second target block comprises: determining a template region based on the shape information of the second target block and the intra prediction mode of the first target block, the template region comprising at least one of a first region, a second region and a third region, the first region being adjacent to the second target block and located at a first orientation of the second target block, the second region being adjacent to the second target block and located at a second orientation of the second target block, the third region being adjacent to the first region and the second region and located at a third orientation of the second target block; wherein the reconstructed reference samples corresponding to the first target block are reconstructed reference samples of the template region.

5. The method of claim 4, wherein, The template region is an L-shaped region formed by the first region, the second region and the third region.

6. The method of claim 4 or 5, wherein, The template region satisfies at least one of the following conditions: in a case where a width value and a height value of the second target block are both less than or equal to a first size threshold, a height value of the first region and a width value of the second region are both equal to a first size value, and a width value and a height value of the third region are both equal to the first size value; in a case where any one of the width value and the height value of the second target block is greater than the first size threshold, the height value of the first region and the width value of the second region are both equal to a second size value, and the width value and the height value of the third region are both equal to the second size value; in a case where an index value of the intra prediction mode is greater than or equal to a first mode index threshold and less than or equal to a second mode index threshold, the width value of the first region is the same as the width value of the second target block, and the height value of the second region is the same as the height value of the second target block, the first mode index threshold being less than the second mode index threshold; in a case where the index value of the intra prediction mode is less than the first mode index threshold or greater than the second mode index threshold, the width value of the first region is twice the width value of the second target block, and the height value of the second region is twice the height value of the second target block. The shape information of the second target block includes a width value and a height value of the second target block.

7. The method of any one of claims 1-6, wherein, The method further includes: obtaining template-based multi-reference line intra prediction mode (TMRL) combination information corresponding to the first target block, the TMRL combination information including multi-reference line information and an intra prediction mode of the first target block; determining the multi-reference line information of the first target block and the intra prediction mode of the first target block based on the TMRL combination information corresponding to the first target block.

8. The method of claim 7, wherein, In a case where the multi-reference line information in the TMRL combination information includes an index value of a first direction and an index value of a second direction, and the index value of the first direction and the index value of the second direction are different, the multi-reference line information of the first target block is the maximum value of the index value of the first direction and the index value of the second direction.

9. A video encoding processing method, comprising: obtaining, by an encoding end, reconstructed reference samples corresponding to a first target block based on multi-reference line information of the first target block; determining, by the encoding end, a prediction value of the first target block based on the reconstructed reference samples; performing, by the encoding end, encoding processing based on the prediction value.

10. The method of claim 9, wherein, The method further includes: determining a second target block according to the multi-reference line information of the first target block and shape information of the first target block; obtaining, based on the second target block, the reconstructed reference samples corresponding to the first target block.

11. The method of claim 10, wherein, The method further includes: determining a weight matrix according to the shape information of the second target block and the intra prediction mode of the first target block; determining, based on the weight matrix and the reconstructed reference samples, the prediction value of the first target block.

12. The method of claim 10 or 11, wherein, The method further includes: determining a template region based on the shape information of the second target block and the intra prediction mode of the first target block, the template region including at least one of a first region, a second region and a third region, the first region being adjacent to the second target block and located at a first direction of the second target block, the second region being adjacent to the second target block and located at a second direction of the second target block, the third region being adjacent to the first region and the second region and located at a third direction of the second target block; wherein the reconstructed reference samples corresponding to the first target block are reconstructed reference samples of the template region.

13. The method of claim 12, wherein, The template region is an L-shaped region formed by the first region, the second region and the third region.

14. The method of claim 12 or 13, wherein, The template region satisfies at least one of the following conditions: in a case where the width value and the height value of the second target block are both less than or equal to a first size threshold, the height value of the first region and the width value of the second region are both equal to a first size value, and the width value and the height value of the third region are both equal to the first size value; in a case where either of the width value and the height value of the second target block is greater than a first size threshold, the height value of the first region and the width value of the second region are both equal to a second size value, and the width value and the height value of the third region are both equal to the second size value; in a case where the index value of the intra prediction mode is greater than or equal to a first mode index threshold and less than or equal to a second mode index threshold, the width value of the first region is the same as the width value of the second target block, and the height value of the second region is the same as the height value of the second target block, the first mode index threshold being less than the second mode index threshold; in a case where the index value of the intra prediction mode is less than the first mode index threshold or greater than the second mode index threshold, the width value of the first region is twice the width value of the second target block, and the height value of the second region is twice the height value of the second target block. The shape information of the second target block includes a width value and a height value of the second target block.

15. The method of any one of claims 9-14, wherein, The method further includes: obtaining TMRL combination information corresponding to the first target block, the TMRL combination information including multi-reference line information and an intra prediction mode; determining, based on the TMRL combination information corresponding to the first target block, the multi-reference line information of the first target block and the intra prediction mode of the first target block.

16. The method of claim 15, wherein, In a case where the multi-reference line information in the TMRL combination information includes an index value of a first direction and an index value of a second direction, and the index value of the first direction and the index value of the second direction are different, the multi-reference line information of the first target block is the maximum of the index value of the first direction and the index value of the second direction.

17. The method of any one of claims 9-16, wherein, The method further includes: calculating a prediction value of the first target block using a plurality of TMRL combination information respectively, to obtain a prediction value corresponding to each TMRL combination information; calculating a rate-distortion cost corresponding to each TMRL combination information based on the prediction value corresponding to the TMRL combination information; determining an optimal intra prediction mode of the first target block based on the rate-distortion cost corresponding to each TMRL combination information.

18. A video decoding processing apparatus, comprising: a first obtaining module configured to obtain a reconstructed reference sample corresponding to a first target block based on multi-reference line information of the first target block; a first determining module configured to determine a prediction value of the first target block based on the reconstructed reference sample; a processing module configured to perform decoding processing based on the prediction value.

19. The apparatus of claim 18, wherein, The first obtaining module includes: a determining unit configured to determine a second target block according to the multi-reference line information of the first target block and shape information of the first target block; an obtaining unit configured to obtain the reconstructed reference sample corresponding to the first target block based on the second target block.

20. The apparatus of claim 19, wherein, The first determining module is specifically configured to: determine a weight matrix according to the shape information of the second target block and an intra prediction mode of the first target block; and determine the prediction value of the first target block based on the weight matrix and the reconstructed reference sample.

21. The apparatus of claim 19 or 20, wherein, The obtaining unit is specifically configured to: determine a template region based on the shape information of the second target block and the intra prediction mode of the first target block, the template region comprising at least one of a first region, a second region and a third region, the first region being adjacent to the second target block and located at a first orientation of the second target block, the second region being adjacent to the second target block and located at a second orientation of the second target block, the third region being adjacent to both the first region and the second region and located at a third orientation of the second target block; wherein the reconstructed reference samples corresponding to the first target block are reconstructed reference samples of the template region. 22.A video encoding processing apparatus, comprising: a first obtaining module, configured to obtain reconstructed reference samples corresponding to a first target block based on multi-reference line information of the first target block; a first determining module, configured to determine a prediction value of the first target block based on the reconstructed reference samples; a processing module, configured to perform encoding processing based on the prediction value.

23. The apparatus of claim 22, wherein, The first obtaining module comprises: a determining unit, configured to determine a second target block according to the multi-reference line information of the first target block and shape information of the first target block; an obtaining unit, configured to obtain the reconstructed reference samples corresponding to the first target block based on the second target block.

24. The apparatus of claim 23, wherein, The first determining module is specifically configured to: determine a weight matrix based on the shape information of the second target block and the intra prediction mode of the first target; determine the prediction value of the first target block based on the weight matrix and the reconstructed reference samples.

25. The apparatus of claim 23 or 24, wherein, The obtaining unit is specifically configured to: determine a template region based on the shape information of the second target block and the intra prediction of the first target block, the template region comprising at least one of a first region, a first region, a second region and a third region, the first region being adjacent to the target block and located at a first orientation of the target block, the second region being adjacent to the target block and located at a second orientation of the target block, the third region being adjacent to both the first region and the second region and located in a third orientation of the target block; wherein the reconstructed reference samples corresponding to the first target block are reconstructed reference samples in the template region. 26.An electronic device, comprising a processor and a memory, the memory storing programs or instructions executable on the processor, the programs or instructions being executed by the processor to implement steps of the video decoding processing method according to any one of claims 1 to 8, or implement steps of the video encoding processing method according to any one of claims 9 to 17. 27.A readable storage medium, the readable storage medium storing programs or instructions, the programs or instructions being executed by a processor to implement steps of the video decoding processing method according to any one of claims 1 to8, or implement steps of the video encoding processing method according to any one of claims 9to 17.

28. A chip, comprising a processor and a communication interface, the communication interface and the processor coupled, the processor configured to execute a program or instructions to implement steps of the video decoding processing method of any one of claims 1 to 8, or to implement steps of the video encoding processing method of any one of claims 9 to 17.

Images