Video encoding, decoding method and system

Abstract

A method of video decoding, comprising: obtaining a bitstream comprising a plurality of coded frames in a video signal; decoding the plurality of coded frames into a plurality of superblocks and decoding the plurality of superblocks into a plurality of residual blocks; recovering each of the plurality of residual blocks into a coded block based on a multi-reference line intra prediction flag and reference samples in each coded frame, wherein a plurality of reference lines are divided into an upper reference line and a left reference column, and one upper reference line and one left reference column are selected for intra prediction; reconstructing each frame in the video signal by storing the recovered coded block of each of the plurality of residual blocks in a frame buffer; and outputting the reconstructed frames in succession to recover the video signal.

Description

[0001] Cross-references to related applications

[0002] This application claims priority to U.S. non-provisional patent application No. 17 / 077,872, filed November 22, 2020, which claims priority to U.S. provisional patent application No. 62 / 964,992, filed January 23, 2020, the entire contents of which are incorporated herein by reference. Technical Field

[0003] This disclosure relates to the field of video coding technology, specifically to a method and system for video encoding and decoding. Background Technology

[0004] The Open Media Consortium (AOMedia) Video 1 (AV1) is an open video coding format suitable for video transmission over the Internet. Developed by the AOMedia Consortium (AOMedia) as the next-generation product of VP9, ​​the AOMedia Consortium was established in 2015 and includes semiconductor companies, video-on-demand providers, video content creators, software development companies, and web browser vendors. Based on the VP9 codebase, AV1 incorporates other technologies, some of which were developed in experimental formats. The first version 0.1.0 of the AV1 reference codec was released on April 7, 2016. The consortium released the AV1 bitstream specification, along with software-based reference encoders and decoders, on June 25, 2018. On June 25, 2018, the validation version 1.0.0 of the AV1 bitstream specification was released. On January 8, 2019, validation version 1.0.0 of the specification and its errata table 1 were released. The AV1 bitstream specification includes a reference video codec.

[0005] The ITU-T VCEG (Q6 / 16) and the ISO / IEC MPEG (JTC 1 / SC 29 / WG 11) released the first version of the H.265 / HEVC (High Efficiency Video Coding) standard in 2013, the second version in 2014, the third version in 2015, and the fourth version in 2016, respectively. In 2015, these two standards organizations merged to form the Joint Video Exploration Team (JVET) to explore the possibility of developing a next-generation video coding standard that surpasses HEVC. In November 2017, they issued a joint call for proposals (CfP) seeking joint proposals that would exceed HEVC's video compression capabilities. As of February 15, 2018, a total of 22 CfP responses were received regarding Standard Dynamic Range (SDR), 12 regarding High Dynamic Range (HDR), and 12 regarding 360-degree video classification. In April 2018, at the 122nd MPEG / 10th JVET meeting, all received CfP responses were evaluated. As an outcome of this meeting, JVET officially embarked on the next-generation video coding standardization process, surpassing HEVC. The new standard was named Versatile Video Coding (VVC), and JVET was renamed the Joint Video Experts Group.

[0006] Multiple reference line prediction (MRLP) is applied to the video coding schemes of the two video coding standards mentioned above. The result of MRLP is stored in a line buffer. The line buffer is a typical and primary on-chip memory design architecture used in image / video processing circuits. Since line buffers typically occupy a very large amount of on-chip circuit area, reducing their hardware cost through efficient architecture design is crucial. For MRLP, more than one (e.g., four) reference lines may be used for intra-frame prediction, which increases the size of the line buffer and occupies a significant amount of on-chip circuit area.

[0007] The methods and systems disclosed herein are intended to solve one or more of the problems described above, as well as other problems. Summary of the Invention

[0008] One aspect of this disclosure provides a video decoding method, the method comprising: acquiring a bitstream including a plurality of coded frames in a video signal; decoding each of the plurality of coded frames into a plurality of superblocks, and decoding each of the plurality of superblocks into a plurality of residual blocks; recovering each of the plurality of residual blocks into a coded block (CB) based on a multi-reference row intra-frame prediction (MRLP) flag and a plurality of reference samples contained in each of the plurality of coded frames, wherein the plurality of reference rows are divided into an upper reference row and a left reference column, and selecting one of the upper reference row and one of the left reference columns for intra-frame prediction; reconstructing each frame of the video signal by storing the recovered coded blocks of each of the plurality of residual blocks in a frame buffer; and continuously outputting the reconstructed frames to recover the video signal.

[0009] Another aspect of this disclosure provides a video coding method, the method comprising: acquiring a current frame of a video input; dividing the acquired current frame into a series of superblocks according to a nested multi-type tree coding block structure, and dividing each superblock into multiple coding blocks (CBs); acquiring residual blocks of each coding block through multi-reference row intra-frame prediction, storing multi-reference row intra-frame prediction flags and reference samples of selected reference rows for intra-frame prediction in a line buffer, wherein the multiple reference rows are divided into upper reference rows and left reference columns, selecting one upper reference row and one left reference column for intra-frame prediction to minimize the size of the line buffer; and encoding the current frame through the residual blocks of each coding block of the current frame to generate a bitstream including the residual blocks of each coding block.

[0010] Other aspects of this disclosure will be understood by those skilled in the art based on the description, claims, and drawings. Attached Figure Description

[0011] Figure 1 A schematic diagram of the operating environment of an embodiment of this disclosure is shown;

[0012] Figure 2 A schematic diagram of an electronic device according to an embodiment of the present disclosure is shown;

[0013] Figure 3 A schematic diagram of a computer system according to an embodiment of the present disclosure is shown;

[0014] Figure 4 A schematic diagram of a video encoder according to an embodiment of the present disclosure is shown;

[0015] Figure 5 A schematic diagram of a video decoder according to an embodiment of the present disclosure is shown;

[0016] Figure 6A schematic diagram of an exemplary video encoding method according to an embodiment of the present disclosure is shown;

[0017] Figure 7 A schematic diagram of an exemplary video decoding method according to an embodiment of the present disclosure is shown;

[0018] Figure 8 This illustration shows a schematic diagram of four multi-type tree partitioning modes according to an embodiment of the present disclosure;

[0019] Figure 9 A schematic diagram illustrating the signaling mechanism for partition information in an exemplary coding tree structure according to an embodiment of this disclosure is shown.

[0020] Figure 10 A schematic diagram of an exemplary coding tree structure according to an embodiment of this disclosure is shown;

[0021] Figure 11 Another exemplary coding tree structure diagram of an embodiment of this disclosure is shown;

[0022] Figure 12 This illustration shows yet another exemplary coding tree structure diagram according to an embodiment of the present disclosure;

[0023] Figure 13 A schematic diagram of eight nominal angles of the intra-prediction mode according to an embodiment of the present disclosure is shown;

[0024] Figure 14 A schematic diagram showing the top, left, and upper left positions of a PAETH mode according to an embodiment of this disclosure is provided.

[0025] Figure 15 A schematic diagram of a recursive intra-frame filtering mode according to an embodiment of the present disclosure is shown;

[0026] Figure 16 A schematic diagram of four reference lines according to an embodiment of the present disclosure is shown;

[0027] Figure 17 A schematic diagram of the upper reference row and left reference column of an embodiment of the present disclosure is shown;

[0028] Figure 18 This illustration shows a schematic diagram of selecting the upper reference row 0 and the left reference column 0 for intra-frame prediction according to an embodiment of the present disclosure;

[0029] Figure 19 A schematic diagram illustrating the selection of upper reference row 0 and left reference column 1 for intra-frame prediction according to an embodiment of the present disclosure is shown;

[0030] Figure 20 A schematic diagram illustrating the selection of upper reference row 0 and left reference column 2 for intra-frame prediction according to an embodiment of the present disclosure is shown;

[0031] Figure 21 A schematic diagram illustrating the selection of upper reference row 0 and left reference column 3 for intra-frame prediction according to an embodiment of the present disclosure is shown;

[0032] Figure 22 A schematic diagram is shown illustrating an embodiment of the present disclosure of projecting the integer positions of reference samples from non-adjacent upper reference rows onto adjacent upper reference rows in a direction predicted within an angular frame. Detailed Implementation

[0033] The technical solutions in the embodiments of the present invention will now be described with reference to the accompanying drawings. Where possible, the same reference numerals will be used throughout the drawings to denote the same or similar parts. Obviously, the described embodiments are only some, not all, embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention. Some terms used in this disclosure will first be explained below.

[0034] Universal Video Coding (VVC): VVC is a video decoding standard developed by the Joint Video Exploration Team (JVET) to succeed HEVC (also known as ITU-TH.265). VVC specifies a video coding technique with compression capabilities that substantially exceed those of previous generations of such standards, and is highly effective and versatile in a wider range of applications. The VVC standard is incorporated herein by reference.

[0035] AO Media Video 1 (AVI): AVI is an open-source video coding standard developed by the Open Media Consortium (AO Media) for transmitting video over the Internet. The AVI standard is incorporated herein by reference.

[0036] Allowed Intra Prediction Modes (AIPM): Defined as a set of modes that can be used to derive intra prediction of the current block based on the intra prediction modes of neighboring blocks.

[0037] Disallowed Intra Prediction Modes (DIPM): Defined as a set of modes that cannot be used for signal transmission or for intra prediction of the current block derived from the intra prediction modes of neighboring blocks.

[0038] Chromaticity component prediction using luma components (CfL): CfL uses the correlation between luma and color information and reference samples of luma components from the same coding block to perform intra-frame prediction of chroma components.

[0039] Multi-reference line intra-frame prediction (MRLP): MRLP uses not only the nearest reference line in intra-frame prediction, but also multiple reference lines that are far away from the current coding block.

[0040] Coding Unit (CTU): The CTU is the basic processing unit in various video coding schemes of the HEVC and VVC standards. A video frame is divided into multiple CTUs, and each CTU is further divided into various smaller units or blocks for efficient video coding. In the AV1 standard video coding scheme, the structure equivalent to a CTU is called a superblock.

[0041] Context Adaptive Binary Arithmetic Coding (CABAC): CABAC is a form of entropy coding used in various video coding standards.

[0042] Universal Video Coding Test Model (VTM): VTM provides an encoder-side description of the VVC algorithm and test procedures.

[0043] Incremental Bit Rate (BDR or BD-Rate): BDR is a method to evaluate coding efficiency by measuring the bit rate change of the same peak signal-to-noise ratio (PSNR) of a video signal.

[0044] Figure 1 The operating environment 100 of some embodiments of this disclosure is illustrated. For example... Figure 1 As shown, the operating environment 100 may include various devices with video capabilities. For example, the operating environment 100 may include terminal device 102, camera device 104, and Internet of Things (IoT) device 106. Other types of devices may also be included.

[0045] The operating environment 100 may further include a server 122, a human vision application 124, a machine vision application 126, and a communication link 140 connecting various devices to the server 122. A user 130 may use, access, or control one or more of the various devices.

[0046] The terminal device 102 may include any user terminal, such as a personal digital assistant (PDA), mobile phone, smartphone, integrated information device (IMD), tablet computer, laptop computer, desktop computer, and other computing devices. The camera device 104 may include any image or video capture device, such as a digital camera, camcorder, security camera, vehicle camera, and stereo camera. The IoT device 106 may include any type of IoT device with camera functionality, such as a digital doorbell, autonomous driving sensor, digital voice assistant, smart speaker, smart application, and any industrial or commercial IoT system. Any of the various devices 102, 104, and 106 may be fixed or mobile when carried by a mobile individual; and may also be part of or within a mode of transportation, including cars, trucks, taxis, trains, ships, airplanes, bicycles, motorcycles, or any similar suitable mode of transportation.

[0047] Figure 2 Electronic devices 102, 104, and / or 106 for implementing any of the various devices described above are shown. For example... Figure 2 As shown, the electronic device 200 may include a hardware processor 202, a memory 204, a card reader 206, a display 208, a keyboard 210, a radio frequency (RF) interface 212, a baseband 214, an antenna 216, an encoder 222, a decoder 224, a camera 226, a speaker 232, and a microphone 234, etc. Figure 2 The components shown are exemplary; some components may be omitted, and others may be added.

[0048] The processor 202 can be used to control the electronic device 200. The processor 202 can be connected to other components via one or more buses or other electrical connections to send data to and receive data from those other components. The processor 202 may include one or more processing cores, such as a quad-core or octa-core processor. The processor 202 can be implemented using at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), application-specific integrated circuit (ASIC), and programmable logic array (PLA). The processor 202 may also include a main processor and a coprocessor. The main processor may be a central processing unit (CPU), and the coprocessor may be a graphics processing unit (GPU) configured to display and draw content that the display screen needs to show. In some embodiments, the processor 202 may also include an artificial intelligence (AI) processor configured to handle computational operations related to machine learning.

[0049] The memory 204 may include one or more computer-readable storage media, such as high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. The memory 204 may store data not only in the form of image and audio data, but also instructions for the processor 202. The card reader 206 may include any type of portable card interface, such as a smart card interface, a communication card interface (e.g., a near field communication (NFC) card), or a subscriber identity module (SIM) card, or other card interfaces used to provide user information and suitable for providing verification information for user 130 authentication and authorization.

[0050] Furthermore, the display 208 can be any suitable display technology suitable for displaying images or videos. For example, the display 208 may include a liquid crystal display (LCD) screen, an organic light-emitting diode (OLED) screen, etc., and may be a touch screen. The keyboard 210 may include a physical or virtual keyboard for allowing the user 130 to input information, and may also include other types of input / output devices. The speaker 232 and the microphone 234 can be used to input and output audio for the electronic device 200.

[0051] The RF interface 212 (together with antenna 216) may include RF circuitry for receiving and transmitting RF signals. The RF interface 212 can convert electrical signals into electromagnetic signals for transmission or convert received electromagnetic signals into electrical signals for reception. The RF interface 212 can communicate with other electronic devices via at least one wireless communication protocol. The wireless communication protocol may include metropolitan area networks, various generations of mobile communication networks (2G, 3G, 4G, and 5G), wireless local area networks (LANs), and / or Wi-Fi networks. In some embodiments, the RF interface 212 may also include circuitry associated with near-field communication (NFC). The baseband 214 may include circuitry for processing signals transmitted to or from the RF interface 212.

[0052] Furthermore, the camera 226 may include any type of imaging or video capture device configured to collect images or videos. When the electronic device 200 is a portable device carried by the user 130, the camera 226 may include a front-facing camera and a rear-facing camera. The front-facing camera may be disposed on the front panel of the electronic device, and the rear-facing camera may be disposed on the back panel of the electronic device. In some embodiments, there are at least two rear-facing cameras, each being any one of a main camera, a depth-sensing camera, a wide-angle camera, and a telephoto camera, to achieve background blur functionality by fusing the main camera and the depth-sensing camera, and to achieve panoramic shooting, virtual reality (VR) shooting, or other fusion shooting functions by fusing the main camera and the wide-angle camera. In some embodiments, the camera 226 may also include a flash.

[0053] The encoder 222 and the decoder 224 can be referred to as the encoding and decoding circuit of the electronic device, which is suitable for encoding and decoding audio and / or video data, or assisting the processor 202 in encoding and decoding.

[0054] like Figure 2The illustrated electronic device 200 may include any structure similar to that of a mobile terminal or user equipment in a wireless communication system. However, it may also include any electronic device or apparatus that may require encoding and decoding, or encoding or decoding video or images.

[0055] refer to Figure 1 The electronic device 200 (i.e., any one or more of the various devices 102, 104, and / or 106) can capture or collect various data (e.g., audio data, environmental / operational data, image data, and / or video data) and transmit the data to the server 122 via the communication link 140. The electronic device 200 can process or preprocess the data before sending it to the server 122, or it can send the unprocessed data to the server 122.

[0056] The communication link 140 may include any suitable type of communication network and may include any combination of wired or wireless networks, including but not limited to wireless cellular telephone networks, wireless local area networks (WLANs), Bluetooth personal area networks, Ethernet local area networks, token ring local area networks, wide area networks, and the Internet. The communication link 140 may also include a private or public cloud computing platform for voice / data communication. When the Internet or other types of networks are included, the connection to the Internet may include long-range wireless connections, short-range wireless connections, and various wired connections, including telephone lines, cables, power lines, and similar communication paths.

[0057] The server 122 may include any type of server computer system or multiple computer systems configured in a server cluster or distributed in different locations. The server 122 may also include a cloud server on a cloud computing platform. Figure 3 An exemplary computer system implementing certain aspects of the server 122 is shown.

[0058] like Figure 3 As shown, the computer system 300 may include a processor 302, a storage medium 304, a monitor 306, a communication module 308, a database 310, and peripheral devices 312. Some devices may be omitted, and other devices may be included.

[0059] The processor 302 may include any suitable one or more processors. Furthermore, the processor 302 may include multiple cores for multi-threaded or parallel processing. The storage medium 304 may include memory modules such as read-only memory (ROM), random access memory (RAM), flash memory modules, erasable and rewritable memory, and mass storage devices such as CD-ROMs, USB flash drives, and hard disks. When executed by the processor 302, the storage medium 304 may store computer programs for implementing various operations.

[0060] Furthermore, the peripheral device 312 may include I / O devices, such as a keyboard and mouse. The communication module 308 may include a network device for establishing a connection through the communication network. The database 310 may include one or more databases for storing certain data and performing certain operations on the stored data, such as database searches.

[0061] refer to Figure 2 The encoder 222 can be an encoder that implements an intra-frame mode coding method. In this case, the electronic device 200 can also be considered to have implemented the intra-frame mode coding method. That is, the encoder 222 can be implemented by hardware circuitry in the electronic device 200, or by software program in the electronic device 200, or by a combination of hardware circuitry and software program. Figure 4 A schematic diagram of an exemplary video encoder according to a disclosed embodiment of the present disclosure is shown.

[0062] like Figure 4 As shown, the video encoder 400 includes a block partitioning unit 410, a subtraction unit 420, a prediction unit 430, a transform / quantization unit 440, an inverse quantization / inverse transform unit 450, an addition unit 460, a loop filter 470, and an entropy encoder 480. Input video is fed into the video encoder 400, and in response, the video encoder 400 outputs a bitstream.

[0063] The input video comprises multiple image frames. Each image frame is divided into a series of superblocks or coding unit units (CTUs) by the block partitioning unit 410. For an image frame with three sample arrays, a superblock or CTU consists of NxN luminance sample blocks and two corresponding MxM chrominance sample blocks. The block partitioning unit 410 further divides the CTUs into coding units (CUs) using a nested multi-type tree to accommodate various local features. The nested multi-type tree includes binary and ternary partitioning structures and is represented as a coding tree. At the leaf CU level of the coding tree, a decision is made on whether to use inter-image (temporal) or intra-image (spatial) prediction to encode the image region. The leaf CUs are subjected to prediction processing to obtain residual blocks. The residual blocks are transformed, and relevant information is transmitted to the corresponding decoder in units of CUs. After obtaining the residual blocks by applying the prediction process, the leaf CUs can be partitioned into transform units (TUs) through another tree structure for transformed or untransformed residual coding.

[0064] The prediction unit 430 supports intra-frame prediction, inter-frame prediction, and combinations of inter-frame and intra-frame prediction. Intra-frame prediction is also known as intra-frame pattern coding. To capture arbitrary edge directions presented in natural video, in addition to planar (surface fitting) intra-frame prediction modes and DC (planar) intra-frame prediction modes, the prediction unit 430 also supports 65 directional (or angle) intra-frame prediction modes. The prediction unit 430 also supports the generation of inter-frame prediction samples based on momentum parameters, which include motion vectors, reference image indices, motion parameters using the reference image list index, and additional information required for video coding features. The prediction unit 430 also supports skip modes, where the CU encoded in skip modes is associated with a PU and has no significant residual coefficients, changes in encoded motion vectors, or reference image indices. The prediction unit 430 also supports merging modes, where the motion parameters of the current CU are obtained from neighboring CUs, including spatial and temporal candidates and additional scheduling. The merging mode is not only applicable to the skip modes but can be applied to any inter-frame predicted CU. The prediction unit 430 also supports the substitution of merging modes by explicitly transmitting motion parameters, wherein motion vectors, the corresponding reference image index for each reference image list, and reference image list usage flags, as well as other necessary information, are explicitly transmitted by signal per CU.

[0065] The subtraction unit 420 outputs the difference (or residual) between the CU and the PU to the transformation / quantization unit 440.

[0066] The transform / quantization unit 440 supports large block sizes up to 64x64. For transform blocks of size 64, high-frequency transform coefficients are zeroed out, leaving only low-frequency coefficients. When transforming large blocks in the skip mode, the transform / quantization unit 440 uses the entire block without zeroing out any values. The transform / quantization unit 440 also supports Multiple Transform Selection (MTS) for kernel transform. To control the MTS, the transform / quantization unit 440 uses separate enable flags at the Sequence Parameter Set (SPS) level for intra-frame and inter-frame prediction. When the MTS is enabled at the SPS level, a flag at the CU level is issued to indicate whether the MTS is applied. The transform / quantization unit 440 also supports other transform features described in the video coding schemes of the VVC and AV1 standards, such as Low Frequency Inseparable Transform (LFNST) and Subblock Transform (SBT).

[0067] Furthermore, the transform / quantization unit 440 supports up to 63 quantization parameters (QPs). The transform / quantization unit 440 also supports flexible luma-to-chroma QP mapping by signaling the luma-to-chroma mapping relationship in the SPS. The transform / quantization unit 440 also supports QP adaptation at the CU level, where the changes in the luma and chroma component QP values ​​can be signaled separately. The transform / quantization unit 440 also supports other quantization features described in the video schemes under the VVC and AV1 standards, such as correlated quantization.

[0068] The transform / quantization unit 440 also supports joint encoding of chromaticity residuals, indicated by a flag at the TU level.

[0069] Context-Adaptive Binary Arithmetic Coding (CABAC) can be described as the entropy coding algorithm. The entropy encoder 480 includes a CABAC coding engine to encode the syntactic elements output by the transform / quantization unit 440. The CABAC coding engine supports arithmetic coding algorithms, such as the Golomb-Rice coding algorithm. The entropy encoder 480 supports separate residual coding structures for transform blocks and transform skip blocks. The entropy encoder 480 encodes the transform coefficients of the coded block using non-overlapping coefficient groups (CGs or sub-blocks), and each CG contains the transform coefficients of a variable-size coded block. The size of the coefficient group is selected solely based on the size of the transform block, regardless of the channel type. The size of the CG can include 1x16, 2x8, 8x2, 2x4, 4x2, and 16x1. The CG within each coded block and the transform coefficients within each CG are encoded according to a predefined scan order.

[0070] Each coding block in the transform unit is divided into multiple sub-blocks, and each sub-block is divided into multiple pixels. The multiple sub-blocks include 4x4 or 2x2 sub-blocks.

[0071] In the arithmetic coding algorithm, a probability model is dynamically selected for each syntactic element based on previously encoded information in the local neighborhood.

[0072] The inverse quantization / inverse transform unit 450 executes the inverse of the quantization and transform processes performed by the transform / quantization unit 440, and transmits the output to the adder unit 460 to reconstruct the image frame. The adder unit 460 also uses the output of the prediction unit 430 as another input. The reconstructed image is filtered by the loop filter 470. The loop filter 470 includes a deblocking filter, a sample adaptive offset (SAO) filter, and an adaptive loop filter (ALF), cascaded in this order. The output of the loop filter 470 is input to the prediction unit 430.

[0073] Figure 5 A schematic diagram of a video decoder according to an embodiment of the present disclosure is shown. For example... Figure 5 As shown, the video decoder 500 includes an entropy decoder 510, an inverse transform / inverse quantization unit 520, an addition unit 530, a prediction unit 540, a loop filter 550, and a frame buffer 560. Figure 4 The bitstream output from the video encoder 400 is input to the video decoder 500, which in turn outputs a video signal. The input bitstream can be transmitted via a communication network before being input to the video decoder 500.

[0074] The video decoder 500 performs a video decoding process on the input bitstream to reconstruct the image frame. The video decoding process is the reverse of the video encoding process performed by the video encoder 400. Specifically, the entropy decoder 510 decodes the bitstream encoded by the entropy encoder 480. The inverse transform / inverse quantization unit 520 performs the reverse of the transform / quantization process performed by the transform / quantization unit to recover the residual block. The addition unit 530 calculates the residual block and the predicted block predicted by the prediction unit 540 to recover the encoded block. The loop filter 550 smooths the recovered encoded block. Finally, the encoded block is stored in the frame buffer 560 to reconstruct the image frame.

[0075] Meanwhile, the prediction unit 540 uses the decoded encoded block and various flags in the input bitstream as input to generate the prediction block.

[0076] This disclosure provides a video encoding method. Figure 6 A flowchart illustrating an exemplary video encoding method according to an embodiment of this disclosure is shown. Figure 6 As shown, the video encoding method includes obtaining the current frame of the video input (in S610). The current frame can be divided into a series of superblocks or coding tree units (CTUs) according to a nested multi-type tree coding block structure, and each superblock or CTU can be further divided into coding blocks (CBs) (in S620).

[0077] In the video coding schemes of the HEVC and VVC standards, video frames are divided into a series of CTUs. In the HEVC standard, the CTUs are further divided into CUs using a quadtree structure represented by a coding tree to accommodate various local features. At the CU level, it is determined whether to use inter-image (temporal) or intra-image (spatial) prediction to encode image regions. Each CU can be further divided into one, two, or four PUs based on the PU partitioning type. Within a PU, the same prediction process is applied, and the relevant information is transmitted to the decoder based on the PU. After obtaining the residual block by applying the prediction process based on the PU partitioning type, the CU can be divided into TUs according to another quadtree structure similar to the CU's coding tree. A key feature of the HEVC structure is its multi-partition concept, including CUs, PUs, and TUs.

[0078] In the VVC standard video coding scheme, a quadtree with nested multi-type trees using binary and ternary partitioning structures replaces the concept of multiple partition unit types. That is, unless the CU size exceeds the maximum transform length supported by VCC, it eliminates the distinction between CU, PU, ​​and TU concepts and supports more flexible CU partition shapes. In this coding tree structure, CUs can have square or rectangular shapes. CTUs are first partitioned by a quadtree (also known as a quadtree) structure. Then, the leaf nodes of the quadtree can be further partitioned using a multi-type tree structure. For example... Figure 8 As shown, the multi-type tree structure has four splitting types: vertical binary partition (SPLIT_BT_VER), horizontal binary partition (SPLIT_BT_HOR), vertical ternary partition (SPLIT_TT_VER), and horizontal ternary partition (SPLIT_TT_HOR). The multi-type tree leaf nodes are called CUs. Unless the CU is too large for the maximum transform length, this partitioning is directly used for prediction and transform processing without further partitioning. This means that in most cases, the CU, PU, ​​and TU have the same block size in the quadtree with nested multi-type tree coding block structures. An anomaly occurs when the maximum supported transform length is less than the width or height of the CU color component.

[0079] Figure 9 A schematic diagram of the signaling mechanism for partitioning information in an exemplary coded tree structure according to an embodiment of this disclosure is shown. The CTU is considered the root of a quadtree and is first partitioned by the quadtree structure. Each quadtree leaf node (when large enough to allow it) is further partitioned by a multi-type tree structure. In the multi-type tree structure, a first flag (multi-type tree partitioning coding unit flag) is signaled to indicate whether the node is further partitioned. When a node is further partitioned, a second flag (multi-type tree vertical partitioning coding unit flag) is signaled to indicate the partitioning direction, and then a third flag (multi-type tree binary partitioning coding unit flag) is signaled to indicate whether the partition is a binary or ternary partition. Based on the values ​​of the multi-type tree vertical partitioning coding unit flag and the multi-type tree binary partitioning coding unit flag, the multi-type tree partitioning mode (MttSplitMode) of the CU is derived, as shown in Table 1.

[0080] Table 1 - Derivation of Multi-Type Tree Partitioning Patterns Based on Syntactic Elements

[0081]

[0082]

[0083] Figure 10 A schematic diagram of an exemplary coding tree structure according to an embodiment of this disclosure is shown. Figure 10 As shown, the CTU can be divided into multiple CUs with quadtree and nested multi-type tree coding block structures, where bold block edges represent quadtree partitions and the remaining edges represent multi-type tree partitions. The quadtree with nested multi-type tree partitions provides a content-adaptive coding tree structure composed of CUs. The size of the CU can be the same as the CTU, or as small as 4×4 in units of luma samples. For the chroma format of 4:2:0, the maximum size of the chroma CB is 64×64, and the minimum size of the chroma CB is 2×2.

[0084] In the VVC standard video coding scheme, the maximum supported size for luma transformation is 64×64, and the maximum supported size for chroma transformation is 32×32. When the width or height of the CB (Content Controller) exceeds the maximum transformation width or height, the CB will be automatically divided in the horizontal and / or vertical directions to meet the transformation size limit in that direction.

[0085] In the video coding scheme of the VVC standard (e.g., VTM7), the coding tree scheme supports the ability for luma and chroma to have separate block tree structures. Currently, for slices P and B, the luma and chroma coding tree blocks (CTBs) in a CTU must share the same coding tree structure. However, for slice I, luma and chroma can have separate block tree structures. When a separate block tree mode is applied, the luma CTB is divided into CUs by a different coding tree structure, and the chroma CTB is divided into chroma CUs by a different coding tree structure. This means that a CU in slice I can consist of coding blocks for the luma component or coding blocks for the two chroma components, while a CU in slices P or B, unless the video is monochrome, always consists of coding blocks for all three color components.

[0086] Similarly, in the video coding scheme under the VP9 standard, video frames are divided into a series of CTUs. For example... Figure 11 As shown, a 4-way partitioning tree is used to divide CTUs from a 64x64 level to a 4x4 level, with some additional constraints on 8x8 blocks. Note that partitions designated as R refer to recursion, as the same partitioning tree is repeated at lower sizes until we reach the lowest 4x4 level.

[0087] Similarly, in the video coding scheme under the AV1 standard, video frames are divided into a series of superblocks. For example... Figure 12 As shown, the partition tree expands to a 10-way structure. Simultaneously, the maximum size (referred to as the superblock in VP9 / AV1 terminology) increases from 128x128. Note that this includes 4:1 / 1:4 rectangular partitions not present in the VP9 standard video coding scheme. No rectangular partition can be further subdivided. In addition to the block size, a coding tree depth is defined to indicate the partition depth starting from the root node. Specifically, the coding tree depth at the root node, for example 128x128, is set to 0, and the coding tree depth increases by 1 each time a block is further split.

[0088] Luminance blocks can be divided into transform units of varying sizes, which can be represented by recursively partitioning downwards by two levels, without enforcing a fixed transform unit size. To merge extended block partitions, square transform sizes from 4x4 to 64x64 (2:1 / 1:2 and 4:1 / 1:4) are supported. For chroma blocks, only the largest possible transform units are allowed.

[0089] refer to Figure 6 In S630, after the current frame is divided into a series of CBs, the residual block and MRLP flag of each CB are obtained using Multi-Reference Line Intra-Prediction (MRLP), and the reference sample of the reference line selected to act on the intra-prediction is stored in the line buffer.

[0090] For each block, the intra-prediction mode is derived based on the multi-reference rows of adjacent blocks. A subset of all available intra-prediction modes defined in the video coding scheme based on the AV1 standard is determined using each of the multi-reference rows. This subset of all available intra-prediction modes is also called the allowed intra-prediction modes (AIPM) set. Each of the multi-reference rows is identified by an index number, which is an integer greater than or equal to zero.

[0091] In the AV1 standard video coding scheme, there are 62 available intra-prediction modes, including 56 angular intra-prediction modes, 5 smoothing modes, and one mode that uses the luma component to predict the chroma component. When multiple reference lines are applied in intra-prediction and these multiple reference lines include all intra-prediction modes, the video coding complexity is considerable. These multiple reference lines consist of adjacent reference lines with index number zero and one or more non-adjacent reference lines with index numbers greater than zero. It is necessary to limit the number of intra-prediction modes allowed in each reference line to reduce the video coding complexity.

[0092] MRLP may have certain limitations. For narrow blocks, MRLP may be disabled. The narrow block has a width of W, a height of H, and abs(log2W-log2H)>1. For small blocks, MRLP may be disabled. Small blocks are 8x8 in size or smaller. Other limitations may also apply.

[0093] In the video coding method of this disclosure embodiment, MRLP is employed to improve coding efficiency and compression performance. The corresponding intra-prediction mode determined by each multiple reference line based on coding efficiency and compression performance is a subset of all available intra-prediction modes, defined in the video coding schemes of various coding standards. This subset is also referred to as the AIPM set. The intra-prediction modes included in the AIPM set are signaled by the video coding device, for example... Figure 4 The video encoder 400 shown is used. Since the number of intra-prediction modes included in the AIPM set is less than the number of all available intra-prediction modes defined in the corresponding video coding scheme, fewer bits are needed to inform the intra-prediction modes included in the AIPM set. Furthermore, in addition to the intra-prediction modes, symbol levels can also be predicted for each block in the inter-prediction modes.

[0094] After obtaining the residual block, the residual block is transformed and quantized. The residual block is the difference between the current CB and the predicted CB.

[0095] The AV1 standard's video coding scheme includes 56 angular (or directional) intra-prediction modes, 5 non-angular smooth intra-prediction modes, one chroma-only intra-prediction mode, and several other intra-prediction modes. The eight directional (or angular) modes correspond to angles ranging from 45° to 207°. To utilize the greater spatial redundancy in directional textures, the directional intra-prediction modes are extended to have a finer-grained set of angles. Figure 13 Eight nominal angles of an intra-prediction mode according to an embodiment of this disclosure are shown. For example... Figure 13 As shown, the angles of the eight directional modes vary slightly. These eight nominal angles, also known as nominal interior angles, are named V_PRED, H_PRED, D45_PRED, D135_PRED, D113_PRED, D157_PRED, D203_PRED, and D67_PRED, corresponding to 90°, 180°, 45°, 135°, 113°, 157°, 203°, and 67°, respectively. Each nominal angle is extended into seven finer angles. Therefore, a total of 56 directional angles (also known as interior angles or prediction angles) are defined. Each prediction angle is represented by the nominal interior angle plus an angle increment, i.e., a step size of -3 to 3 multiplied by 3°. To achieve directional intra-prediction modes in a general manner, all 56 directional intra-prediction modes use a unified directional prediction implementation, which projects each pixel onto a reference sub-pixel position and interpolates the reference pixel using a 2-tap bilinear filter.

[0096] The intra-frame prediction modes also include five non-directional smooth intra-frame prediction modes: DC mode, PAETH mode, SMOOTH mode, SMOOTH_V mode, and SMOOTH_H mode. For the DC mode, the average of the left and top adjacent samples is used as the predicted value for the block to be predicted. For the PAETH mode, reference samples are first obtained from the top, left, and top-left sides, and then the closest value (top + left - top-left) is set as the predicted value for the pixel to be predicted. Figure 14 The top, left, and upper left positions of the PAETH pattern according to an embodiment of this disclosure are shown. For the SMOOTH pattern, the SMOOTH_V pattern, and the SMOOTH_H pattern, they predict blocks by quadratic interpolation in the vertical or horizontal direction, or by averaging the two directions.

[0097] To capture the attenuation spatial correlation of references at the edges, a FILTER INTRA mode was designed for the luma blocks. Five filter internal modes were defined. Each of these five filter internal modes is represented by a set of eight 7-tap filters, reflecting the correlation between pixels in a 4x2 block and their seven neighboring pixels. In other words, the weighting factors of the 7-tap filters are position-dependent. Figure 15An example of a recursive intra-frame filtering mode according to an embodiment of this disclosure is shown. For example... Figure 15 As shown, the 8x8 block is divided into eight 4x2 patches. These patches are... Figure 15 The pixels in the current patch are represented by B0, B1, B2, B3, B4, B5, B6, and B7. For each patch, the seven neighboring patches, represented by R0 to R7, are used to predict the pixels in the current patch. For patch B0, all neighboring patches have been reconstructed. However, for other patches, not all neighboring patches have been reconstructed. The predicted values ​​of the neighboring patches are used as a reference. For example, all neighboring patches of patch B7 have not been reconstructed, so the predicted samples of the neighboring patches (i.e., B5 and B6) are used instead.

[0098] Chromaticity component prediction using the luma component (CfL) is a chroma-only intra-frame prediction that models chroma pixels as a linear function of the reconstructed luma pixels. CfL prediction is represented as follows:

[0099] CfL(α)=α×L AC +DC

[0100] Among them, L AC The AC contribution of the luminance component is represented by α, the parameter of the linear model is represented by α, and the DC contribution of the chrominance component is represented by DC. Specifically, the reconstructed luminance pixels are resampled to the chrominance resolution, and then the average value is subtracted to obtain the AC contribution. To estimate the chrominance AC component from the AC contribution, CfL determines the parameter α based on the original chrominance pixels and represents them as signals in the bitstream without requiring the decoder to calculate scaling parameters. This method reduces the complexity of the decoder and enables more accurate predictions. As for the DC contribution of the chrominance component, it is calculated using intra-frame DC mode, which is sufficient for most chrominance content and has mature and fast implementation methods.

[0101] The MRLP uses multiple reference lines for intra-frame prediction, and Figure 4 The video encoder 400 shown determines and signals which reference line will be used to generate intra-prediction. The index number of the reference line is signaled before the intra-prediction mode is determined, and the most probable mode (MPM) or its equivalent mode is only allowed if the notified index number of the reference line is non-zero. Figure 16 A schematic diagram of four reference rows according to an embodiment of this disclosure is shown. For example... Figure 16 As shown, an example with four reference rows is given, where each reference row consists of six partitions (only four partitions are shown in the figure), namely partitions A to F, along with the upper left reference sample. Furthermore, partitions A and F are filled with the nearest samples from partitions B and E, respectively.

[0102] like Figure 16As shown, the index number of the reference row increases sequentially as the distance between the reference row and the current block increases. The reference row immediately adjacent to the current block is called the adjacent reference row, and its index number is zero, as shown below. Figure 16 Reference row 0 in the index. Reference rows with index numbers greater than zero are called non-adjacent reference rows.

[0103] Reference Figure 6 In S640, the current frame is encoded using residual blocks of each CB to generate a bitstream comprising encoded residual blocks of each CB. The residual blocks obtained via MRLP for each CB are transformed and quantized. Transformations of large blocks can reach sizes up to 64x64. For transform blocks of size 64, high-frequency transform coefficients are zeroed out, leaving only low-frequency coefficients. When transforming large blocks in transform skip mode, the entire block is used without zeroing out any values. Furthermore, Multiple Transform Selection (MTS) supports kernel transform. To control MTS, separate enable flags are applied at the Series Parameter Set (SPS) level for intra-frame and inter-frame prediction. When MTS is enabled at the SPS level, a CU level flag signal is emitted to indicate whether MTS is applied. Other transform features, such as Low Frequency Inseparable Transform (LFNST) and Subblock Transform (SBT), can also be supported.

[0104] Furthermore, the transformation and quantization process supports up to 63 quantization parameters (QPs). Flexible luma-to-chroma QP mapping is also supported through the luma-to-chroma mapping represented as signals in SPS. For CU-level QP adaptation, where the QP increment values ​​of the luma and chroma components can be represented as signals separately, this is also supported. Other quantization features, such as dependent correlation quantization, are also supported.

[0105] Furthermore, the residual blocks after transformation and quantization are entropy-encoded. The entropy encoding algorithm can be Context Adaptive Binary Arithmetic Coding (CABAC). A CABAC encoding engine can be used to encode the syntactic elements output by the transformation and quantization processes. The CABAC encoding engine supports arithmetic encoding algorithms, such as the Golomb-Rice encoding algorithm. Transform blocks and transform skip blocks can support separate residual coding structures. The transform coefficients of a coding block can be encoded using non-overlapping coefficient groups (CGs or sub-blocks), and each CG contains the transform coefficients of a variable-size coding block. The size of the coefficient group is chosen solely by the transform block size, regardless of the channel type. The sizes of CGs can include 1x16, 2x8, 8x2, 2x4, 4x2, and 16x1. The CGs within each coding block and the transform coefficients within each CG are encoded according to a predefined scan order.

[0106] Furthermore, a bitstream comprising entropy-coded residual blocks for each CB is generated. As a result of entropy coding, a bitstream comprising entropy-coded residual blocks for each CB is generated. The bitstream can be transmitted and / or stored. A video decoder can perform a video coding method according to this disclosure to recover the original video.

[0107] Therefore, this disclosure provides a video coding method. The method selects multiple reference lines in intra-frame prediction and determines the values ​​of reference samples for the selected reference lines. The values ​​of the reference samples for the selected reference lines are stored in a line buffer. The selection of reference lines and the determination of reference sample values ​​are optimized to reduce the size of the line buffer. Therefore, the on-chip circuit area dedicated to the line buffer is reduced, saving hardware costs.

[0108] According to embodiments of this disclosure, encoders and decoders can be implemented by processing circuitry (e.g., one or more processors or one or more integrated circuits) that execute program instructions stored in a non-transient computer-readable storage medium.

[0109] Figure 17 An example of the top reference row and left reference column of an embodiment of this disclosure is shown. For example... Figure 17 As shown, multiple reference rows are further divided into upper reference rows and left reference columns. The upper reference rows are marked in white, and the reference samples in the upper reference rows are marked with partitions B, C, and D. The left reference columns are marked in gray, and the reference samples in the left reference columns are marked with partition A.

[0110] When the first row of a superblock is located inside the current block (CB), i.e., the current CB is located at the upper boundary of the superblock, reference samples from adjacent upper reference rows (also called upper reference row 0) are used for intra-frame prediction of the current CB, instead of reference samples from non-adjacent upper reference rows. On the other hand, reference samples from both adjacent and non-adjacent left reference columns can be used for intra-frame prediction of the current CB. Regardless of whether the first row of the superblock is located inside the current CB, the index number of the reference row is represented by the same signal. Further, the signal representation of the reference row index number depends on whether the first row of the superblock is located inside the current CB.

[0111] In one embodiment, when the first row of the superblock is located within the current CB, combinations of adjacent upper reference rows and various left reference columns can form various reference samples for intra-frame prediction of the current CB. This combined signal is represented by the second index number of the reference row selected for intra-frame prediction. Figure 18-21 An example of selecting an upper reference row 0 and different left reference columns for intra-frame prediction according to an embodiment of this disclosure is shown. In one example, a combination of adjacent upper reference rows and adjacent left reference columns is used in... Figure 18The text is marked with a diagonal texture pattern. In another example, the combination of the adjacent upper reference row and left reference column 1 is... Figure 18 The text is marked with a diagonal texture pattern. In another example, the combination of the adjacent upper reference row and left reference column 2 is... Figure 20 The text is marked with a diagonal texture pattern. In another example, the combination of the adjacent upper reference row and left reference column 3 is... Figure 21 The text uses a diagonal texture pattern for marking.

[0112] When there is no reference sample in either the upper reference row or the left reference column, the same reference sample filling process is used for both the upper reference row and the left reference column.

[0113] A smoothing filter is used to filter the reference samples selected for the reference rows used in intra-prediction. Different smoothing filters are used for the reference samples of different reference rows selected for intra-prediction. In one example, when the left reference column selected for intra-prediction is not an adjacent left reference column, the smoothing filter for the adjacent upper reference row and the non-adjacent left reference column used for intra-prediction will not work. In another example, when the left reference column selected for intra-prediction is an adjacent left reference column, the smoothing filter for the adjacent left reference column selected for intra-prediction will not work.

[0114] In one embodiment, when the first row of the superblock is inside the current CB and the reference row with index number r is signaled, it indicates that the r-th non-adjacent reference row is being used for intra-frame prediction. Similar to the normal MRLP scheme, the reference sample from the left is selected from the r-th column, and the reference sample at the top is filled with a single value.

[0115] In one embodiment, when the first line of the superblock is within the current CB, a non-adjacent upper reference row with a second index number r and a non-adjacent left reference column with the same second index number r are selected for intra-frame prediction. Furthermore, the reference sample of the non-adjacent upper reference row with the second index number r is set to a single value. In one example, the single value is determined by selecting the value of the reference sample from either the adjacent upper or left reference column. In another example, the single value is determined by selecting 1 << (BitDepth - 1) or (1 << (BitDepth - 1)) - 1, where BitDepth is the number of bits used to encode the reference sample.

[0116] In one embodiment, when the first row of the CTU is located within the current CB, a non-adjacent upper reference row with a second index number r and a non-adjacent left reference column with the same second index number r are selected for intra-frame prediction. Additionally, the value of the reference sample in the non-adjacent upper reference row with the second index number r is determined by projecting the position of the reference sample in the adjacent upper reference row onto the non-adjacent upper reference row with the second index number r in the angular intra-frame prediction direction.

[0117] The value of the reference sample of the non-adjacent upper reference row with index r, obtained by projecting the positions of the reference samples of the adjacent upper reference row onto the non-adjacent upper reference row with index r in the intra-angular frame prediction direction, can be determined by the following process: First, starting from the integer position of the reference sample of the upper reference row with index r, draw parallel lines to the adjacent upper reference rows in the intra-angular frame prediction direction. For example... Figure 22 As shown, the parallel lines intersect the adjacent upper reference row at the score position of the reference sample. Then, interpolation filtering is applied to the values ​​at the score positions of the reference samples in the adjacent upper reference row to obtain the values ​​of the reference samples in the non-adjacent upper reference row with the second index number r. The interpolation filter used in the interpolation filtering process can be a 2-tap filter, a 4-tap filter, or a 6-tap filter. The interpolation filter may include negative filter coefficients. Alternatively, the interpolation filter can be a 4-tap cubic filter, a 4-tap discrete cosine transform interpolation filter (DCT-IF), or a 2-tap bilinear filter.

[0118] In another embodiment, after the parallel line intersects the adjacent upper reference line at the fractional position of the reference sample of the adjacent upper reference line, the reference sample of the adjacent upper reference line for intra-frame prediction is selected using the integer position closest to the fractional position.

[0119] In various embodiments of this disclosure, the video coding method selects multiple reference lines for intra-frame prediction and determines the values ​​of reference samples for the reference lines used for multi-reference-line intra-frame prediction. The values ​​of the reference samples for the selected reference lines are stored in a line buffer. Optimizing the selection of reference lines and the determination of reference sample values ​​reduces the size of the line buffer. This, in turn, reduces the on-chip circuit area dedicated to the line buffer, saving hardware costs.

[0120] This disclosure also provides a video decoding method. Figure 7 An exemplary video decoding method according to an embodiment of this disclosure is shown. For example... Figure 7 As shown, a bitstream comprising multiple encoded frames of video input is obtained (in S710). This bitstream can be generated by performing an operation such as... Figure 6 The video encoding system generated by the video encoding method shown is produced.

[0121] In S720, each of the multiple encoded frames is decoded into multiple superblocks, and each of the multiple superblocks is decoded into multiple residual blocks. The bitstream includes header information to indicate the boundaries of the multiple superblocks and the boundaries of the multiple residual blocks contained within each superblock. Each of the multiple residual blocks can be entropy-coded. The entropy decoding process is followed by an inverse transform and inverse quantization process to obtain each of the multiple residual blocks. The entropy decoding process, along with the inverse transform and inverse quantization processes, are... Figure 6 The entropy coding process, transformation, and inverse quantization process performed by S640 in the video coding method shown are illustrated.

[0122] In S730, the coded blocks of each of multiple residual blocks are recovered based on MRLP flags and reference samples included in each coded frame. Specifically, the CB is decoded from the corresponding residual block by performing the inverse MRLP process based on MRLP flags and reference samples included in each coded frame, such as... Figure 6 S630 in the video encoding method shown.

[0123] Then, the recovered CB of each of the multiple residual blocks is stored in the frame buffer to reconstruct each frame of the video signal (in S740), said multiple residual blocks being divided from each of the multiple superblocks. This process is performed in reverse as follows: Figure 6 The video encoding method S620 shown assembles multiple CBs into a superblock, and then assembles multiple superblocks into a single video signal.

[0124] Finally, the combined frames are output sequentially to recover the video signal (in the S750). This process is... Figure 6 The reverse process of S610 of the video encoding method shown.

[0125] In this embodiment of the disclosure, the video encoding method and the corresponding video decoding method select multiple reference lines for intra-frame prediction and determine the values ​​of reference samples for the multiple reference lines used for intra-frame prediction. The values ​​of the reference samples of the selected reference lines are stored in a line buffer. The selection of reference lines and the determination of reference sample values ​​are optimized to reduce the size of the line buffer. Therefore, the on-chip circuit area dedicated to the line buffer is reduced, saving hardware costs.

[0126] Although the principles and implementations of this disclosure have been described using specific embodiments in the specification, the description of the above embodiments is merely to help understand the methods and core ideas of this disclosure. Furthermore, those skilled in the art can modify specific implementations and application scopes based on the ideas of this disclosure. In short, the content of this specification should not be construed as limiting this disclosure.

Claims

1. A video decoding method, characterized in that, include: Acquire a bitstream, which includes multiple encoded frames of a video signal and header information, wherein the header information is used to indicate the boundaries of multiple superblocks and the boundaries of multiple residual blocks contained in each superblock; Each of the plurality of encoded frames is decoded into a plurality of superblocks, and each of the plurality of superblocks is decoded into a plurality of residual blocks; Based on the multi-reference row intra-prediction flag and reference samples in each coded frame, each of the multiple residual blocks is recovered into a coded block. The multiple reference rows are divided into upper reference rows and left reference columns, and one upper reference row and one left reference column are selected for intra-prediction. Each row of the superblock is identified by a first index number, which increases sequentially from top to bottom. Each row of the upper reference row and each column of the left reference column are identified by a second index number, which increases sequentially in the direction away from the current coded block. The upper reference rows immediately adjacent to the current coded block are adjacent upper reference rows, and the remaining upper reference rows are non-adjacent upper reference rows. The left reference columns immediately adjacent to the current coded block are adjacent left reference columns, and the remaining left reference columns are non-adjacent left reference columns. Each frame of the video signal is reconstructed by storing a recovery-coded block of each of the plurality of residual blocks in a frame buffer; and The reconstructed frames are continuously output to recover the video signal; The step of selecting an upper reference row and a left reference column for intra-frame prediction includes: when the first row of the superblock is located within the current coding block, selecting the adjacent upper reference row and a left reference column for intra-frame prediction; or, When the first row of the superblock is located within the current coding block, a non-adjacent upper reference row with a second index of r and a non-adjacent left reference column with the same second index of r are selected for intra-frame prediction; and Set the reference sample of the non-adjacent upper reference row with the second index number r to a single value; or, When the first row of the superblock is located within the current coding block, a non-adjacent upper reference row with a second index of r and a non-adjacent left reference column with the same index of r are selected for intra-frame prediction. The value of the reference sample of the non-adjacent upper reference row with the second index number r is determined by projecting the position of the reference sample of the adjacent upper reference row onto the direction predicted within the angle frame.

2. The video decoding method as described in claim 1, characterized in that, When either the upper reference row or the left reference column lacks a reference sample, the same reference sample is used to fill both the upper reference row and the left reference column.

3. The video decoding method as described in any one of claims 1-2, characterized in that, Also includes: The reference samples for different reference rows used for intra-frame prediction employ different smoothing filters.

4. The video decoding method as described in claim 3, characterized in that, The smoothing filter satisfies the following condition: when the left reference column selected for intra-frame prediction is a non-adjacent left reference column, the smoothing filter for the adjacent upper reference row and the non-adjacent left reference column selected for intra-frame prediction is disabled.

5. The video decoding method as described in claim 3, characterized in that, The smoothing filter satisfies the following condition: when the selected left reference column is an adjacent left reference column, only the smoothing filter for the adjacent left reference column selected for intra-frame prediction is disabled.

6. The video decoding method as described in claim 1, characterized in that, When the reference sample of the non-adjacent upper reference row with the second index number r is a single value, the single value is determined by selecting the value of the reference sample from the adjacent upper reference row or from the adjacent left reference column.

7. The video decoding method as described in claim 1, characterized in that, When the reference sample of the non-adjacent upper reference row with the second index number r is set to a single value, the single value is determined by selecting 1<<(bit depth - 1) or (1<<(bit depth - 1)) - 1; where the bit depth is the number of bits used to encode the reference sample.

8. The video decoding method as described in claim 1, characterized in that, The step of projecting the position of the reference sample of the adjacent upper reference row onto the non-adjacent upper reference row with the second index number r in the direction predicted within the angle frame, to determine the value of the reference sample of the non-adjacent upper reference row with the second index number r, includes: Starting from the integer position of the reference sample in the upper reference row with the second index number r, draw parallel lines in the direction predicted within the angle frame; Intercept the parallel line at the fractional position of the reference sample in the adjacent upper reference row; and Interpolation filtering is applied to the values ​​of the reference samples at the fractional positions of the adjacent upper reference rows to obtain the values ​​of the reference samples of the non-adjacent upper reference rows with the second index number r; The interpolation filter used in the interpolation filtering process is a 6-tap filter, a 4-tap cubic filter, a 4-tap discrete cosine transform interpolation filter, or a 2-tap bilinear filter; the interpolation filter includes negative filter coefficients.

9. The video decoding method as described in claim 1, characterized in that, The step of projecting the position of the reference sample of the adjacent upper reference row onto the non-adjacent upper reference row with the second index number r in the direction predicted within the angle frame, to determine the value of the reference sample of the non-adjacent upper reference row with the second index number r, includes: Starting from the integer position of the reference sample in the upper reference row with the second index number r, draw parallel lines in the direction predicted within the angle frame; Intercept the parallel line at the fractional position of the reference sample in the adjacent upper reference row; and Apply interpolation filtering to the value of the reference sample in the adjacent upper reference row at the integer position closest to the fraction position to obtain the value of the reference sample in the non-adjacent upper reference row with the second index number r; The interpolation filter used in the interpolation filtering process is a 6-tap filter, a 4-tap cubic filter, a 4-tap discrete cosine transform interpolation filter, or a 2-tap bilinear filter; the interpolation filter includes negative filter coefficients.

10. A video encoding method, characterized in that, include: Get the current frame of the video input; Based on the nested multi-type tree coding block structure, the acquired current frame is divided into a series of super blocks, and each super block is divided into multiple coding blocks; The residual block of each coding block is obtained through multi-reference row intra-frame prediction. The multi-reference row intra-frame prediction flag and reference samples of the reference rows selected for intra-frame prediction are stored in a row buffer. Multiple reference rows are divided into upper reference rows and left reference columns, and one upper reference row and one left reference column are selected for intra-frame prediction to minimize the size of the row buffer. Each row of the superblock is identified by a first index number, which increases sequentially from top to bottom. Each of the upper reference rows and the left reference columns is identified by a second index number, which increases sequentially in the direction away from the current coding block. The upper reference rows immediately adjacent to the current coding block are adjacent upper reference rows, and the remaining upper reference rows are non-adjacent upper reference rows. Similarly, the left reference columns immediately adjacent to the current coding block are adjacent left reference columns, and the remaining left reference columns are non-adjacent left reference columns. The current frame is encoded using the residual blocks of each coded block to generate a bitstream, the bitstream including the coded residual blocks of each coded block; The step of selecting an upper reference row and a left reference column for intra-frame prediction includes: when the first row of the superblock is located within the current coding block, selecting the adjacent upper reference row and the left reference column for intra-frame prediction; or, When the first row of the superblock is located within the current coding block, a non-adjacent upper reference row with a second index of r and a non-adjacent left reference column with the same second index of r are selected for intra-frame prediction; and Set the reference sample of the non-adjacent upper reference row with the second index number r to a single value; or, When the first row of the superblock is located within the current coding block, a non-adjacent upper reference row with a second index of r and a non-adjacent left reference column with the same index of r are selected for intra-frame prediction. The value of the reference sample of the non-adjacent upper reference row with the second index number r is determined by projecting the position of the reference sample of the adjacent upper reference row onto the direction predicted within the angle frame.

11. The video encoding method as described in claim 10, characterized in that, When either the upper reference row or the left reference column lacks a reference sample, the same reference sample is used to fill both the upper reference row and the left reference column.

12. The video coding method according to any one of claims 10-11, characterized in that, The reference samples for different reference rows used for intra-frame prediction employ different smoothing filters.

13. The video encoding method as described in claim 12, characterized in that, The smoothing filter satisfies the following condition: when the left reference column selected for intra-frame prediction is a non-adjacent reference column, the smoothing filter for the adjacent upper reference row and the non-adjacent left reference column selected for intra-frame prediction is disabled.

14. The video encoding method as described in claim 12, characterized in that, The smoothing filter satisfies the following condition: when the left reference column selected for intra-frame prediction is an adjacent left reference column, the smoothing filter for the adjacent left reference column selected for intra-frame prediction is disabled only.

Images