Video encoding and decoding method and apparatus for triangular prediction

Abstract

A method and apparatus for video encoding and decoding is provided. [Solution] A method for video encoding and decoding includes partitioning a video image into a plurality of coding units (CUs), at least one of which is further partitioned into two prediction units (PUs), each including at least one triangular PU having one of partition directions from the upper left corner to the lower right corner and from the upper right corner to the lower left corner; constructing a single-prediction motion vector candidate list; determining whether a current CU is to be coded as a triangular prediction mode according to coded information; signaling a partition direction flag indicating the partition direction; and signaling an index value indicating a selected entry in the constructed single-prediction motion vector candidate list.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application is a patent application filed on December 30, 2018, entitled "Video Encoding and Decoding No. 62 / 786,545, entitled "Triangle Prediction Flag Signaling in a Digital Signal Processor," and and a patent application filed on December 31, 2018, entitled "Triangle Prediction Syntax Encoding and Decoding" This application claims priority to U.S. Provisional Application No. 62 / 787,219, which is hereby incorporated by reference. The entire specification of this patent application is incorporated herein by reference. [Technical Field]

[0002] The present invention relates generally to video encoding, decoding and compression, and more particularly, but not exclusively, to video METHOD AND APPARATUS FOR MOTION COMPENSATED PREDICTION USING A TRIANGULAR PREDICTION UNIT IN ELECTRON CODING AND DECODING - Patent application Regarding. [Background technology]

[0003] The following abbreviations and acronyms are defined herein, at least some of which may be used in the following descriptions: Used in. International Telecommunication Union (ITU), ITU Telecommunication ITU Telecommunication Standardization Sector (ITU-T) ISO / IEC: International Organization for Standardization, International IEC (International Electrotechnical Commission), Video Expert Group Moving Picture Experts Group (MPEG), Advanced Video Coding (AVC) Video Coding), High Efficiency Video Coding (HEVC), Versatile Video Coding (VVC), Joint Exploration Model (JEM) Joint Exploration Test Model), VVC Test Model (VTM), Joint Joint Video Experts Team (JVET), a video coding expert group Video Coding Experts Group (VCEG), Motion Vector (MV) ), Motion Vector Prediction (MVP), Motion Vector Difference (MVD :Motion Vector Difference), Motion Vector Field (MVF), High Advanced Motion Vector Prediction (AMVP), motion vector Motion Vector Competition (MVC), Temporal Motion Vector Prediction (TMVP) Temporal Motion Vector Prediction, Control Point Motion Vector (CPMV) Motion Vector), Adaptive Loop Filter (ALF), Bi-Prediction (B: Bi-predictive), Block Copy (BC: Block Copy), Context-based Adaptive Context-based Adaptive Binary Arithmetic Coding (CABAC) coding), Context Adaptive Variable Length Coding (CAVLC) Length Coding, Coding Block (CB), Encoder / Decoder (CO DEC), Coded Picture Buffer (CPB), Coding Tree Block CTB (Coding Tree Block), CTU (Coding Tree Unit) ), Coding Unit (CU), Discrete Cosine Transform (DCT) ne Transform), Decoded Picture Buffer (DPB), Intra ( I), Intra Block Copy (IBC), Prediction (P), Predicted Block (P B: Prediction Block), Probability Interval Partition Entropy (PIPE: Probability Interval Partitioning Entropy, Picture Order Count (POC), Image Unit Units (PU), Sum Of Absolute Difference (SAD) , Sample Adaptive Offset (SAO), Syntax-based Content Syntax-Based Context-Adaptive Binary Arithmetic Coding (SBAC) Rithmetic Coding, Sequence Parameter Set (SPS) , Sum of Square Difference (SSD), Transform Unit (TU) m Unit), Quadtree Plus Binary Tree (QBTB), pixel or Picture elements (pels), Advanced Temporal Motion Vector Prediction (ATMVP) ral Level Motion Vector Prediction (UMVE), Ultimate Motion Vector Expression Motion Vector Expression, Merge Mode by Motion Vector Difference (MMVD:Merge M (code with Motion Vector Difference), index (IDX), inter prediction and inter Combined Inter and Intra Prediction (CIIP), the most likely Most Probable Mode (MPM).

[0004] Digital television, laptop or desktop computer, tablet computer Computers, digital cameras, digital recording devices, digital media players, Video game consoles, smartphones, video conferencing equipment, video streaming equipment, etc. All electronic devices in the world support digital video. Electronic devices perform video compression / decompression. It receives, transmits, encodes, decodes and stores digital video data. Digital video equipment includes Versatile Video Coding (VVC), Joint Exploration Model (JEM), MPEG-2, MPEG-4, ITU-T H.263, ITU-T H.264 / MPEG- 4, Part 10, Advanced Video Coding (AVC), ITU-T H.265 / High Efficiency Video Video Coding (HEVC) as defined in the standards and extensions of such standards Implement coding and decoding techniques.

[0005] Video encoding and decoding generally involves prediction due to redundancies present in a video image or sequence. Video coding and decoding techniques are used. One of the key goals of this technique is to maximize the video quality while avoiding or minimizing video degradation. The goal is to compress data into a form with a lower bit rate. As more services become available, coding / decoding techniques with better coding / decoding efficiency will become available. is required.

[0006] Video compression typically involves spatial (intra-frame) prediction and / or temporal (inter-frame) prediction. to reduce or remove redundancy inherent in the video data. In video coding based on IEEE 802.11b, a video frame is called a coding tree unit (CTU). Each CTU is divided into one or more slices, each containing a number of video blocks. Contains one coding unit (CU) or until a predetermined minimum CU size is reached. Each CU (also called a leaf CU) may be recursively split into smaller CUs. It consists of one or more transform units (TUs) and one or more prediction units (PUs). Each CU can be in intra, inter, or IBC mode. Intra-coded video within a video frame can be coded in any order. (I) Video blocks in a slice are based on references in neighboring blocks in the same video frame. The image is coded using spatial prediction with respect to the reference sample. The video blocks in a coded (P or B) slice are Spatial predictions with respect to reference samples in neighboring blocks, or other previous and / or future The temporal prediction is based on the reference samples in the reference video frame.

[0007] Previously coded reference blocks, e.g., based on spatial or temporal prediction of neighboring blocks. The prediction block for the current video block being coded is obtained by using the reference block. The process of finding can be realized by a block matching algorithm. Residual data indicating pixel differences between the current block to be coded and the predicted block is The residual block is called the prediction error. Inter-coded blocks are used to generate the prediction block. A motion vector pointing to a reference block in the selected reference frame and a residual block are then The process of determining motion vectors is usually called motion estimation. The coded block is the one coded by the intra prediction mode and the residual block. For the resulting compression, the residual block is transformed from the pixel domain to a transform domain, e.g., the frequency domain. , resulting in the residual transform coefficients quantified in the future. Then, first in a two-dimensional matrix The aligned and quantified transform coefficients are scanned to generate a one-dimensional transform coefficient vector, which It is then entropy coded into a video bitstream to achieve further compression. do.

[0008] and storing the encoded video bitstream on a computer-readable storage medium. stored in a memory card (e.g., flash memory) and transmitted by electronic devices with digital video capabilities. or transmitted directly to this electronic device via wire or wirelessly. The electronic device may, for example, analyze the encoded video bitstream and generate a Gets syntax elements from the bitstream and uses at least one of the syntax elements obtained from this bitstream. Also, based in part on this encoded video stream, digitally convert it back to its original format. Reconstructing video data allows video decompression (the opposite process to video compression described above). This reconstructed digital video data is then displayed on the electronic device. Reproduce in I.

[0009] As digital video quality progresses from high definition to 4Kx2K and even 8Kx4K, As the number of video streams increases, the amount of video data to be encoded / decoded increases exponentially. Efficiently encodes and decodes video data while maintaining the image quality of the encoded video data It is always a challenge to do so.

[0010] At the Joint Video Experts Team (JVET) meeting, Versatile Video Coding (VVC) The first draft of the VVC Test Model 1 (VTM1) coding method was defined. A quadtree with nested multi-type trees based on a ternary partition coding block structure is used for VVC. Since then, the coding method Reference software for running VTM and Draft VVC decoding processes is JVET Developed during the conference. Summary of the Invention

[0011] This disclosure generally relates to motion compensated prediction using a triangular prediction unit in video encoding and decoding. Examples of measurement techniques will be explained.

[0012] According to the first aspect of the present application, the video image is further divided into at least one upper left corner and a lower right corner. , and at least one triangle having one of the compartment directions from the upper right corner to the lower left corner. The coding unit (CU) is divided into two prediction units (PUs) including a PU. and constructing a single predicted motion vector candidate list, and determining whether the current CU is a determining whether to encode the image as a triangular prediction mode according to the encoded information; signaling a partition direction flag indicating the partition direction; and Signaling an index value indicating the selected entry in the motion vector candidate list and

[0013] According to a second aspect of the present invention, the video images are divided into at least one and at least one triplet. Multiple codes partitioned into two prediction units (PUs) including a rectangular prediction unit (PU) A triangle indicating the triangle prediction mode for the current CU. determining whether a shape prediction flag should be decoded and determining whether a previous shape prediction flag should be decoded for the current CU; When the triangle prediction flag is determined to be decoded, a context-based adaptive deriving the triangle prediction flags by binary arithmetic encoding (CABAC); and the CABAC context for the triangle prediction flag includes a region size of a CU. , at least one selected from the group consisting of the shape of the CU and the common context The present invention provides a method for video encoding and decoding that is derived based on the above.

[0014] In accordance with a third aspect of the present application, a processor and instructions executable by the processor are stored. and a memory configured to store therein, wherein the processor, upon executing the instructions, The video image is rotated at least once from the top left corner to the bottom right corner and once from the top right corner to the bottom left corner. Two prediction units each including at least one triangular PU having one of the partition directions The video is divided into multiple coding units (CUs) that are divided into PUs, and a single predicted motion vector is used. Construct a candidate list, and the current CU selects the triangle prediction mode according to the coded information. and determining whether to encode the segment, and notifying a segment direction flag indicating the segment direction by a signal. an index indicating a selected entry in the single-prediction motion vector candidate list generated by the and signaling a video codec value. To provide an apparatus for the

[0015] In accordance with a fourth aspect of the present application, a processor and instructions executable by the processor are stored. and a memory configured to store therein, wherein the processor, upon executing the instructions, A video image is further divided into at least one triangular prediction unit (PU) The image is divided into a plurality of coding units (CUs) each of which is divided into two prediction units PU, each of which includes a current Whether the triangle prediction flag indicating the triangle prediction mode for the current CU should be decoded and determining that the triangle prediction flag should be decoded for the current CU. Once this is determined, the above is coded using context-based adaptive binary arithmetic coding (CABAC). derive a triangle prediction flag, and perform operations such as The ABAC context is determined by the CU region size, CU shape, and common context. The business model is configured to be derived based on at least one selected from the group consisting of: An apparatus for video encoding and decoding is provided. [Brief explanation of the drawings]

[0016] A more particular description of the examples of the present disclosure will be given by reference to specific examples which are illustrated in the accompanying drawings. These drawings show only some examples and are therefore not limiting in scope. These examples may be used to illustrate additional features and advantages of the present invention, provided that they are not intended to be limiting. The isomerism and details are explained.

[0017] [Figure 1] FIG. 1 is a block diagram illustrating a video encoding and decoding system according to an embodiment of the present disclosure. [Figure 2] FIG. 2 is a block diagram illustrating a video encoder according to an embodiment of the present disclosure. [Figure 3] FIG. 3 is a block diagram illustrating a video decoder according to an embodiment of the present disclosure. [Figure 4] FIG. 4 is a schematic diagram illustrating a QTBI structure according to an embodiment of the present disclosure. [Figure 5] FIG. 5 is a schematic diagram illustrating an example of an image divided into CTUs, according to an embodiment of the present disclosure. [Figure 6] FIG. 6 is a schematic diagram illustrating a multi-type tree splitting mode according to an embodiment of the present disclosure. [Figure 7] FIG. 7 is a schematic diagram illustrating dividing a CU into triangular prediction units according to an embodiment of the present disclosure. [Figure 8] FIG. 8 is a schematic diagram illustrating the position of adjacent blocks according to an embodiment of the present disclosure. [Figure 9] FIG. 9 is a schematic diagram illustrating an example of weighting with a set of weighting factors according to an embodiment of the present disclosure. [Figure 10] FIG. 10 is a schematic diagram illustrating context derivation for encoding / decoding triangle prediction flags according to an embodiment of the present disclosure. [Figure 11] FIG. 11 is a schematic diagram illustrating an example of a PU compartment, according to an embodiment of the present disclosure. [Figure 12] FIG. 12 is a schematic diagram illustrating the locations of spatial merge candidates according to an embodiment of the present disclosure. [Figure 13]FIG. 13 is a schematic diagram illustrating motion vector scaling of temporal merging candidates according to an embodiment of the present disclosure. [Figure 14] FIG. 14 is a schematic diagram illustrating candidate locations for temporal merge candidates according to an embodiment of the present disclosure. [Figure 15] FIG. 15 is a block diagram illustrating an example apparatus for video encoding and decoding, according to an embodiment of this disclosure. [Figure 16] FIG. 16 is a flowchart illustrating an example process of video encoding and decoding for motion compensated prediction with triangular prediction, according to an embodiment of this disclosure. [Figure 17] FIG. 17 is a flowchart illustrating an example process of video encoding and decoding for triangle prediction flags, according to an embodiment of this disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0018] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described in detail below with reference to the accompanying drawings. In order to facilitate understanding of the concepts set forth herein, a number of specific, non-limiting details are provided. However, it will be understood by those skilled in the art that the present invention can be practiced in various modifications. For example, the subject matter described in this specification may be multimedia equipment with digital video capabilities. It will be apparent to those skilled in the art that the invention may be implemented in many different types of electronic devices.

[0019] The description of elements in each figure may refer to elements in other figures. It refers to an element and includes variations of the same element.

[0020] As used herein, the terms "one embodiment," "embodiment," "example," "an embodiment," "an "Example" or similar phrases means that the particular feature, structure, or characteristic in the description is present in at least one Therefore, throughout this specification, "1 "in one embodiment," "in an example," "in an embodiment," and similar expressions The examples may, but do not necessarily, all refer to the same embodiment. This may or may not include all of the disclosed embodiments. Any features, structures, elements or characteristics described in connection with some embodiments may be used interchangeably unless expressly indicated otherwise. Unless otherwise indicated, other embodiments are also applicable.

[0021] In the drawings, schematic flow charts and / or schematic block diagrams are used to illustrate different embodiments. The architecture, functionality, and other features of possible embodiments of apparatus, systems, methods, and program products are described below. In this regard, each block in the schematic flow chart and / or the schematic block diagram A block is a module, segment, or one or more components that perform a specific logical function. However, those skilled in the art will recognize that The flowcharts do not necessarily have to be executed in the order shown, and may be executed in one or more particular order. It may be possible to perform the process without certain steps or with other steps not shown. should be understood.

[0022] It should be noted that in some variations, the functions of the blocks shown may occur out of the order depicted in the figures. For example, two blocks shown consecutively are actually substantially different due to their related functions. They may be performed simultaneously or in reverse order. The functions and methods may be identical in function, logic, or effect to one or more of the blocks, or portions thereof. It can be considered that the same.

[0023] The terminology used in this disclosure is for the purpose of describing particular examples only and is not intended to limit the scope of the disclosure. The terms "including" and "comprises" are not intended to be limiting unless expressly indicated otherwise. Unless otherwise specified, "including but not limited to" means "including but not limited to."

[0024] Furthermore, these terms may be used interchangeably with the functions, integers, steps, operations, elements and / or components being described. refers to the presence of one or more other features, integers, steps, operations, or elements , components, and / or combinations thereof. It should be understood that this is not the case.

[0025] The listing of enumerated items refers to any of these items or Not all are meant to be mutually exclusive.

[0026] As used in this disclosure and the appended claims, the singular forms "one," "one," and "an" are used interchangeably. "This" is intended to include the plural, unless the context clearly indicates otherwise. "one or more" and "at least one" should be interpreted as equivalent to "one or more."

[0027] As used herein, the term "and / or" refers to one or more related It is understood that the term "combination" also refers to and includes any and all possible combinations of the listed items. For example, "A and / or B" should be interpreted as meaning that only A exists, only B exists, This refers to any of three combinations of the coexistence of both A and B.

[0028] The symbol " / " generally indicates an "or" relationship between related terms, but it can also be used to indicate "and" between related terms. It may also include relationships. For example, "A / B" is used unless the context clearly dictates otherwise. , which may include the coexistence of both A and B.

[0029] Throughout this disclosure, terms such as "first," "second," "third," etc., all refer to the associated element, For example, for reference to devices, components, configurations, steps, etc. used only in this context and does not imply spatial or chronological order unless the context clearly dictates otherwise. For example, "first device" and "second device" are two separate A combined device or two parts, components, or operating states of the same device and can be named arbitrarily.

[0030] The first and second elements may be present independently. For example, some embodiments may include a first element. The second element can contain only the first element and not the second element. It may be described prior to the description of the first element or without the description of the first element. The "first step" of the process is after the "second step" or " can be executed simultaneously.

[0031] As used herein, "if..." or "if..." The term "and" is understood to mean "when" or "depending on" depending on the context. These terms, when they appear in a claim, are used to qualify the associated limitation or feature. For example, one way is to a) if condition X exists, then function or action X' is performed; and ii) if condition Y exists, then function or action X' is performed. If Y exists, then function or operation Y' is performed. is realized by including both the ability to perform action X' and the ability to perform function or action Y'. Both functions X' and Y' must be executed in multiple runs of this method at different times. Furthermore, the ability to detect or assess the satisfaction of condition X and the ability to assess the satisfaction of condition Y This may also be implemented with the ability to detect or assess the foot.

[0032] "module", "submodule", "circuit", "subcircuit", "unit" or The term "subunit" refers to code or programs that can be executed by one or more processors. Contains memory (shared, dedicated, or group) that stores instructions. A module is a collection of code or It may contain one or more circuits that may or may not store instructions. A module or circuit consists of one or more components connected directly or indirectly. These components may be physically connected to each other. , can be physically disconnected or adjacent to each other.

[0033] A unit or module may be implemented entirely in software, It may be implemented by hardware or a combination of hardware and software. A fully software implementation may be implemented by combining units or or modules linked together directly or indirectly to perform a specific function. It may contain functionally related code blocks or software components that can.

[0034] FIG. 1 illustrates a block diagram of a video block encoding and decoding system according to an embodiment of the present disclosure. As shown in FIG. 1, the system 10 is It includes a source device 12 that generates and encodes video data that is decoded by a destination device 14. The source device 12 and the destination device 14 may be desktop or laptop computers. computers, tablet computers, smartphones, set-top boxes, digital Televisions, cameras, display devices, digital media players, video game consoles, video The device may be any of a variety of electronic devices, including streaming devices, etc. In this example, source device 12 and target device 14 are equipped with wireless communication capabilities.

[0035] In one embodiment, target device 14 receives the encoded video to be decoded via link 16. Link 16 receives encoded video data from source device 12. The communication medium may be any type of communication medium or device that transfers the information to the target device 14. Link 16 transmits encoded video data from source device 12 to destination device 14. The encoded video data may be transmitted over a communication medium that can be directly transmitted in real time. The signal is modulated according to a communication standard, such as a wired communication protocol, and transmitted to the target device 14. The body may be exposed to the radio frequency (RF) spectrum or one or more physical The communication medium may be any wireless or wired communication medium, such as a transmission line. packet-based networks such as wide area networks or may be configured as part of a global network such as the Internet. The medium may include routers, switches, base stations, or other devices that facilitate communication from source device 12 to destination device 14. It may also include any other device that stands.

[0036] In another embodiment, the encoded video data is output from the output interface 22. The encoded video data stored in the storage device 32 is then transmitted to the storage device 32. The data can be accessed by the target device 14 via the input interface 28. The storage device 32 may include a hard drive, a Blu-ray disc, DVD, CD-ROM, flash memory, volatile or non-volatile memory, or coded and any other suitable digital storage medium for storing video data. It can include either distributed or locally accessed data storage media. In yet another example, the storage device 32 may be a file server or a source device 12. another intermediate storage device capable of holding the encoded video data generated by The target device 14 may support the stream via streaming or download. The video data stored in the file server can be accessed from the storage device 32. stores the encoded video data and transmits the encoded video data to the target device 14. The file server may be any type of computer that can communicate with the server. Servers can be web servers (e.g., for websites), FTP servers, network-attached storage The target device 14 may be a file system, a network attached storage (NAS) device, or a local disk drive. A wireless channel suitable for accessing the encoded video data stored on the file server ( Wi-Fi connection), wired connection (e.g., DSL, cable modem, etc.), or Transmits encoded video data over any standard data connection, including combinations thereof. The encoded video data can be accessed from the storage device 32. may be a streaming transmission, a download transmission, or a combination thereof. .

[0037] As shown in FIG. 1, the source device 12 includes a video source 18, a video encoder 20, and and an output interface 22. The video source 18 includes a video capture device (e.g. video cameras), video archives containing previously captured video, video conferencing A video feed interface for receiving video from content providers, and / or or a computer for generating computer graphics data as source video may include sources such as digital graphics systems, or a combination thereof. As an example, if the video source 18 is a video camera in a security surveillance system, If the source device 12 and the destination device 14 are cameras or video cameras, However, the embodiments described in this disclosure are generally applicable to video coding. and may be applicable to wireless and / or wired applications.

[0038] Captured video, pre-captured video, or computer-generated video The video can be encoded by video encoder 20. The video data is transmitted directly to the destination device 14 via the output interface 22 of the source device 12. Additionally (or alternatively), encoded video data can be transmitted. The data can then be accessed by the target device 14 or other device for decryption and / or The image data may be stored in the storage device 32 for playback or playback. 22 may further include a modem and / or a transmitter.

[0039] The target device 14 includes an input interface 28, a video decoder 30, and a display device 31. 4. Input interface 28 includes a receiver and / or modem, and includes link 16 Receives encoded video data via link 16 or streams The encoded video data provided to storage device 32 includes Various syntax elements that are generated and used by video decoder 30 to decode video data The encoded video data may be transmitted over a communication medium or stored on a storage medium. Such syntax elements, whether stored in the may include:

[0040] In some embodiments, target device 14 may include an integrated display or other device capable of communicating with target device 14. The display 34 may include an external display configured to display: The decoded video data is displayed to the user on a liquid crystal display (LCD), plasma display, or other display. display, organic light-emitting diode (OLED) display, or another type of display device The display device may be any of various display devices such as a display device.

[0041] The video encoder 20 and the video decoder 30 are compatible with VVC, HEVC, MPEG-4 , Part 10, Advanced Video Coding (AVC), or extensions of such standards It should be noted that this disclosure does not apply to any particular video encoding / decoding standard. It should be understood that the present invention is not limited to this standard and may be applicable to other video encoding / decoding standards. Video encoder 20 of source device 12 may support any of these current or future standards. Similarly, the target device 14 is configured to encode video data according to either Video decoder 30 may receive video data in accordance with any of these current or future standards. The method is configured to decode the

[0042] The video encoder 20 and the video decoder 30 each include one or more microphones. processors, digital signal processors (DSPs), application specific integrated circuits (ASICs) , Field Programmable Gate Array (FPGA), Discrete Logic, Software software, hardware, firmware, or any combination thereof. The encoder circuitry may be implemented as any of a variety of suitable encoder circuits. When implemented by software, the electronic device executes the software instructions in an appropriate non-transitory manner. Stored on a computer-readable medium and executed by one or more processors in hardware The video encoding / decoding operations described in this disclosure may be performed by executing instructions by the The video encoder 20 and the video decoder 30 are combined in their respective devices. One or more encoders integrated as part of a standard encoder / decoder (CODEC) It may be included in the reader or decoder.

[0043] FIG. 2 is a block diagram illustrating a video encoder 20 according to an embodiment described in this disclosure. 1 is a diagram showing a video encoder 20 that performs an input / output (INT) process on a video block within a video frame. Intra-predictive coding and inter-predictive coding can be performed. Relies on spatial prediction to detect spatial redundancy in video data within a particular video frame or image Inter-predictive coding relies on temporal prediction to reduce or eliminate reducing or eliminating temporal redundancy of video data within adjacent video frames or images in will be deleted.

[0044] As shown in FIG. 2, the video encoder 20 includes a video data memory 40, a prediction processing unit 4 1, a decoded picture buffer (DPB) 64, an adder 50, a conversion processing unit 52, a quantification unit 54, The prediction processing unit 41 includes a motion estimation unit 42, a motion compensation unit 56, and an entropy coding unit 56. unit 44, division unit 45, intra prediction processing unit 46, intra block copy (IBC) unit 4 8 and an intra / inter mode decision unit 49. Video encoder 20 also includes an inverse quantification unit 58 for video block reconstruction, an inverse transform processing unit 59 for video block reconstruction, and an inverse quantification unit 59 for video block reconstruction. The DPB 64 further includes a block 60 and an adder 62. Between the adder 62 and the DPB 64, Blocky artifacts are removed from the reconstructed video by filtering the boundaries between blocks. It is possible to install a deblocking filter to remove the block. In addition to this deblocking filter, an in-router The video encoder 20 may use a fixed or programmable filter 63. may be in the form of a single hardware unit, or may be in the form of one or more fixed or may be divided among programmable hardware units.

[0045] Video data memory 40 stores video data to be encoded by components in video encoder 20. The video data in the video data memory 40 is stored as, for example, video The DPB64 receives the video data from a source 18 via a video encoder 20. Used when encoding (e.g., in intra-predictive or inter-predictive coding modes) This is a buffer for storing reference video data. can be any of a variety of memory devices. In various examples, video data memory 40 may be on-chip with other components in video encoder 20, or or off-chip for those components.

[0046] As shown in FIG. 2, after receiving the video data, the division unit 45 in the prediction processing unit 41 Then, the video data is divided into video blocks. This division involves The video frame is divided into sub-frames according to a predetermined division structure, such as a quad-tree structure. This involves dividing the image into slices, tiles, or other larger coding units (CUs). A video frame may be divided into multiple video blocks (or video tiles). The prediction processor 41 divides the current video block into a set of blocks. For a given block, multiple inputs are selected based on the error results (e.g., coding rate and distortion level). one of the tra-predictive coding modes or one of the inter-predictive coding modes Then, the prediction processor selects one of a plurality of possible predictive coding modes, such as 41 provides the resulting intra or inter prediction coded block to adder 50 to generate the residual and add the resulting intra- or inter-prediction coded block to adder 6. 2 and then reconstruct the coded blocks to be used as part of the reference frame. The prediction processing unit 41 also receives a motion vector, an intra mode indicator, and a division The syntactic elements, such as the information and other syntactic information, are provided to an entropy coder 56 .

[0047] The intra prediction unit 46 in the prediction unit 41 selects an intra prediction block suitable for the current video block. To select the intra-prediction coding mode, the same frame as the current block to be coded is used. Intra prediction of the current video block for one or more neighboring blocks in a frame. The motion estimation unit in the prediction processing unit 41 can perform spatial prediction by performing motion estimation coding. 42 and motion compensation unit 44 calculate one or more prediction blocks in one or more reference frames. For each block, perform inter-predictive coding of the current video block to perform temporal prediction. The video encoder 20 performs multiple passes of encoding to, for example, encode video data. An appropriate coding mode may be selected for each block in the image.

[0048] In one embodiment, the motion estimator 42 estimates a predetermined time period for a sequence of video frames. The current video frame is compared to the predicted block in the reference video frame according to the pattern. Generate motion vectors that indicate the displacement of prediction units (PUs) of video blocks within a frame. The motion estimation unit 42 determines the inter prediction mode for the current video frame by generating the motion estimation result. The motion estimation performed by the The motion vectors are, for example, the vectors of the current video frame (or other coding units). unit) or a reference frame for the current video block being coded within the image. For a prediction block within a frame (or other coding unit), the current video It can indicate the displacement of the PU of a video block within a frame. Turns designates video frames in a sequence as P-frames or B-frames The intra BC unit 48 determines the motion vectors for inter prediction by the motion estimation unit 42. In a similar manner to determining the vectors, vectors for intra BC coding, e.g., block Alternatively, the motion estimation unit 42 may be used to determine the block vectors. You may do so.

[0049] Determined by sum of absolute differences (SAD), sum of squared differences (SSD) or other difference metrics With a scalable pixel difference, the predicted block closely matches the PU of the video block being coded. In one embodiment, the video encoding Da 20 calculates the sub-integer pixel position values ​​of the reference frame stored in DPB64 For example, video encoder 20 may use a quarter pixel position in the reference frame, Values ​​at 1 / 8 pixel positions, or other fractional pixel positions, may be interpolated. The estimation unit 42 performs a motion search process for all pixel positions and fractional pixel positions. This allows motion vectors with fractional pixel accuracy to be output.

[0050] The motion estimation unit 42 determines the position of the PU of the video block in the inter-prediction coded frame. and a first reference frame identifying one or more reference frames each stored in the DPB64. the first reference frame list (List0) or the second reference frame list (List1) Inter-predictive coding is performed by comparing the position of the predicted block in the selected reference frame with the position of the predicted block in the selected reference frame. The motion estimation unit 42 calculates the motion vector of the PU of the video block relative to the frame. The calculated motion vectors are sent to the motion compensation unit 44 and then to the entropy coding unit 56. Send.

[0051] The motion compensation performed by the motion compensation unit 44 involves the use of the motion determined by the motion estimation unit 42. The motion compensation unit 4 may include obtaining or generating a prediction block based on the motion vector. 4. After receiving the motion vector of the PU of the current video block, The predicted block to which this motion vector points is located in one of the DPB64 blocks. This predicted block is found and transferred to adder 50. 0 is provided by motion compensation unit 44 from the pixel values ​​of the current video block being coded. The pixel values ​​of the predicted block are subtracted from the pixel values ​​of the predicted block to form a residual video block of pixel differences. The pixel difference values ​​forming the residual video block may be luma or chroma difference components, or Motion compensation unit 44 may also include motion compensation for video blocks of a video frame. to generate further syntax elements for decoding the video blocks of the video frame by the video decoder 30. The syntax element includes, for example, a motion vector to identify this predicted block. Syntax elements that define vectors, any flags that indicate prediction modes, or It should be noted that the motion estimation unit 42 and the motion compensation unit 44 are conceptually Although shown separately for illustrative purposes, they may be highly integrated.

[0052] In one embodiment, the intra BC unit 48 is configured to perform the motion estimation and compensation in accordance with the motion estimation and compensation unit 42 and 44. Then, a vector can be generated in the same manner as described above, and a prediction block can be obtained. where the predicted block is in the same frame as the current block being coded. The vectors are called block vectors, not motion vectors. In particular, the intra BC unit 48 determines one intra prediction mode to be used for the current block. In one example, the intra BC unit 48 may encode the individual paths, e.g. In the encoding process, various intra prediction modes are used to encode the current block, and the rate It is possible to test their performance by distortion analysis. The BC unit 48 selects one suitable intra prediction mode from the various intra prediction modes tested. is selected and used to generate the corresponding intra mode indicator. The BC unit 48 performs a rate-distortion analysis to calculate the rate-distortion values ​​of the various intra-prediction modes tested. Then, the intra prediction mode with the best rate-distortion performance is selected from the tested modes. In rate-distortion analysis, the coding and a coded block for generating the coded block, The amount of distortion (or error) between the original block and the unencoded block. The bit rate (i.e., number of bits) used to generate the block is determined. The intra BC unit 48 calculates a ratio based on the distortion and rate for each coded block. Calculate the rate and determine which intra prediction mode gives the best rate-distortion value for this block. The UE may also determine whether the UE is

[0053] In another example, the intra BC unit 48 may operate the motion estimation unit 42 and the motion compensation unit 44 globally. or in part, functions used for intra BC prediction according to embodiments described herein. In either case, for intra block copies, the predicted block The checks are calculated by sum of absolute differences (SAD), sum of squared differences (SSD), or other difference metrics. The pixel difference that can be determined by the and the identification of the predicted block may include calculating values ​​for sub-integer pixel positions. be.

[0054] Video encoder 20 may also encode the predicted blocks as blocks from the same frame according to intra prediction. whether they are from the same frame or from different frames according to inter prediction. The pixel values ​​of the predicted block are subtracted from the pixel values ​​of the current video block being coded. By forming the difference values, a residual video block can be generated. The pixel difference values ​​forming the block can include both luminance and chrominance component differences. do.

[0055] The intra-prediction processing unit 46 is implemented by the motion estimation unit 42 and the motion compensation unit 44 described above. Inter prediction is performed by the intra prediction unit 48, or intra block prediction is performed by the intra BC unit 49. Instead of cross-copy prediction, intra prediction can be performed for the current video block. In particular, the intra prediction processing unit 46 determines one intra prediction mode and performs the current block. For this purpose, the intra prediction processing unit 46 can encode individual blocks. During the encoding process of this pass, various intra prediction modes are used to code the current block. The intra-prediction processor 46 (or in some examples, the mode selector) An appropriate intra prediction mode may be selected and used from the intra prediction modes. The intra prediction processing unit 46 receives information indicating the intra prediction mode selected for this block. may be provided to the entropy coding unit 56. The entropy coding unit 56 may Information indicating the selected intra-prediction mode can be coded into the bitstream.

[0056] The prediction processor 41 performs inter-prediction or intra-prediction on the current video block. After determining the predicted block, adder 50 subtracts the predicted block from the current video block. The residual video data in the residual block is subtracted to form a residual video block. The transformation is provided to the transformation processing unit 52 in one or more transformation units (TUs). The transform processor 52 performs a discrete cosine transform (DCT) or a transform conceptually similar to the discrete cosine transform. The residual video data may be transformed into residual transform coefficients, such as by a transform using

[0057] The conversion processing unit 52 transmits the obtained conversion coefficients to the quantification unit 54. These transform coefficients are then quantified to further reduce the bit rate. It is also possible to reduce the bit depth associated with some or all of these coefficients. or can be changed by adjusting the quantification parameters. In an example, the quantification unit 54 may perform a scan of a matrix containing the quantified transform coefficients. Alternatively, this scanning can be performed by the entropy coding unit 56. do.

[0058] Following quantification, the entropy coding unit 56 may use, for example, context-adaptive variable length coding (CVLC). Context-Adaptive Binary Arithmetic Coding (CAVLC), Context-Adaptive Binary Arithmetic Coding (CABAC), Syntax-Based Context-adaptive binary arithmetic coding (SBAC), probability interval partition entropy ( quantified variations by PIPE coding or another entropy coding method or technique. The transform coefficients are entropy coded into a video bitstream. The bitstream may be transmitted to a video decoder 30, or thereafter to a video to a storage device 32 for transmission to the decoder 30 or retrieval by the video decoder 30. The entropy coding unit 56 may also archive the current Motion vectors and other syntax elements for video frames may be entropy coded. stomach.

[0059] The inverse quantification unit 58 and the inverse transformation processing unit 60 respectively perform inverse quantification and inverse transformation to obtain: Residuals in the pixel domain for generating reference blocks used to predict other video blocks As described above, the motion compensation unit 44 reconstructs the video blocks stored in the DPB 64. generating a motion compensated prediction block from one or more reference blocks of the frame being processed; The motion compensation unit 44 can also apply one or more interpolation filters to this prediction block. The data can also be applied to calculate sub-integer pixel values ​​used for motion estimation.

[0060] Adder 62 adds the reconstructed residual block to the motion compensation block generated by motion compensation unit 44. This is added to the compensated predicted block to generate a reference block to be stored in the DPB64. The reference block is a predicted block, and is processed by the intra BC unit 48, the motion estimation unit 42, and the motion estimation unit 43. is used by compensation unit 44 to interpolate another video block in a subsequent video frame. -It is possible to predict.

[0061] In a video encoding example using video encoder 20, a video frame is For each video block, either inter prediction or intra prediction is performed. In inter prediction, the predictor or predicted block is Based on pixels from a previously reconstructed frame, the motion estimation and compensation are used to generate the image. In intra prediction, the predictor is the reconstructed vector of the current frame. Depending on the mode decision, the current block can be predicted based on the pixel. The best predictor for the criterion can be selected.

[0062] The prediction residual or residual block (i.e., the difference between the current block and its predictor) is , are sent to a transformation module, for example, transformation processor 52. The transformation coefficients are then converted into a quantification module The quantified coefficients are then sent to a module, e.g., a quantification module 54, for entropy reduction. is supplied to an entropy coding / decoding module, for example, entropy coding unit 56. As shown in Figure 2, the block division information, motion vectors, reference image indices, and intra prediction modes. Prediction related information from the counter and / or intra prediction modules is also entropy coded. The encoded data is then passed through an encoding / decoding module, e.g., an entropy encoder 56, to produce a bitstream. It will be saved.

[0063] In the video encoder 20, a decoder is provided for pixel reconstruction for prediction purposes. The relevant modules may require: First, the prediction residuals are inversely quantified and inversely transformed. The reconstructed prediction residual is then combined with the predictor to generate the current block. An unfiltered reconstruction of the block is generated.

[0064] To improve coding / decoding efficiency and visual quality, in-loop filters are generally used. 63 is used. For example, the deblocking filter is used in AVC, HEVC and V In HEVC, in order to further improve the efficiency of coding and decoding, An additional in-loop filter called AO (Sample Adaptive Offset) can be defined. In VVC, an in-loop filter called ALF (Adaptive Loop Filter) is applied. It is possible.

[0065] These in-loop filter operations are selectable. Turning on the in-loop filter This usually improves coding / decoding efficiency and visual quality, and also reduces computational complexity. To avoid this, the in-loop filter can be turned off as an encoder decision. .

[0066] Note that intra prediction is usually based on unfiltered reconstruction pixels. However, if these filters are turned on by the encoder, intra prediction is performed It may be based on the pixels of the filtered reconstruction.

[0067] FIG. 3 is a block diagram illustrating a video decoder 30 according to an embodiment of the present disclosure. The video decoder 30 includes a video data memory 79, an entropy decoding unit 80, a prediction processing unit 81, and a video data memory 79. The image processing unit 80 includes a processing unit 81, an inverse quantification unit 86, an inverse transformation processing unit 88, an adder 90, and a DPB 92. The prediction processing unit 81 includes a motion compensation unit 82, an intra prediction processing unit 84, an intra BC unit 85, and The video decoder 30 further includes an intra / intermediate selection unit 87. The decoding process is roughly the reverse of the encoding process described above with respect to video encoder 20. For example, the motion compensation unit 82 can perform the process from the entropy decoding unit 80. The intra prediction unit 84 generates prediction data based on the received motion vector. The prediction data is generated based on the intra prediction mode indicator received from the PEEK decoder 80. It can be generated.

[0068] In an example of video decoding using video decoder 30, the received bitstream is , decoded through the entropy decoding unit 80 and quantified coefficient levels (or quantified The quantified coefficients and prediction-related information are derived. The quantified coefficient levels are then 8. The reconstructed residual block is processed through the inverse quantification unit 86 and the inverse transform unit 88. The predictor or predicted block is obtained based on the decoded prediction related information. It is formed by intra-prediction or motion compensation processes. The remaining pixels in the reconstruction are extracted by summing the reconstructed residual block with the predictor. If the in-loop filter is on, the Then, a filtering operation is performed, and finally the reconstructed video is derived and output. .

[0069] In one example, one component within video decoder 30 may perform an implementation of the present disclosure. Also, in some examples, implementations of the present disclosure may be implemented in one or more of the following ways: For example, the intra BC unit 85 may be divided into multiple components. It may be implemented alone, or may be implemented as a combination of a motion compensation unit 82, an intra-prediction processing unit 84, and an entropy processing unit. This is realized in combination with other components in the video decoder 30, such as the P / P decoding unit 80. In some examples, video decoder 30 may not include intra BC section 85, The function of the BC unit 85 is performed by other components in the prediction processing unit 81 such as the motion compensation unit 82. This may be realized as follows.

[0070] Video data memory 79 is used for decoding by other components in video decoder 30. It can store video data such as encoded video bitstreams. The video data stored in the video data memory 79 is transferred from the storage device 32 to the It can acquire video from a local video source such as a camera, or it can receive video data over a wired or or via wireless network communication, or on physical data storage media (e.g., flash memory). The video data can be obtained by accessing the video data memory 79. is a code that stores the encoded video data from the encoded video bitstream. The decoded picture buffer in the video decoder 30 may include a decoded picture buffer (CPB). The DPB 92 is a signal to the video decoder 30 (e.g., intra-prediction or inter-prediction). Stores reference video data used for decoding video data in the specified encoding mode. The video data memory 79 and the DPB 92 are implemented using synchronous DRAM (SDRAM), magnetoresistive R Dynamic random access memory (MRAM), including resistive random access memory (RRAM) of various memory devices, such as memory (DRAM), or other types of memory devices. For convenience of explanation, the video data memory 79 and The DPB 92 and the DPB 93 are shown in FIG. 3 as two distinct components within the video decoder 30. However, those skilled in the art will understand that the video data memory 79 and the DPB 92 are the same memory. It is clear that this can be provided by a separate memory device or a separate memory device. , video data memory 79, along with other components in video decoder 30, The components may be chip-based or off-chip to those components.

[0071] During the decoding process, video decoder 30 decodes the video data of the encoded video frames. An encoded video bitstream indicating ob-blocks and associated syntax elements is received. Video decoder 30 may be configured at the video frame level and / or at the video block level. The entropy decoding unit 80 of the video decoder 30 can receive sentence elements. The bitstream is entropy decoded to obtain quantized coefficients, motion vectors, or Generates intra-prediction mode indicators and other syntax elements. The decoding unit 80 transfers the motion vectors and other syntax elements to a prediction processing unit 81 .

[0072] If the video frame is coded as an intra-predictive (I) frame, or other When used for intra-coded predicted blocks of a frame of this type, the prediction processing unit 81 The intra prediction processing unit 84 in and reference data from previously decoded blocks from the current video frame The prediction data for the video block may be generated.

[0073] Video frames are coded into inter-predictive (i.e., B or P) frames. In this case, the motion compensation unit 82 in the prediction processing unit 81 receives the entropy from the entropy decoding unit 80. The video blocks of the current video frame are extracted based on the motion vectors and other syntax elements. One or more prediction blocks of a reference frame can be generated. The video decoder 30 generates the frame from a reference frame in one of the frame lists. Based on the reference frame stored in DPB92, the default construction technique Reference frame lists, eg, List0 and List1, can be constructed.

[0074] In one example, the video block is coded according to the intra BC mode described herein. In this case, the intra BC unit 85 in the prediction processing unit 81 receives the data from the entropy decoding unit 80. Prediction of the current video block based on the block vector and other syntax elements received from The predicted block is defined by video encoder 20. It may be within the same reconstruction region of the image as the current video block.

[0075] The motion compensation unit 82 and / or the intra BC unit 85 process the motion vectors and other syntax elements. determining prediction information for a video block of the current video frame by analyzing the matrix; and the predicted block for the current video block being decoded using this prediction information. For example, the motion compensation unit 82 generates this video using some of the received syntax elements. A prediction mode (e.g., intra-prediction or is inter-predicted), the inter-predicted frame type (e.g., B or P), this frame Structural information for one or more reference frame lists, each instance of this frame the motion vectors for the inter-predictively coded video blocks of this frame, Inter prediction state for the decoded video block and the video block in the current video frame. Determine other information for decoding the video block.

[0076] Similarly, the intra BC unit 85 uses some of the received syntax elements, for example, a flag. It indicates that the current video block is predicted in intra BC mode, and that The configuration regarding which video blocks are in the reconstruction domain and should be stored in DPB 92. block vectors for each intra BC predicted video block in this frame. the intra BC predicted video block for each intra BC predicted video block in this frame; measurement state and other information for decoding video blocks in the current video frame can be determined.

[0077] Furthermore, motion compensation unit 82 is used to compensate for the motion of the video block encoded by video encoder 20. The interpolation is performed using the interpolation filter used in the previous step to interpolate the sub-integer pixels of the reference block. In this case, the motion compensation unit 82 calculates an interpolated value for the received syntax element and determining the interpolation filter used by the video encoder 20 from may be used to generate the prediction block.

[0078] Inverse quantification unit 86 calculates the number of video blocks in this video frame by video encoder 20. The same quantification parameters calculated to determine the degree of quantification for the lock , which are provided in the bitstream and are decoded by the entropy decoder 80. The inverse transform processor 88 inversely quantifies the quantified transform coefficients in the pixel domain. To reconstruct the residual block in the A conceptually similar inverse transform process is applied to the transform coefficients.

[0079] The motion compensation unit 82 or the intra BC unit 85 performs the motion compensation based on this vector and other syntax elements. After generating the prediction block for the current video block using the adder 90, the adder 90 outputs the prediction block to the inverse transform processor 88. and the corresponding residual blocks generated by the motion compensation unit 82 and the intra BC unit 85. The decoded video block is calculated for the current video block by adding the predicted block to the The in-loop filter 91 is disposed between the adder 90 and the DPB 92. The decoded video block can then be further processed. A decoded video block in a given frame is then decoded based on the motion of the next video block. The reference frame for compensation is stored in the DPB 92. The decoded video is then stored in a memory device separate from the DPB92 or the DPB92. The image data can be stored for display on a display device such as display device 34.

[0080] In a typical video coding process, a video sequence is typically split into ordered frames. Each frame contains three subframes: SL, SCb, and SCr. SL is a 2D matrix of luma samples. SCb is a 2D matrix of Cb samples. SCr is a two-dimensional matrix of chroma samples. SCr is a two-dimensional matrix of Cr chroma samples. In this example, the frame may be monochrome, in which case a two-dimensional image of one luminance sample is Only matrices are included.

[0081] FIG. 4 illustrates a quadtree plus binary tree (QTBI) according to an embodiment of the present disclosure. s binary tree) structure.

[0082] The above mentioned videos such as VVC, JEM, HEVC, MPEG-4, Part10, etc. The encoding / decoding standards are conceptually similar. For example, they are all block-based. The block division schemes in some standards are detailed below. Explain in detail.

[0083] HEVC is based on a hybrid block-based motion-compensated transform coding scheme. The basic unit of compression is called a coding tree unit (CTU). For mattes, the maximum CTU size is 64x64 luminance pixels, and two 32x32 Each CTU contains one coding unit (CU), and Or recursively down to four smaller CUs until a predetermined minimum CU size is reached. Each CU (also called leaf CU) can have one or more predictions. It contains a unit (PU) and a tree of one or more transform units (TUs).

[0084] In general, except for monochrome content, a CTU consists of one luminance coding tree block (CT B) and two corresponding chroma CTBs, and the CU contains one luma coding block (CB) and two corresponding chroma CBs, and PU contains one luma prediction block (PB) and The TU contains one luma transform block (TB) and two corresponding chroma transform blocks (PB). It may contain a corresponding chroma TB, but the minimum TB size is 1 for both luma and chroma. 4x4 (i.e., 2x2 chroma TB is supported in 4:2:0 color format) (not included) and regardless of the number of intra-luminance PBs in the corresponding intra-luminance CB. There is always only one intra saturation CB, and only one intra saturation PB.

[0085] In intra CU, luminance CB can be predicted by one or four luminance PBs, and two chrominance PBs. Each luminance CB is always predicted by one chroma PB, where each luminance PB has one There is an intra-luminance prediction mode, and two chroma PBs share one intra-chroma prediction mode. Furthermore, in an intra-CU, the TB size cannot be larger than the PB size. In each PB, intra prediction is applied to derive the PB from samples of neighboring TBs. For each PB, 33 directional intra prediction modes are used. In addition to the modes, DC and planar modes are also supported, resulting in flat areas and gradually Predict the area that changes.

[0086] For each inter PU, three prediction modes such as inter, skip and merge Generally speaking, the motion vector competition (MVC) method is A method is introduced to select motion candidates from a predetermined candidate set including spatial and temporal motion candidates. Multiple references for motion estimation result in two possible reconstructed reference image lists ( That is, you can find the best references in List0 and List1. -mode (called AMVP mode, where AMVP stands for Advanced Motion Vector Prediction) In this case, the inter prediction indicator (List0, List1, or bi-prediction) The reference index, motion candidate index, motion vector difference (MVD) and prediction residual are transmitted. For skip and merge modes, only the merge index is sent. The current PU is then assigned a merge index from the neighboring PU referenced by the encoded merge index. , the inter prediction indicator, the reference index and the motion vector. For pre-coded CUs, the residual signal is also omitted.

[0087] The Joint Exploration Test Model (JEM) is built on the HEVC test model. The basic encoding and decoding flow of C has not changed in JEM. However, Structure, intra and inter prediction, residue transform, loop filter, entropy coding The design elements of the most important modules, including the module, have been slightly changed and additional coding tools have been added. JEM includes the following new coding features:

[0088] In HEVC, CTUs are divided into CUs using a quadtree structure, referred to as a coding tree. It is based on a set of local features and adapts to various local characteristics. It uses either inter-picture (temporal) or intra-picture (spatial) prediction. The decision to use to encode an image region is made at the CU level. Each CU has a PU partition type. Depending on the group, it can be further divided into one, two, or four PUs. Within U, the same prediction process is applied and the relevant information is sent to the decoder on a PU basis. After obtaining the residual block by applying prediction processing based on the U-division type, the code of the CU is This CU can be divided into transformation units (TUs) according to another quadtree structure similar to the transformation tree. One of the key features of the HEVC structure is the concept of multiple partitions, including CUs, PUs, and TUs. There is.

[0089] The QTBT structure removes the concept of multiple partition types, i.e., CU, PU, ​​and T Removes the separation of the U concept and supports more flexibility in CU partition shapes. In the block structure, the CU can be square or rectangular in shape. As shown in Figure 4, The coding tree units (CTUs) are first divided into quadtrees. The leaf nodes can be further divided by a binary tree structure. There are two types of partitioning: symmetric horizontal partitioning and symmetric vertical partitioning. The leaf nodes of a binary tree are The segmentation is called a coding unit (CU), and the , used for prediction and transform processing. This is because CU, PU, ​​and TU are QTBT coded. This means that the CU has the same block size in the block structure. It may consist of coded blocks (CB) of different color components, e.g., 4:2:0 chroma For P and B slices in the format, one CU has one luminance CB and two In JEM, CU may consist of a single component CB, For example, for an I slice, one CU contains only one luminance CB, or , only two saturations CB are included.

[0090] For the QTBT partition method, the following parameters are defined: -CTU size: The root node size of the quadtree, which is the same concept as HEVC; -MinQTSize: Minimum allowed leaf node size in a quadtree; -MaxBTSize: Maximum allowed root node size of a binary tree; -MaxBTDepth: Maximum allowed depth of binary tree; -MinBTSize: The minimum allowed leaf node size in a binary tree.

[0091] In one example of a QTBT partition structure, the CTU size is two corresponding 64x64 blocks of color. 128x128 luma samples with chroma samples (4:2:0 chroma format), MinQTS Size is 16x16, MaxBTSize is 64x64, MinBTSize (both width and height) is 4x4, M axBTDepth is set to 4. The quadtree partition is first applied to the CTU, and the quadtree Leaf nodes are generated. The leaf nodes of the quadtree are 16x16 (i.e. MinQTSize) It can have sizes from 128x128 (i.e., CTU size). If the leaf nodes of the tree are 128x128, then the size is MaxBTSize (i.e. 64x64 ), so it cannot be further split by the binary tree. Leaf nodes can be further split by binary trees. The fnode is also the root node of the binary tree, and is at depth 0 of the binary tree. When MaxBTDepth (i.e., 4) is reached, no further splits are considered. Once a BTSize has width equal to MinBTSize (i.e., 4), no further horizontal divisions are considered. Similarly, once a binary tree node has a height equal to MinBTSize, no further vertical splits are considered. The leaf nodes of the binary tree are used for prediction and transformation without further splitting. In JEM, the maximum CTU size is 256x256 luminance samples. It is.

[0092] Figure 4 shows an example of a block partition and the corresponding tree representation according to the QTBT method. The solid lines indicate quadtree division, and the dotted lines indicate binary tree division. The CTU 400 is first divided into four quadtrees. Three of the nodes 402, 404, 406, and 408 are quadtree or binary tree structures. For example, a leaf node 406 of the quadtree is further divided by the quadtree division. The leaf node 404 of the quadtree is further split into two by binary tree splitting. The leaf node 402 of the quadtree is also divided into leaf nodes 404a and 404b. , which is further divided by binary tree splitting. At each split (i.e., non-leaf) node in the binary tree, ,one flag is signaled indicating the split type used (i.e., horizontal or vertical) , where 0 indicates a horizontal split and 1 indicates a vertical split. For example, leaf node 4 of a quadtree In the case of 04, 0 is signaled to indicate a horizontal split, and the leaf node 402 of the quadtree In this case, 1 is signaled to indicate a vertical split. A quadtree split always splits a block horizontally. The division time is used to divide both horizontally and vertically to generate four sub-blocks of equal size. There is no need to specify the type.

[0093] The QTBT method also supports the ability to have separate QTBT structures for luma and chroma. Currently, for P slices and B slices, the luminance CTB and chroma CTU are The CTBs share the same QTBT structure, except for the I slice, where the luminance CTB is 1 The saturation CTB is divided into CUs by one QTBT structure, and the saturation CTB is divided into CUs by another QTBT structure. This means that a CU in an I slice is divided into two coding blocks of the luminance component. The CU in a P slice or B slice consists of three color coding blocks. This means that it consists of coded blocks of all components.

[0094] At a Joint Video Experts Team (JVET) meeting, the JVET decided to develop a versatile video codec. Defines the first draft of the VVC coding standard and the VVC Test Model 1 (VTM1) coding method. It has nested multi-type trees with binary and ternary partition coding block structure. It was decided that the quadtree would be included as the first new coding feature in VVC.

[0095] In VVC, the input video is divided into coding tree units (CTUs) according to the image partition structure. The CTU is a quadtree with a nested multi-type tree structure. where a leaf coding unit (CU) is It defines regions that share the same prediction mode (e.g., intra or inter). In,the term "unit" defines the region of the image that covers all,components. The term "block" is used to define an area covering a particular component (e.g., luminance). If a chroma sampling format such as 4:2:0 is considered, The spatial location may vary. Partitioning of the image into CTU

[0096] FIG. 5 is a schematic diagram illustrating an example of an image segmented into CTUs, according to an embodiment of the present disclosure. do.

[0097] In VVC, an image is divided into a set of CTUs, where the concept of a CTU is different from that of HEVC. For an image with three sample arrays, the CTU is an NxN luminance sample. Figure 5 shows the CTU50 sample block and two corresponding chroma sample blocks. An example of an image 500 divided into two is shown.

[0098] The maximum allowed size of a luminance block in a CTU is specified as 128x128 ( However, the maximum size of a luminance transformation block is 64x64. CTU partitioning using a tree structure

[0099] FIG. 6 is a schematic diagram illustrating a multi-type tree splitting mode according to an embodiment of the present disclosure. be.

[0100] In HEVC, the CTU is represented as a coding tree to adapt to various local characteristics. The image is divided into CUs by a quadtree structure. Inter-picture (temporal) prediction or intra-picture (spatial) prediction The prediction or coding of an image region is decided at the leaf CU level. A CU can be further divided into 1, 2, or 4 PUs depending on the PU division type. Within one PU, the same prediction process is applied, and the relevant information is sent to the decoder on a PU-by-PU basis. After applying the prediction process based on the PU partition type to obtain the residual block, the leaf A CU is divided into transform units (TUs) according to another quadtree structure similar to the coding tree of this CU. One of the key features of the HEVC structure is that it can be divided into multiple parts, including CUs, PUs, and TUs. There is a concept of division.

[0101] VVC supports multiple types of nesting with binary and ternary segmentation structures. The quadtree with tree replaces the concept of multiple partition unit types, i.e., CU, Removed the separation of PU and TU concepts (in the case of CU where the maximum transformation length is too large) In the coding tree structure, CUs can be square or A coding tree unit (CTU) is a set of four blocks, each of which can be 4-byte or 8-byte. The leaf nodes of this quadtree are then split into a multitype tree structure. As shown in Figure 6, the multi-type tree structure , vertical binary split 602 (SPLIT_BT_VER), horizontal binary split 604 (SPLIT_BT_HOR), vertical Three-way split 606 (SPLIT_TT_VER), and horizontal three-way split 608 (SPLIT_TT_HOR) There are split types. The leaf nodes of a multitype tree are called coding units (CUs). This segmentation will not be further segmented unless the maximum transform length of the CU is too large. This is mostly used for prediction and transformation without any input. TU is a quadtree with a coding block structure of nested multitype trees with the same block The exception is that the maximum transform support length is the width of the color components of the CU. In VTM1, if the video is monochrome, i.e. CUs are composed of coding blocks (CBs) of different color components, except when there is only one color component per CU. For example, one CU includes one luminance CB and two chroma CBs. Partitioning a CU into multiple prediction units

[0102] In VVC, for each CU divided based on the above structure, block content The prediction can be performed for the entire CU block or in a sub-block manner as described below. Such a unit of operation for prediction is called a prediction unit (or PU).

[0103] In the case of intra prediction (or intra-frame prediction), the size of the PU is usually CU In other words, prediction is performed on the entire CU block. In the case of prediction (or interframe prediction), the size of the PU must be less than or equal to the size of the CU. In other words, a CU may be divided into multiple PUs for prediction purposes.

[0104] Examples where the PU size is smaller than the CU size include affine prediction mode, advanced temporal level This includes motion vector prediction (ATMVP) mode, and triangular prediction mode.

[0105] In affine prediction mode, a CU can be divided into multiple 4x4 PUs for prediction. A motion vector can be derived for each 4x4 PU, and then the motion vector is calculated for each 4x4 PU. In ATMVP mode, one or more 8-bit CUs can be used for prediction. It is possible to divide the image into 8×8 PUs. A motion vector is derived for each 8×8 PU, and the corresponding In triangular prediction mode, the CU is divided into 2 It is possible to divide the image into two triangular prediction units. A motion vector is derived for each PU. and perform motion compensation accordingly. Triangular prediction modes are supported for inter prediction. The details of the triangular prediction modes are shown below: Triangle Prediction Mode

[0106] FIG. 7 illustrates dividing a CU into triangle prediction units according to an embodiment of the present disclosure. FIG.

[0107] The concept of triangular prediction mode introduces triangular partitions for motion compensated prediction. As shown in Figure 7, the CUs 702 and 704 are arranged diagonally or anti-diagonally. The triangle is divided into triangle prediction units PU1 and PU2 (i.e., from the top left corner to the bottom right corner). (whether to split from top right corner to bottom left corner or split from top right corner to bottom left corner). The frame is then assigned its own single-prediction motion vector and reference frame derived from the single-prediction candidate list. These triangle prediction units are predicted using the inter-prediction unit index. After that, adaptive weighting is performed on the diagonal edges. This mode is applied to the entire CU. Note that this mode is not supported for the current VVC skip mode and Applies only to the CU and Merge modes. As shown in Figure 7, CUs are represented as square blocks. However, the triangular prediction mode can also be applied to CUs with non-square (i.e., rectangular) shapes. do.

[0108] The single prediction candidate list includes one or more candidates, each of which is a motion vector. Therefore, throughout this disclosure, the terms "single prediction candidate list," "single prediction The terms "motion vector candidate list" and "unipredictive merge list" are used interchangeably. It can be used. Single predictor motion vector candidate list

[0109] FIG. 8 is a schematic diagram illustrating the position of adjacent blocks according to an embodiment of the present disclosure. do.

[0110] In one example, the single-predictor motion vector candidate list includes two to five single-predictor motion vectors. In some cases, other numbers are possible. The single predictor motion vector candidate list is derived from the block as shown in Figure 8. As shown, five spatially adjacent blocks (1 to 5) and two temporally co-located blocks were The 7 adjacent blocks, including the selected block (6 to 7), are used. The motion vectors of two adjacent blocks are first collected in a merge list. According to the order of the merged list, a single prediction candidate list is formed based on the motion vectors of the first merged list. Based on that order, the single predicted motion vector from the first merge list is , the bi-predictive motion vector candidate list is entered, and then the reference image Li st0 or L0 motion vector, and reference image List1 or Followed by the L1 motion vector, and so next the L0 and L1 motion vectors of the bi-predictive motion vectors At that point, the number of candidates is still the target number (current In VVC, if there are fewer than 5), zero motion vectors are listed to meet the target number. will be added.

[0111] For each triangle PU, a predictor is derived based on its motion vector. Note that the derived predictor covers a larger area than the actual triangle PU, so the two There is an overlapping region of the two predictors along the shared diagonal edge of the triangle PU. To derive the final prediction, an adaptive weighting process is performed on the diagonal edge between the two predictors. Currently, two sets of weighting factors are used: - The first set of weighting factors: {7 / 8, 6 / 8, 4 / 8, 2 / 8, 1 / 8} and {7 / 8, 4 / 8, 1 / 8} for the luminance used for the sample and saturation samples, respectively; - Second set of weighting factors: {7 / 8, 6 / 8, 5 / 8, 4 / 8, 3 / 8, 2 / 8, 1 / 8} and {6 / 8, 4 / 8, 2 / 8 } is used for luma and chroma samples, respectively.

[0112] The weighting factor set is based on the comparison of the motion vectors of these two triangular prediction units. More specifically, the reference images of these two triangular prediction units are selected based on the If the difference between the motion vectors of these two triangular prediction units is greater than 16 pixels, If it is greater than the first set of weighting factors, the second set of weighting factors is used; otherwise, the first set of weighting factors is used. FIG. 9 illustrates a method for performing a weighting coefficient set according to an embodiment of the present disclosure. 9 shows an example of weighting. In FIG. 9, the first weighting factor is applied to the luma samples 902 and the chroma samples 903. Used for sample 904. Triangle prediction mode syntax and signaling

[0113] In the current VVC, the triangular prediction mode is signaled by the triangular prediction flag. This flag signals if the CU is coded in skip or merge mode. For a CU, if the flag value is 1, the corresponding CU is notified as a triangular prediction model. Otherwise, the CU is coded using a non-triangle prediction mode. The image was coded using the prediction mode.

[0114] According to the current VVC standard draft, the triangle prediction flag is used to indicate whether skip mode or marking is enabled. First, the enable / disable flag for the triangle predictor. This flag is signaled in the Sequence Parameter Set (or SPS). If and only if true, the triangle prediction flag is signaled at the CU level. Triangle prediction tools are only allowed on B slices. Therefore, on B slices only, The triangular prediction flag is signaled at the CU level. Third, the triangular prediction mode is Only CUs with a size equal to or greater than this threshold are signaled. If the value is less than the threshold, the triangle prediction flag is not signaled. Encoding with sub-block merging mode including both Finn prediction mode and ATMVP mode The triangle prediction flag is signaled only if the above four In this case, if the triangle prediction flag is not signaled, the decoder assumes it is 0. In some cases, the triangle prediction flag is explicitly signaled in the bitstream. Instead, all other merge-related modes are signaled as invalid. If so, triangular prediction mode is presumed to be enabled.

[0115] According to the current VVC standard draft, if the triangle prediction flag is signaled, The context is signaled by the CABAC entropy coder at the start of the test. Context-guided method for encoding and decoding triangle prediction flags according to some embodiments of the disclosure As shown in FIG. 10, the shape is determined based on the triangle prediction flag values ​​from the neighboring blocks. It is done.

[0116] As shown in FIG. 10, the triangle prediction flag for the current block 1000 (or the current CU) is In order to encode or decode (i.e., encode or decode) the data, the upper block 10 Triangles from both block 04 and left block 1002 (or upper CU and left CU) The predictive flags are derived and their values ​​are summed. This results in three possible values ​​corresponding to the following cases: possible contexts are generated. 1) Both the left block and the top block have triangle prediction flags of 0; 2) Both the left block and the top block have triangle prediction flags of 1; 3) In any other case.

[0117] Separate probabilities are maintained for each of the three contexts. Once the context value is determined, the triangle prediction flags of the current block are set to that context. The data is encoded and decoded using the CABAC probability model corresponding to the base value.

[0118] If the triangle prediction flag is true, the table index associated with the triangle prediction mode is This index value is used to derive the following information: It can be used. 1) The direction of the triangular prediction block, i.e., whether this block is divided from the upper left corner to the lower right corner. , whether it is divided from the top right corner to the bottom left corner; 2) The motion vectors in the single-prediction motion vector candidate lists of each of these two partitions Vector.

[0119] More specifically, this is done by, for example, the triangle prediction merge list index as shown in Table 1. This is achieved by table lookup operations using the VVC report table. In M, this table is an array of size 40x3 defined as follows: [Table 1]

[0120] This 2D array has 40 rows and 3 columns. The value in the first column indicates the direction of the section, with 0 being The values ​​in the second and third columns indicate the direction of the two compartments, and 1 indicates the direction of the other compartment. For each of the two triangular partitions, a single prediction It is an index value to the corresponding single predictor motion vector in the motion vector candidate list. As shown previously, according to the current VVC standard draft, the single predictor motion vector candidate list The list contains two to five single-prediction motion vectors. The index value is The sigma-based motion vectors indicate which motion vectors in the list are used for the corresponding triangle partition.

[0121] In triangular prediction mode, the range of table index values ​​signaled is 0 to 39. The box value, once decrypted, looks up a row of data containing three numbers according to the table above. These three numbers determine the direction of the triangle prediction block and the triangle The merge list of single predictions for each of the two motion vectors in the shape partition is Represents the list index value.

[0122] The current VVC standard draft provides additional features to extend and / or simplify triangle prediction modes. Therefore, the concept of a triangular PU represents an example of a PU partition according to an embodiment of the present disclosure. It is extended to other partitions of the prediction unit as shown in Figure 10. The method or concept used in the method is to use other partitions in addition to the two diagonal triangular PU partitions. Basically, the given partition shown in Figure 10 can be applied to the PU partition. For each partition (e.g., PU0 and PU1), two single partitions are created for each image type. A one-way predictor is derived and adaptive weighting is applied around the boundary region of the two partitions. For example, the method used for triangular PUs can be applied to horizontal and / or vertical PUs. Can be used. Motion vector candidate list for normal merge mode

[0123] According to the current VVC, normal merging is predicted without splitting the entire CU into multiple PUs. In mode, the motion vector candidate list or merge candidate list is It is created using a different procedure than when

[0124] First, FIG. 12 is a schematic diagram illustrating the locations of spatial merge candidates according to an embodiment of the present disclosure. As shown in Fig. 1, the spatial motion vector is calculated based on the motion vectors from the neighboring blocks. In the derivation of spatial merge candidates, the candidates are arranged in the positions shown in Figure 12. From the candidates, select up to four merge candidates. The order of derivation is A1 → B1 → B0 → A 0 → (B2). Position B2 is the position where PUs at positions A1, B1, B0, and A0 are not available or are unavailable. This is only taken into account when the data is encoded and decoded.

[0125] Next, temporal merge candidates are derived. In the derivation of the temporal merge candidates, a predetermined reference Belongs to the image with the smallest Picture Order Count (POC) difference from the current image in the image list The motion vector to be scaled is derived based on the PU located at the same position. The reference image list used to derive the PU placed at the slice position is explicitly specified in the slice header. The scaling motion vectors for the temporal merge candidates are signaled to the 13 showing motion vector scaling of temporal merge candidates according to one embodiment of the present invention. The scaling motion vectors for the temporal merge candidates are obtained as shown by the lines. is calculated from the motion vector of the co-located PUcol_PU using the POC distances tb and td. where tb is the reference image curr_ref and the current image curr_pic td is defined as the POC difference between the reference image col_ref and the co-located image It is defined as the difference in POC between this co-located image col_pic. The reference image index of the image candidate is set to 0. A practical implementation of the scaling process is It is stated in the HEVC draft specification. For B slices, one motion vector is referenced. One motion vector is for the reference image List0, and the other is for the reference image List1. Take two motion vectors such as and combine them to create a bi-predictive merge candidate. do.

[0126] FIG. 14 is a schematic diagram illustrating candidate locations for temporal merge candidates, according to an embodiment of the present disclosure. be.

[0127] The position of the PU placed at the same position can be chosen from two candidate positions C3 and H as shown in Figure 14. The PU at position H is selected from the following: If it is outside the current CTU, then location C3 is used to derive temporal merge candidates. Otherwise, position H is used to derive temporal merge candidates.

[0128] As mentioned above, both spatial and temporal motion vectors are merged into a candidate list. After inserting, add a history-based merge candidate. are kept in a separate motion vector list and managed according to specific rules. It contains motion vectors from decoded CUs.

[0129] After inserting the history-based candidates, if the merge candidate list is not full, As the name suggests, we add more pairwise average motion vector candidates to this The candidates for the group are constructed by averaging the candidates already in the current list. Each time, two candidates are taken from the merge candidate list based on a specific order. The average motion vector of the two candidates is added to the current list.

[0130] According to the current VVC, the next motion vector pairing order is It is determined which two candidates to take to derive the average motion vector candidate. JPEG2026041750000003.jpg8127

[0131] The two values ​​in each parenthetical pair represent the inverse of the two motion vector candidates in the merge candidate list. Therefore, the first average motion vector candidate is the first in the merge candidate list. It is generated by averaging motion vector candidate 0 and motion vector candidate 1. The second average motion vector candidate is the motion vector candidate 0 and the motion vector candidate 1 in the merge candidate list. It is generated by averaging candidate 2, and so on.

[0132] After inserting the pairwise average motion vectors, if the merge candidate list is not yet full, If so, add zero motion vectors until the list is full. Merge motion vector difference (MMVD) / ultimate motion vector representation (UMVE) Do

[0133] The Ultimate Motion Vector Representation (UMVE) is adopted in VVC and is the reference software V UMVE was later renamed Merge Mode with MVD (MMVD). The proposed motion vector representation method allows for skip mode or is used for merge mode.

[0134] MMVD reuses merge candidates in the same way as VVC. This is further enhanced by the proposed motion vector representation method.

[0135] MMVD provides a new motion vector representation with simplified signaling. This representation includes the starting point, the degree of movement, and the direction of movement.

[0136] The proposed technique uses the merge candidate list as is. However, the default Only candidates for merge type (MRG_TYPE_DEFAULT_N) are used to expand the MMVD.

[0137] The base candidate index (IDX) specifies the starting point. The base candidate index is The best candidate among the candidates in the table is shown. [Table 2]

[0138] If the number of base candidates is equal to 1, no base candidate IDX is signaled.

[0139] The distance index is the information of the degree of movement. Indicates the predetermined distance. The predetermined distance is as follows: [Table 3]

[0140] The direction index represents the direction of the MVD relative to the starting point. Four directions can be represented as shown in the table below. [Table 4]

[0141] The MMVD flag is signaled immediately after the skip and merge flags are sent. The MMVD flag is parsed if the skip and merge flags are true. If the MMVD flag is equal to 1, the MMVD syntax is parsed, otherwise the AFFINE The flags are parsed. The AFFINE flag, if 1, indicates AFFINE mode, otherwise , the skip / merge index is parsed for the VTM's skip / merge mode. Combined Inter and Intra Prediction (CIIP)

[0142] In VTM, when a CU is encoded and decoded in merge mode, this CU must have at least 64 contains 1000 luma samples (i.e., the product of CU width and CU height is 64 or greater) and an additional flag is signaled to signal Combined Inter / Intra Prediction (CIIP) Indicates that the mode applies to the current CU.

[0143] To form the CIIP, we first derive the intra prediction modes from two additional syntax elements: Up to four possible intra prediction modes are available, such as DC, planar, horizontal, or vertical. Then, the normal intra and inter decoding processes are used to perform inter prediction and Finally, we derive the inter- and intra-prediction signals to obtain the CIIP prediction. and perform weighted averaging of the intra-prediction signal.

[0144] The intra prediction mode derivation can be DC, PLANAR, HORIZONTAL, or Up to four modes, including VERTICAL, can be used to control the luminance component in CIIP mode. If the CU shape is very wide (i.e., its width is more than twice its height), If the CU shape is very narrow (i.e., its height is Vertical mode is not allowed in these cases (where the length is greater than twice the width). Only intra prediction modes are allowed.

[0145] The CIIP mode uses three most probable modes (MPM) for intra prediction. The CIIP MPM candidate list is formed as follows:

[0146] Let the left and upper neighboring blocks be A and B, respectively.

[0147] The intramodes for Block A and Block B are denoted as intraModeA and intraModeB, respectively. The tidal prediction mode is derived as follows. Let iX be A or B; ii. 1) Block X is not available; or 2) Block X is in CIIP mode or 3) If block B is outside the current CTU, then the intr aModeX is set to DC; iii. Otherwise, intraModeX is set to 1) if block X's intra prediction mode is DC or 2) if block X is a plane, it is set to DC or plane; or 2) if block X is an intra prediction mode, it is set to DC or plane. If the mode is in "vertical-like" angle mode (greater than 34), it will be set to vertical, or or 3) the intra prediction mode of block X is a "horizontal-like" angle mode (34 or less) If , it is set horizontally;

[0148] If intraModeA and intraModeB are the same: i.If intraModeA is plane or DC, the three MPMs are in order {plane, DC, vertical} is set to; ii. Otherwise, the three MPMs are set to {intraModeA, Planar, DC} in order. be;

[0149] Otherwise (different from intraModeA and intraModeB): i. The first two MPMs are set to {intraModeA, intraModeB} in order; ii. Planar, DC, and vertical uniqueness for the first two MPM candidate modes in this order If a unique mode is discovered, it is added as a third MPM.

[0150] If the CU shape is too wide or too narrow as defined above, the MPM flag is signaled. otherwise, the MPM flag is signaled. , whether the CIIP intra prediction mode is one of the CIIP MPM candidate modes. Indicates whether

[0151] If the MPM flag is 1, the MPM index is further signaled to identify the MPM candidate. Indicates which of the modes is used in CIIP intra prediction; otherwise, If the MPM flag is 0, the intra prediction mode is a "missing" mode in the MPM candidate list. For example, if the planar mode is not in the MPM candidate list, the planar mode is set to Missing. Since this is a CIIP mode, the intra prediction mode is set to Planar. Various intra prediction modes are allowed, and the MPM candidate list contains three intra prediction modes. Since the 4 possible modes are all contained within the 4 possible modes, one of them must be the missing mode. There is a need.

[0152] For the chroma component, the DM mode is always applied without any additional signaling, i.e. In this example, the chroma intra prediction mode is the same as the luma prediction mode. The intra prediction mode for the DM model is directly reused. It is called a code.

[0153] The intra prediction mode of a CIIP-coded CU is preserved, and the intra prediction mode of future neighboring CUs is preserved. Used for lamode encoding and decoding.

[0154] To combine inter-predicted and intra-predicted signals, the normal merge mode is used. The inter prediction signal in CIIP mode is generated by the same process as the applied inter prediction process. The signal P_inter is derived by the CIIP intra prediction mode according to the normal intra prediction process. Then, the intra-predicted signal P_intra is calculated by weighted averaging as follows: The intra prediction signal is combined with the inter prediction signal, where the weight value depends on the intra prediction mode. The samples are in a coding block.

[0155] If the intra prediction mode is DC mode or planar mode, or the block width or If the height is less than 4, the intra-predicted signal and the inter-predicted signal are given equal weights. Apply.

[0156] Otherwise, the weights depend on the intra prediction mode (horizontal or vertical mode) and the block size. The prediction is determined based on the sample position in the block. (The derivation of the weights for the linear modes is similar, but for the orthogonal directions), W is the width of the block, H Let W be the height of the block. First, the coding blocks are each of size (W / 4) × H. The part closest to the intra prediction reference sample is divided into four equal area parts with different sizes. The four regions or areas start with the region furthest from the intra-prediction reference sample and end with the region furthest from the intra-prediction reference sample. The weights wt of each region are set to 6, 5, 3 and 2 respectively. The final CIIP prediction signal is derived. JPEG2026041750000007.jpg1188

[0157] As explained above, the triangle prediction merge list index is a 40x3 size prediction. The first column of this table indicates the direction of the triangle section. The second and third columns show the individual triangle prediction units for each of the two triangle prediction units of the CU. This table shows the index values ​​of the motion vector candidates in one motion vector predictor candidate list. , need to be stored on both the encoder side and the decoder side. ,It has been proposed that the size of this table can be reduced. Separate signals for triangle partition orientation and merge list index

[0158] Defined in VVC for signaling triangle prediction merge list index The current Table 1 can be derived by data training. In this table, the directions and merges of the triangle partitions are List indexes are grouped for signaling purposes. In theory, The direction of this triangular section is always more likely to be used than the direction of the other triangular section. Therefore, based on the present disclosure, the notification of the direction of the triangular section by the signal is not considered to be a may be separated from the signaling of the merge list index.

[0159] More specifically, according to the present disclosure, in triangular prediction mode, for example, a partition direction flag or A separate flag, called the triangle section direction flag, determines which of the two triangle section directions is used. A uni-predictive merge list or uni-predictive motion vectors can be signaled to indicate whether they are used. Two merge list indices indicating the selected entries in the vector candidate list The values ​​are for one triangle partition and are table index values are signaled together as

[0160] For example, the following 20x2 size table 5 shows the merge of these two triangle prediction units. It can be used to signal list index values. In this case, both merge lists Indicates which index values ​​are used for each of these two triangle partitions. To do this, a table index value in the range 0 to 19 can be signaled. Table 2. Simplified triangle prediction merge list index signaling g_triangleCombination

[20] [2] = { { 0, 1}, { 1, 0}, { 0, 2}, { 2, 0}, { 1, 2}, { 2, 1}, { 0, 3}, { 3, 0}, { 1, 3}, { 3, 1}, { 0, 4}, { 4, 0}, { 2, 3}, { 3, 2}, { 1, 4}, { 4, 1}, { 2, 4}, { 4, 2}, { 3, 4}, { 4, 3} }.

[0161] Note that the above table is only an example, and other combination orders may be used.

[0162] The combination of triangle partition direction flag and table index value (merge list index) The order of the signals (indicating the combination of triangles) may also be changed. The flag may be signaled before or after the table index value. stomach.

[0163] According to another example of the present disclosure, the triangle section direction flag has a probability of having a value of 0 and a probability of having a value of 1. This can be easily coded by assuming that the probability is equal to the probability of the event (i.e., 50% probability). This flag can always be coded as a CABAC bypass bin. be.

[0164] Thus, in one example, the method may include coding a video image into multiple coding units (CUs). At least one of the plurality of coding units is further divided into a plurality of coding units from the upper left corner to the lower right corner. PU of at least one triangle in one of the compartment directions from the upper right corner to the lower left corner It is divided into two prediction units (PUs) containing a single motion vector candidate list. Whether to encode the current CU into triangular prediction mode according to the encoded information signal a segment direction flag indicating the segment direction; The selected entry in the candidate vector list is signaled by an index value. Includes: Separate signaling of merge index value to partition mapping

[0165] According to another example of the present disclosure, to further reduce memory requirements, in triangular prediction mode: For example, additional flags called mapping flags determine how index values ​​are mapped between two The triangle partition is assigned to two index values ​​and two triangles. The mapping scheme between the partitions of the same shape can be signaled. be.

[0166] More specifically, this method is based on the fact that there is a symmetry in the data shown in Table 5. For example, if we look at the first two rows in this table, {0, 1} and {1, 0}, The two rows represent the two triangle partitions in the single-prediction merge list. {0, 1} represents the same motion vector in the merge list, but with different correspondences. The motion vector candidate is assigned to the first partition and the second partition in its merge list. {1, 0} indicates that the th motion vector candidate is assigned to the second partition. indicates that the second motion vector candidate in the merge list is assigned to the first partition. The first motion vector candidate in the list is assigned to the second partition. According to the example of this disclosure, these two cases are both represented by {0, 1}. This is signaled along with a mapping flag to distinguish between these two cases. It is possible. Table 3. Further simplified triangle prediction merge list index signaling g_triangleCombination

[10] [2] = { { 0, 1}, { 0, 2}, { 1, 2}, { 0, 3}, { 1, 3}, { 0, 4}, { 2, 3}, { 1, 4},{ 2, 4}, { 3, 4}, }.

[0167] Based on the above example, this table can be further simplified as Table 6, 10x2 size. In this case, a table index value in the range 0 to 9 is signaled. The merge list index value of indicates which of the two triangles is used for partitioning. Along with the index value, one mapping flag is signaled. For example, When an index value of 0 is signaled, the first and second motion vectors in the merge list are If the mapping flag is 0, the mapping is The first motion vector candidate in the list is assigned to the first partition, The second motion vector candidate in that list is assigned to the second partition. Otherwise, if the mapping flag is 1, The second motion vector candidate in the merge list is assigned to the first partition. The first motion vector candidate in the list is assigned to the second partition. It is possible to show that

[0168] Note that the above table is merely an example, and other combination sequences may be used.

[0169] Note that the combination of mapping flag and table index value (merge list index) The order of notification by the signals (indicating the mapping flag) may also be changed. may be signaled before the table index value or after the table index value.

[0170] According to another example of the present disclosure, the mapping flag may be a value of 0, a value of 1, or a probability of 0. (i.e., 50% probability). Lag can always be coded as a CABAC bypass bin . Sharing merge list index combination table with pairwise average motion vectors

[0171] As explained earlier, during the normal merge mode motion vector candidate list generation process, The pairing order of the input vectors is used when generating the pairwise average motion vector. The example motion vector pairing order is as follows: JPEG2026041750000008.jpg8127

[0172] According to another example of the present disclosure, it is possible to define only one table, which includes: a) a) generating pairwise average motion vectors for normal mode, and b) merging for triangular prediction mode. The signaling of the store index can be shared for both the

[0173] For example, Table 6 is also used to generate pairwise average motion vectors for normal mode. This can be further changed to Table 7 below. Table 4. Shared index combinations g_triangleCombination

[10] [2] = { { 0, 1}, { 0, 2}, { 1, 2}, { 0, 3}, { 1, 3}, { 2, 3}, { 0, 4}, { 1, 4},{ 2, 4}, { 3, 4}, }.

[0174] Note that the first six entries in Table 7 above are in the motion vector pairing order shown previously. Therefore, these six consecutive entries can be considered as a shared part. Cut.

[0175] In another aspect, as mentioned above, according to the current VVC standard draft, the triangle prediction flag is , coded based on the context derived from the triangle prediction flag values ​​of neighboring blocks However, in actual video content, the selection of the triangular prediction mode is However, there are cases where the correlation between adjacent blocks is not high. Another way of creating context has been proposed.

[0176] If you select CIIP mode in the current video encoding / decoding standard, the triangular prediction mode This mode is not normally used, regardless of whether CIIP mode is selected for the CU. First, it is not efficient to always signal the triangle prediction flag of the current CU.

[0177] Additionally, according to the current VVC standard draft, the MMVD flag is true or false. The triangular prediction flag is signaled regardless of whether MMVD mode is used or not. When not used with the CODEC, signaling is mutually It can also be exclusive. Encoding and decoding of triangle prediction flags with different contexts

[0178] According to an example of the present disclosure, the triangle prediction flag for the current CU is coded and decoded (coded). If the context needs to be modified (encoded or decoded), the context is Therefore, it is possible that it may be derived differently from the current VVC standard draft.

[0179] In one example of the present disclosure, the context is the CU region size (i.e., the CU width plus the CU height value). This means that CUs of the same size are You can share your information.

[0180] In some other examples of the present disclosure, the context may be the CU shape (i.e., CU height relative to CU width). For example, a horizontal rectangular CU (i.e., the width of the CU is C U height) share the first context and are vertically oriented rectangular CUs ( (i.e., CU height is greater than CU width) share the second context and are square CUs (i.e., CUs whose height is equal to their width) share a third context. It is possible.

[0181] In another example, the ratio of the long side to the short side is close, whether horizontally or vertically. Rectangular CUs share one context, and square CUs share another. In this case, each has a different ratio of long CU to short CU, and / or uses multiple contexts corresponding to different ranges of the ratio of the long side of the CU to the short side of the CU. For example, for a CU where the ratio of the long side of the CU to the short side of the CU is greater than 4, The first context for the CU long side to the CU short side ratio is greater than 2 and less than 4. The second context is for small CUs, where the ratio of the long side of the CU to the short side of the CU is greater than 1.5. A third context for CUs greater than or equal to 2, the long side of the CU against the short side of the CU. The fourth context is for CUs where the ratio is greater than 1 and less than 1.5, and the square There is a fifth context for the CU: Other embodiments are possible.

[0182] According to yet another example of the present disclosure, the triangle prediction flag is a CU parameter and / or Regardless of the conditions, encoding and decoding are always performed using one common context. Encoding and decoding of triangle prediction flags with respect to CIIP flags

[0183] According to an example of the present disclosure, when the CIIP prediction mode is selected for a CU, The corner prediction flag may not be coded / decoded for this CU. Triangle prediction flag is set only if CIIP prediction mode is not selected for the CU. can be signaled for this CU. If the triangular prediction flag is not signaled, In this case, the decoder can estimate it as 0. Encoding and decoding of triangle prediction flags with respect to MMVD flags

[0184] According to another example of the present disclosure, when the MMVD prediction mode is selected for the CU, , the triangle prediction flag may not be coded or decoded for this CU. Therefore, the triangle prediction frame is used only if the MMVD prediction mode is not selected for the CU. A lag can be signaled for this CU. The triangular prediction flag is signaled. If not, the decoder can infer it to be 0. Enable MMVD prediction on the triangle prediction unit

[0185] According to yet another example of the present disclosure, on the triangle prediction unit, an MMVD prediction mode is used. In this case, the MMVD prediction mode is selected for the CU. Even if the Both MMVD prediction mode and triangular prediction mode are signaled for a CU. If true, the MVD related syntax in MMVD mode uses two triangle prediction units. Each of the ports can be additionally signaled.

[0186] FIG. 15 is a block diagram illustrating an apparatus for video encoding and decoding according to an embodiment of the present disclosure. The device 1500 is a block diagram of a mobile phone, a tablet computer, a digital broadcast terminal, etc. , a tablet device, or a terminal such as a personal digital assistant.

[0187] As shown in FIG. 15, the device 1500 includes a processing unit 1502, a memory 1504, a power supply unit, and 1506, multimedia section 1508, audio section 1510, input / output (I / O) One or more of the interface 1512, the sensor unit 1514, and the communication unit 1516 It may include.

[0188] The processing unit 1502 normally handles display, telephone calls, data communications, camera operations, and recording operations. The processing unit 1502 controls the overall operation of the device 1500, including operations related to the above. One or more instructions for carrying out instructions for implementing all or part of the steps of the method It is possible for the processing unit 1502 to include multiple processors 1520. One or more modules that contribute to the interaction between 1502 and other components For example, the processing unit 1502 may include a multimedia unit 1508 and a processing unit. 1502 includes a multimedia module for contributing to the interaction between Good too.

[0189] The memory 1504 stores different types of data to support the operation of the device 1500. Examples of such data include any data stored in the device 1500. instructions for applications or methods, contact data, phone book data, messages , images, videos, etc. The memory 1504 may be any type of volatile or non-volatile memory. The memory 1504 may be implemented by a statically-running memory device or a combination thereof. Static Random Access Memory (SRAM), electrically erasable Programmable Read-Only Memory (EEPROM) Erasable Read-Only Memory (EPROM), Erasable Programmable Read-Only Memory (EPROM) sable Programmable Read-Only Memory (PRO M: Programmable Read-Only Memory, ROM: Read-Only Memory y), magnetic memory, flash memory, magnetic disk, or compact disk. That's fine.

[0190] The power supply unit 1506 supplies power to each component of the device 1500. a processing system, one or more power sources, and a device 1500 configured to generate, manage, and distribute power; Other components related to the distribution may also be included.

[0191] The multimedia section 1508 provides an output interface between the device 1500 and the user. In one example, the screen includes an LCD and a touch panel (T If the screen includes a touch panel, the screen may include a The touch panel may be implemented as a touch screen that receives input signals from the One or more sensors for sensing touches, slides, and gestures on the touch panel The touch sensor may include a number of touch sensors. It not only senses the field, but also the duration and pressure associated with the touch or slide action. In one example, the multimedia unit 1508 may include a front camera and The device 1500 may include a camera or a rear camera. When in this operating mode, the front and / or rear camera may not function as an external multimedia device. It is possible to receive media data.

[0192] The audio section 1510 is configured to output and / or input audio signals. For example, the audio unit 1510 includes a microphone (MIC). The phone controls whether the device 1500 is in a certain operating mode, such as a call mode, a recording mode, or a voice recognition mode. When in the audio mode, the device is configured to receive an external audio signal. The signal may be further stored in memory 1504 or transmitted via communication unit 1516. In one example, the audio unit 1510 may include a switch for outputting an audio signal. It further includes a peaker.

[0193] The I / O interface 1512 connects the processing unit 1502 to the peripheral interface module. The peripheral interface module mentioned above provides an interface between the These buttons may include a keyboard, a click wheel, or buttons. This includes, but is not limited to, the volume buttons, start button, and lock button. It will not be done.

[0194] The sensor unit 1514 may include one or more sensors for providing status assessments of different aspects of the device 1500. For example, the sensor unit 1514 may detect the on / off state of the device 1500 and For example, the components may be located in the device 1500. The sensor section 1514 is also a display and keypad. The positional changes of the components of the device 1500, the presence or absence of a user's touch on the device 1500, The sensor can detect changes in the orientation or acceleration / deceleration of the sensor, as well as changes in the temperature of the sensor. The sensor unit 1514 is a proximity sensor configured to detect the presence of a nearby object without physical contact. The sensor unit 1514 may include a CMOS sensor used in imaging applications. It may further include an optical sensor such as a MOS or CCD image sensor. The sensor unit 1514 includes an acceleration sensor, a gyro sensor, a magnetic sensor, and a pressure sensor. , or may further include a temperature sensor.

[0195] The communication unit 1516 is configured to facilitate wired or wireless communication between the device 1500 and other devices. The device 1500 may be configured to communicate with a communication network such as WiFi, 4G, or a combination thereof. In one example, the communication unit 151 can access a wireless network based on a standard. 6 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In one example, the communication unit 1516 may be a near field communication (NFC) communication unit to facilitate short-range communication. For example, the NFC module may further include a radio frequency identification (NFC) module. (RFID) technology, Infrared Data Association (IrDA) technology, Ultra-Wideband (UWB) technology, It may be implemented based on Bluetooth (BT) technology and other technologies.

[0196] In one example, the apparatus 1500 may include an application specific integrated circuit (AS) for performing the above method. IC), Digital Signal Processor (DSP), Digital Signal Processing Device (DSPD), Programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controller, microcontroller, microprocessor, or other electronic This may be achieved by one or more of the elements.

[0197] The non-transitory computer-readable storage medium may be, for example, a hard disk drive (H DD), solid state drives (SSD), flash memory, hybrid drives and solid state high Hybrid Drive (SSHD), Read-Only Memory (ROM), Compact Disc Reader These include read-only memory (CD-ROM), magnetic tape, and floppy disks.

[0198] FIG. 16 illustrates a video encoding scheme for motion compensated prediction with triangular prediction according to an embodiment of the present disclosure. 1 is a flowchart illustrating an exemplary process for encoding and decoding.

[0199] In step 1602, the processor 1520 processes the video image by at least one Furthermore, in the partition directions from the upper left corner to the lower right corner 702 and from the upper right corner to the lower left corner 704 The partitioning is performed into two prediction units (PUs) each including at least one triangular PU. In step 1604, the processor Step 1606 constructs a single-prediction motion vector candidate list. , the processor 1520 determines whether the current CU is in a triangular prediction mode according to the coded information. In step 1608, processor 1 determines whether the 520 signals a partition direction flag indicating the partition direction. The processor 1520 then selects the selected element in the constructed single predictor motion vector candidate list. The index value indicating the entry is signaled.

[0200] FIG. 17 illustrates a video encoding / decoding method for triangle prediction flags according to an embodiment of the present disclosure. 1 is a flowchart illustrating an exemplary process for generating a

[0201] In step 1702, the processor 1520 processes the video image by at least one Further, a complex image is partitioned into two PUs each including at least one triangular prediction unit (PU). The image is divided into a number of coding units (CUs).

[0202] In step 1704, the processor 1520 executes a triangular prediction model for the current CU. Determines whether the triangle prediction flag indicating the code should be decoded.

[0203] In step 1706, the processor 1520 performs a triangle prediction function for the current CU. This triangle prediction flag is used by CABAC to determine if a lag should be decoded. is derived.

[0204] The CABAC context for triangle prediction flags is the region size of the CU, the shape of the CU, and a common context. It is served.

[0205] In one example, an apparatus for video encoding and decoding is provided. The apparatus includes a processor 1520, and a memory configured to store instructions executable by the processor. 1504, wherein the processor, upon execution of the instruction, 17. The method according to claim 17, wherein the first and second inputs are connected to a first input port and a second input port.

[0206] In another example, a non-transitory computer-readable storage medium 15 having instructions stored thereon. 04. These instructions, when executed by the processor 1520, The processor executes the method shown in FIG. 16 or the method shown in FIG.

[0207] The description of the present disclosure has been presented for convenience of illustration and is not intended to be exhaustive or to be exhaustive of the disclosed embodiments. The present invention is not intended to be limited to the present invention. Various modifications, variations, and alternative implementations are possible. This will be apparent to one skilled in the art having the benefit of the teachings presented in the foregoing descriptions and the associated drawings. be.

[0208] The embodiments best explain the principles and practical applications of the present invention, and allow those skilled in the art to easily implement the invention. The present invention has been described in detail in order to facilitate a more complete understanding of the principles underlying the invention and various modifications thereof to suit particular applications. They have been selected and described to make the implementation best possible. Therefore, the scope of the disclosure is not limited to the particular examples of implementations disclosed, and includes modifications and other implementations. It should be understood that the present invention is also within the scope of the present disclosure.

Claims

1. The video image is rotated in at least one direction from the top left corner to the bottom right corner and from the top right corner to the bottom left corner. Two prediction units each including at least one triangular PU having one of the partition directions to partitioning the image into a plurality of coding units (CUs) which are partitioned into PUs; constructing a single predictive motion vector candidate list; Whether the current CU is coded as a triangular prediction mode according to the coded information. and notifying a partition direction flag indicating the partition direction by a signal; An image showing a selected entry in the configured single-prediction motion vector candidate list. signaling an index value; 1. A method for video encoding and decoding, comprising:

2. The single predictor motion vector candidate list includes two to five single predictor motion vector candidates. The method of claim 1 , comprising:

3. The partition direction flag is used in context-based adaptive binary arithmetic coding (CABAC) ) encoded as bypass bins.

4. The triangular prediction mode of the current CU is used with all other merge-related modes disabled.

10. The method of claim 1, wherein the method is presumed to be valid based on a determination that the method has been notified by the user. 。

5. The video image is further divided into at least one triangular prediction unit (PU) ) into multiple coding units (CUs) that are partitioned into two prediction units (PUs) each including To do, A triangular prediction flag indicating the triangular prediction mode for the current CU should be decoded. and When it is determined that the triangular prediction flag for the current CU should be decoded, , the triangle prediction is performed by context-based adaptive binary arithmetic coding (CABAC). deriving a measurement flag; Including, The CABAC context for the triangle prediction flags includes the region size of the CU, and a common context. A method for video encoding and decoding derived from the above.

6. Determining whether the triangle prediction flag should be decoded includes: A combined inter and intra prediction (CIIP) mode is selected for the current CU. determining whether the A merge mode (MMVD) prediction mode for the current CU is determining whether the selected 6. The method of claim 5, comprising at least one selected from the group consisting of:

7. Both the MMVD prediction mode and the triangular prediction mode are selected for the current CU. and then, for each of the two PUs, a motion vector difference (MVD) related syntax is signalling, The method of claim 5 further comprising:

8. a processor; a memory configured to store instructions executable by the processor; Including, When the processor executes the instructions, The video image is rotated in at least one direction from the top left corner to the bottom right corner and from the top right corner to the bottom left corner. Two prediction units each including at least one triangular PU having one of the partition directions to partitioning the image into a plurality of coding units (CUs) which are partitioned into PUs, constructing a single predictive motion vector candidate list; Whether the current CU is coded as a triangular prediction mode according to the coded information. Decide whether to notifying a partition direction flag indicating the partition direction by a signal; An image showing a selected entry in the configured single-prediction motion vector candidate list. Signals the index value, 10. An apparatus for video encoding and decoding configured to perform the operations:

9. The single predictor motion vector candidate list includes two to five single predictor motion vector candidates. The apparatus of claim 8 .

10. The partition direction flag is used in context-based adaptive binary arithmetic coding (CABAC) 9. The apparatus of claim 8, wherein the first bin is encoded as a bypass bin.

11. The triangular prediction mode of the current CU is used with all other merge-related modes disabled.

9. The method of claim 8, wherein the device is presumed to be valid based on a determination that the device has been signaled by the device. Equipment.

12. a processor; a memory configured to store instructions executable by the processor; Including, When the processor executes the instructions, The video image is further divided into at least one triangular prediction unit (PU) 2. Partitioning the CU into a plurality of coding units (CUs), each of which is partitioned into two prediction units PU, each of which includes a A triangular prediction flag indicating the triangular prediction mode for the current CU should be decoded. Decide whether to When it is determined that the triangular prediction flag for the current CU should be decoded, , the triangle prediction is performed by context-based adaptive binary arithmetic coding (CABAC). Derive the measurement flag, Perform an operation like The CABAC context for the triangle prediction flags includes the region size of the CU, and a common context.

23. An apparatus for video encoding and decoding, configured to derive a video encoding / decoding signal.

13. The processor further A combined inter and intra prediction (CIIP) mode is selected for the current CU. Determine whether the A merge mode (MMVD) prediction mode for the current CU is Determine whether it is selected, 13. The apparatus of claim 12, configured to perform the following operations:

14. The processor further Both the MMVD prediction mode and the triangular prediction mode are selected for the current CU. and then, for each of the two PUs, a motion vector difference (MVD) related syntax is Notify by signal, 13. The apparatus of claim 12, configured to perform the following operations:

Images