Method and device for video processing and medium

Abstract

Embodiments of the present disclosure provide a solution for video processing. A method for video processing is presented. The method includes determining affine information for use in a video unit for a conversion between the video unit of the video and a bitstream of the video, where the video unit is not an affine Merge coded block or an affine advanced motion vector prediction (AMVP) coded block; storing affine information used in the video unit; and performing a conversion based on the stored affine information.

Description

Technical Field

[0001] The embodiments of this disclosure generally relate to video processing techniques, and more specifically, to the storage and use of affine motion information. Background Technology

[0002] Today, digital video capabilities are being applied to all aspects of people's lives. Various video compression technologies have been proposed for video encoding / decoding, such as MPEG-2, MPEG-4, ITU-T H.263, ITU-T H.264 / MPEG-4 Part 10 Advanced Video Codec (AVC), ITU-T H.265 High Efficiency Video Codec (HEVC) standard, and Multifunctional Video Codec (VVC) standard. However, the overall encoding and decoding efficiency of video encoding and decoding technologies is expected to be further improved. Summary of the Invention

[0003] Embodiments of this disclosure provide a solution for video processing.

[0004] In a first aspect, a method for video processing is proposed. This method includes: for the conversion between video units and the video bitstream, determining affine information used in the video units, where the video units are neither blocks encoded / decoded by Affine Merge nor blocks encoded / decoded by Affine Advanced Motion Vector Prediction (AMVP); storing the affine information used in the video units; and performing the conversion based on the stored affine information. In this way, affine motion information can be stored and used, thereby improving encoding / decoding efficiency and performance.

[0005] In a second aspect, an apparatus for video processing is provided. The apparatus includes a processor and a non-transitory memory having instructions thereon. When executed by the processor, the instructions cause the processor to perform the method according to the first aspect of this disclosure.

[0006] In a third aspect, a non-transitory computer-readable storage medium is proposed. This non-transitory computer-readable storage medium stores instructions that cause a processor to execute the method according to the first aspect of this disclosure.

[0007] In a fourth aspect, another non-transitory computer-readable recording medium is proposed. This non-transitory computer-readable recording medium stores a bitstream of video generated by a method performed by an apparatus for video processing. The method includes: determining affine information used in video units of the video, wherein the video unit is neither a block encoded / decoded by affine Merge nor a block encoded / decoded by affine Advanced Motion Vector Prediction (AMVP); storing the affine information used in the video unit; and generating a bitstream based on the stored affine information.

[0008] In a fifth aspect, a method for storing a bitstream of video is proposed. The method includes: determining affine information used in video units of the video, wherein the video unit is neither a block encoded by Affine Merge nor a block encoded by Affine Advanced Motion Vector Prediction (AMVP); storing the affine information used in the video unit; generating a bitstream based on the stored affine information; and storing the bitstream in a non-transitory computer-readable recording medium.

[0009] This summary aims to present, in a simplified form, the selected concepts further described below in the detailed embodiments. This summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter. Attached Figure Description

[0010] The above and other objects, features, and advantages of exemplary embodiments of the present disclosure will become more apparent from the following detailed description with reference to the accompanying drawings. In the exemplary embodiments of the present disclosure, the same reference numerals generally refer to the same components.

[0011] Figure 1 A block diagram illustrating an example video codec system according to some embodiments of the present disclosure is shown; Figure 2 A block diagram illustrating a first example video encoder according to some embodiments of the present disclosure is shown; Figure 3 A block diagram illustrating an example video decoder according to some embodiments of the present disclosure is shown; Figure 4 This shows the locations of spatial and temporal neighbor blocks used in the construction of the AMVP / Merge candidate list; Figure 5 This shows the positions of non-adjacent candidates in the ECM; Figure 6A and Figure 6B An affine motion model based on control points is shown; Figure 7 An example affine MVF for each sub-block is shown; Figure 8 The location of the inherited affine motion prediction value is shown; Figure 9 This demonstrates the inheritance of control point motion vectors; Figure 10 The locations of candidate positions for the constructed affine Merge pattern are shown; Figure 11A and Figure 11B The spatial nearest neighbor used to derive the affine Merge candidate is shown; Figure 12The affine Merge candidates from non-nearest neighbors to the constructed ones are shown; Figure 13 An example of generating HAPC is shown; Figure 14 A diagram illustrating the regression-based affine Merge candidate derivation is shown. Figure 15 This demonstrates template matching execution over the search area surrounding the initial MV; Figure 16 The template and the corresponding reference template are shown; Figure 17 A template and a reference template are shown for a block with sub-block motion that uses motion information of the current block's sub-blocks; Figure 18 The derivation of the sub-CU motion field obtained by applying motion displacement based on neighbor motion information is shown; Figure 19 An example of GPM partitioning grouped at the same angle is shown; Figure 20 The unidirectional prediction MV selection for geometric segmentation patterns is shown; Figure 21 An exemplary generation of the hybrid weight w_0 using a geometric segmentation pattern is shown; Figure 22 The ramp function for weighting GPM mixing is shown, based on the displacement (d) from the predicted sample location to the GPM segmentation boundary and the mixing region size (τ). Figures 23A-23C The available IPM candidates are shown respectively; Figure 23D GPM with inter-frame and intra-frame prediction is shown; Figure 24 The edges on the template are shown; Figure 25 A flowchart of a method for video processing according to embodiments of the present disclosure is shown; and Figure 26 A block diagram of a computing device in which various embodiments of the present disclosure may be implemented is shown.

[0012] Throughout all the accompanying figures, the same or similar reference numerals generally refer to the same or similar elements. Detailed Implementation

[0013] The principles of this disclosure will now be described with reference to some embodiments. It should be understood that these embodiments are described for illustrative purposes only and to help those skilled in the art understand and implement this disclosure, and do not imply any limitation on the scope of this disclosure. In addition to the methods described below, the disclosure described herein can be implemented in various other ways.

[0014] In the following description and claims, unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains.

[0015] The terms "an embodiment," "embodiment," "example embodiment," etc., used in this disclosure refer to embodiments that may include specific features, structures, or characteristics, but not every embodiment is required to include that specific feature, structure, or characteristic. Furthermore, these phrases do not necessarily refer to the same embodiment. Moreover, when a specific feature, structure, or characteristic is described in conjunction with an example embodiment, it is claimed that, whether explicitly described or not, such a feature, structure, or characteristic affecting its relation to other embodiments is within the knowledge of those skilled in the art.

[0016] It should be understood that although the terms “first” and “second”, etc., may be used herein to describe various elements, these elements should not be limited to these terms. These terms are used only to distinguish one element from another. For example, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element, without departing from the scope of the exemplary embodiments. As used herein, the term “and / or” includes any and all combinations of one or more of the listed terms.

[0017] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments. As used herein, the singular forms “a,” “an,” and “the” are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the terms “comprising,” “including,” “having,” “containing,” and / or “comprising” as used herein indicate the presence of the said features, elements, and / or components, but do not exclude the presence or addition of one or more other features, elements, components, and / or combinations thereof.

[0018] Example Environment Figure 1 This is a block diagram illustrating an example video encoding / decoding system 100 from which the techniques of this disclosure may be utilized. As shown, the video encoding / decoding system 100 may include a source device 110 and a destination device 120. The source device 110 may also be referred to as a video encoding device, and the destination device 120 may also be referred to as a video decoding device. In operation, the source device 110 may be configured to generate encoded video data, and the destination device 120 may be configured to decode the encoded video data generated by the source device 110. The source device 110 may include a video source 112, a video encoder 114, and an input / output (I / O) interface 116.

[0019] Video source 112 may include sources such as video capture devices. Examples of video capture devices include, but are not limited to, interfaces for receiving video data from video content providers, computer graphics systems for generating video data, and / or combinations thereof.

[0020] Video data may include one or more images. Video encoder 114 encodes the video data from video source 112 to generate a bitstream. The bitstream may include a sequence of bits forming an encoded representation of the video data. The bitstream may include encoded images and associated data. An encoded image is an encoded representation of an image. Associated data may include sequence parameter sets, image parameter sets, and other syntax structures. I / O interface 116 may include a modulator / demodulator and / or a transmitter. Encoded video data can be directly transmitted to destination device 120 via network 130A through I / O interface 116. Encoded video data may also be stored on storage medium / server 130B for access by destination device 120.

[0021] The destination device 120 may include an I / O interface 126, a video decoder 124, and a display device 122. The I / O interface 126 may include a receiver and / or a modem. The I / O interface 126 may acquire encoded video data from the source device 110 or the storage medium / server 130B. The video decoder 124 may decode the encoded video data. The display device 122 may display the decoded video data to a user. The display device 122 may be integrated with the destination device 120, or it may be external to the destination device 120, which is configured to interface with an external display device.

[0022] The video encoder 114 and the video decoder 124 can operate according to video compression standards such as the High Efficiency Video Codec (HEVC) standard, the Multi-Functional Video Codec (VVC) standard, and other existing and / or future standards.

[0023] Figure 2 This is a block diagram illustrating an example of a video encoder 200 according to some embodiments of the present disclosure. The video encoder 200 may be... Figure 1 An example of a video encoder 114 in system 100 is shown.

[0024] The video encoder 200 can be configured to implement any or all of the technologies disclosed herein. Figure 2 In the example, the video encoder 200 includes multiple functional components. The techniques described in this disclosure can be shared among the various components of the video encoder 200. In some examples, the processor can be configured to perform any or all of the techniques described in this disclosure.

[0025] In some embodiments, the video encoder 200 may include a segmentation unit 201, a prediction unit 202, a residual generation unit 207, a transform unit 208, a quantization unit 209, an inverse quantization unit 210, an inverse transform unit 211, a reconstruction unit 212, a buffer 213, and an entropy coding unit 214. The prediction unit 202 may include a mode selection unit 203, a motion estimation unit 204, a motion compensation unit 205, and an intra-frame prediction unit 206.

[0026] In other examples, the video encoder 200 may include more, fewer, or different functional components. In one example, the prediction unit 202 may include an intra-block copy (IBC) unit. The IBC unit can perform prediction in an IBC mode, in which at least one reference picture is the picture in which the current video block is located.

[0027] Furthermore, although some components (such as motion estimation unit 204 and motion compensation unit 205) can be integrated, for interpretable purposes, these components are... Figure 2 The examples are shown separately.

[0028] The segmentation unit 201 can segment an image into one or more video blocks. The video encoder 200 and the video decoder 300 can support various video block sizes.

[0029] The mode selection unit 203 can, for example, select one of several coding modes (intra-coding or inter-coding) based on the error result, and provide the resulting intra-coded or inter-coded block to the residual generation unit 207 to generate residual block data, and to the reconstruction unit 212 to reconstruct the coded block for use as a reference image. In some examples, the mode selection unit 203 can select an intra-inter-prediction joint prediction (CIIP) mode, in which prediction is based on inter-prediction signals and intra-prediction signals. In the case of inter-prediction, the mode selection unit 203 can also select a resolution for the block based on the motion vector (e.g., sub-pixel precision or integer pixel precision).

[0030] To perform inter-frame prediction on the current video block, motion estimation unit 204 can generate motion information for the current video block by comparing one or more reference frames from buffer 213 with the current video block. Motion compensation unit 205 can determine the predicted video block for the current video block based on the motion information and decoded samples of images from buffer 213 other than the image associated with the current video block.

[0031] Motion estimation unit 204 and motion compensation unit 205 can perform different operations on the current video block, for example, depending on whether the current video block is in an I-strip, P-strip, or B-strip. As used herein, an "I-strip" can refer to a portion of an image composed of macroblocks, all of which are based on macroblocks within the same image. Furthermore, as used herein, in some aspects, "P-strip" and "B-strip" can refer to portions of an image composed of macroblocks independent of macroblocks within the same image.

[0032] In some examples, motion estimation unit 204 can perform unidirectional prediction on the current video block, and can search reference images in list 0 or list 1 to find a reference video block for the current video block. Motion estimation unit 204 can then generate a reference index indicating the reference image containing the reference video block in list 0 or list 1, and a motion vector indicating the spatial displacement between the current video block and the reference video block. Motion estimation unit 204 can output the reference index, prediction direction indicator, and motion vector as motion information for the current video block. Motion compensation unit 205 can generate a predicted video block for the current video block based on the reference video block indicated by the motion information of the current video block.

[0033] Alternatively, in other examples, motion estimation unit 204 can perform bidirectional prediction on the current video block. Motion estimation unit 204 can search reference images in list 0 to find a reference video block for the current video block, and can also search reference images in list 1 to find another reference video block for the current video block. Motion estimation unit 204 can then generate multiple reference indices and multiple motion vectors, the multiple reference indices indicating multiple reference images containing multiple reference video blocks in lists 0 and 1, and the multiple motion vectors indicating multiple spatial displacements between the multiple reference video blocks and the current video block. Motion estimation unit 204 can output the multiple reference indices and multiple motion vectors of the current video block as motion information for the current video block. Motion compensation unit 205 can generate a predicted video block for the current video block based on the multiple reference video blocks indicated by the motion information of the current video block.

[0034] In some examples, the motion estimation unit 204 can output a complete set of motion information for use in the decoder's decoding process. Alternatively, in some embodiments, the motion estimation unit 204 can reference the motion information of another video block to transmit the motion information of the current video block via a signal. For example, the motion estimation unit 204 can determine that the motion information of the current video block is sufficiently similar to the motion information of neighboring video blocks.

[0035] In one example, motion estimation unit 204 may indicate a value in the syntax structure associated with the current video block that indicates to video decoder 300 that the current video block has the same motion information as another video block.

[0036] In another example, motion estimation unit 204 can identify another video block and motion vector difference (MVD) in the syntax structure associated with the current video block. The motion vector difference indicates the difference between the motion vector of the current video block and the motion vector of the indicated video block. Video decoder 300 can use the motion vector of the indicated video block and the motion vector difference to determine the motion vector of the current video block.

[0037] As discussed above, the video encoder 200 can transmit motion vectors via signals in a predictive manner. Two examples of predictive signaling techniques that can be implemented by the video encoder 200 include Advanced Motion Vector Prediction (AMVP) and Merge Pattern Signaling.

[0038] Intra-prediction unit 206 can perform intra-prediction on the current video block. When intra-prediction unit 206 performs intra-prediction on the current video block, it can generate prediction data for the current video block based on decoded samples from other video blocks in the same frame. The prediction data for the current video block can include the predicted video block and various syntax elements.

[0039] The residual generation unit 207 can generate residual data for the current video block by subtracting (e.g., indicated by a minus sign) multiple predicted video blocks from the current video block. The residual data for the current video block can include residual video blocks corresponding to different sample components of the samples in the current video block.

[0040] In other examples, such as in skip mode, residual data for the current video block may not exist, and residual generation unit 207 may not perform a subtraction operation.

[0041] The transform processing unit 208 can generate one or more transform coefficient video blocks for the current video block by applying one or more transforms to the residual video blocks associated with the current video block.

[0042] After the transform processing unit 208 generates a transform coefficient video block associated with the current video block, the quantization unit 209 can quantize the transform coefficient video block associated with the current video block based on one or more quantization parameter (QP) values ​​associated with the current video block.

[0043] The inverse quantization unit 210 and the inverse transform unit 211 can apply inverse quantization and inverse transform to the transform coefficient video block, respectively, to reconstruct the residual video block from the transform coefficient video block. The reconstruction unit 212 can add the reconstructed residual video block to the corresponding samples of one or more predicted video blocks generated by the prediction unit 202 to produce a reconstructed video block associated with the current video block for storage in the buffer 213.

[0044] After the video block is reconstructed by reconstruction unit 212, a loop filtering operation can be performed to reduce video block artifacts in the video block.

[0045] Entropy encoding unit 214 can receive data from other functional components of video encoder 200. When entropy encoding unit 214 receives data, it can perform one or more entropy encoding operations to generate entropy-encoded data and output a bitstream including the entropy-encoded data.

[0046] Figure 3 This is a block diagram illustrating an example of a video decoder 300 according to some embodiments of the present disclosure. The video decoder 300 may be... Figure 1 An example of video decoder 124 in system 100 is shown.

[0047] The video decoder 300 can be configured to perform any or all of the technologies disclosed herein. Figure 3 In the example, the video decoder 300 includes multiple functional components. The techniques described in this disclosure can be shared among the various components of the video decoder 300. In some examples, the processor can be configured to perform any or all of the techniques described in this disclosure.

[0048] exist Figure 3 In the example, the video decoder 300 includes an entropy decoding unit 301, a motion compensation unit 302, an intra-frame prediction unit 303, an inverse quantization unit 304, an inverse transform unit 305, a reconstruction unit 306, and a buffer 307. In some examples, the video decoder 300 can perform a decoding process that is generally contrasted with the encoding process described with respect to the video encoder 200.

[0049] Entropy decoding unit 301 can retrieve the encoded bitstream. The encoded bitstream may include entropy-encoded video data (e.g., encoded blocks of video data). Entropy decoding unit 301 can decode the entropy-encoded video data, and motion compensation unit 302 can determine motion information from the entropy-decoded video data, including motion vectors, motion vector precision, reference picture list indices, and other motion information. Motion compensation unit 302 can determine such information, for example, by performing AMVP and Merge mode. AMVP is used, which involves deriving several most likely candidates based on data from neighboring PBs and reference pictures. Motion information typically includes horizontal motion vector displacement values ​​and vertical motion vector displacement values, one or two reference picture indices, and, in the case of a prediction region in a B-strip, an identifier of which reference picture list is associated with each index. As used herein, in some aspects, "Merge mode" may refer to deriving motion information from spatially or temporally neighboring blocks.

[0050] The motion compensation unit 302 can generate motion compensation blocks, possibly by performing interpolation based on an interpolation filter. Identifiers for interpolation filters used with sub-pixel precision can be included in the syntax elements.

[0051] The motion compensation unit 302 can use the interpolation filter used by the video encoder 200 during the encoding of a video block to calculate the interpolated values ​​of sub-integer pixels for the reference block. The motion compensation unit 302 can determine the interpolation filter used by the video encoder 200 based on the received syntax information, and the motion compensation unit 302 can use the interpolation filter to generate a prediction block.

[0052] Motion compensation unit 302 may use at least some of the syntax information to determine the size of the blocks used to encode the encoded video sequence (multiple frames) and / or (multiple stripes), segmentation information describing how each macroblock of the image of the encoded video sequence is segmented, a pattern indicating how each segment is encoded, one or more reference frames (and a list of reference frames) for each inter-frame coded block, and other information for decoding the encoded video sequence. As used herein, in some respects, a “strip” can refer to a data structure that can be decoded independently of other stripes of the same image in terms of entropy encoding / decoding, signal prediction, and residual signal reconstruction. A strip can be an entire image or a region of an image.

[0053] Intra-prediction unit 303 can use, for example, an intra-prediction mode received in the bitstream to form prediction blocks from spatially adjacent blocks. Dequantization unit 304 dequantizes (i.e., dequantizes) the quantized video block coefficients provided in the bitstream and decoded by entropy decoding unit 301. Inverse transform unit 305 applies an inverse transform.

[0054] The reconstruction unit 306 can obtain the decoded block, for example, by adding the residual block to the corresponding prediction block generated by the motion compensation unit 302 or the intra-frame prediction unit 303. If necessary, a deblocking filter can also be applied to filter the decoded block to remove block artifacts. The decoded video block is then stored in a buffer 307, which provides a reference block for subsequent motion compensation / intra-frame prediction and also generates decoded video for presentation on a display device.

[0055] Some exemplary embodiments of this disclosure will be described in detail below. It should be noted that section headings are used in this document for ease of understanding and not to limit the embodiments disclosed in a section to that section. Furthermore, although some embodiments are described with reference to multi-function video codecs or other specific video codecs, the disclosed techniques are also applicable to other video codec techniques. Furthermore, although some embodiments describe video encoding steps in detail, it should be understood that the corresponding decoding steps for decoding will be implemented by the decoder. Additionally, the term video processing includes video encoding or compression, video decoding or decompression, and video transcoding, in which video pixels are represented from one compression format to another or at different compression bitrates. 1. Summary of the Invention This disclosure relates to video coding and decoding techniques. Specifically, it concerns affine motion prediction methods in video coding and decoding. These ideas can be applied individually or in various combinations to any standard or non-standard video codec.

[0057] 2. Introduction The exponential growth of multimedia data has posed significant challenges to video encoding and decoding. To meet the ever-increasing demand for more efficient compression technologies, the ITU-T and ISO / IEC have developed a series of video encoding and decoding standards over the past few decades. Specifically, the ITU-T developed the H.261 and H.263 standards, and ISO / IEC developed the MPEG-1 and MPEG-4 visual standards. These two organizations have jointly developed the H.262 / MPEG-2 video standard, the H.264 / MPEG-4 Advanced Video Coding (AVC) standard, the H.265 / HEVC standard, and the latest VVC standard. Starting with H.262 / MPEG-2, a hybrid video encoding and decoding framework has been adopted, in which intra / inter-frame prediction plus transform coding and decoding are used. Figure 4 The location of spatial and temporal neighbor blocks used in the construction of the AMVP / Merge candidate list is shown.

[0058] 2.1 MVP in Video Encoding and Decoding Inter-frame prediction aims to eliminate temporal redundancy between adjacent frames, an indispensable component in hybrid video codec frameworks. Specifically, inter-frame prediction utilizes the content specified by motion vectors (MVs) as the predicted version of the current block to be encoded / decoded, thus only the residual signal and motion information are transmitted in the bitstream. To reduce the cost of MV signaling, motion vector prediction (MVP) emerged as an efficient mechanism for conveying motion information. Early strategies simply used the MV of a specified neighboring block or the median MV of neighboring blocks as the MVP. H.265 / HEVC introduced a contention mechanism where rate-distortion optimization (RDO) selects the best MVP from multiple candidates. Specifically, Advanced MVP (AMVP) mode and Merge mode with different motion information signaling strategies were designed. Using AMVP mode, a reference index, an MVP candidate index referencing the AMVP candidate list, and motion vector difference (MVD) are transmitted via signaling. Regarding Merge mode, only the Merge index referencing the Merge candidate list is transmitted via signaling, and all motion information associated with the Merge candidate is inherited. Both the AMVP and Merge modes require building an MVP candidate list, and the details of the building process for these two modes are described below.

[0059] AMVP mode: AMVP utilizes the spatial-temporal correlation of motion vectors with neighboring blocks, which is used for explicit transfer of motion parameters. For each list of reference images, the motion vector candidate list is constructed by first checking the availability of temporally adjacent positions to the left and top, removing redundant candidates, and adding zero vectors to ensure the candidate list has a constant length. For the spatial motion vector candidate derivation, two motion vector candidates are ultimately based on those located in... Figure 4 The motion vectors of the five blocks at different locations shown are derived. The five neighboring blocks located at B0, B1, B2 and A0, A1 are classified into two groups, where group A includes the three spatial neighboring blocks above and group B includes the two spatial neighboring blocks to the left. Two MV candidates are derived respectively using the first available candidates from group A and group B in a predefined order. For the derivation of temporal motion vector candidates, as follows... Figure 4 As shown, a motion vector candidate is derived based on two distinct co-positions examined sequentially (bottom right (C0) and center (C1)). To avoid redundant MV candidates, duplicate motion vector candidates in the list are discarded. If the number of potential candidates is less than 2, additional zero motion vector candidates are added to the list. Figure 5 The positions of non-adjacent candidates in the ECM are shown.

[0060] Merge modeSimilar to the AMVP model, the MVP candidate list for the Merge model also includes spatial and temporal candidates. For spatial motion vector candidate derivation, after performing availability and redundancy checks, a maximum of four candidates are selected, in the order A1, B1, B0, A0, and B2. For temporal Merge candidate (TMVP) derivation, a candidate is selected from at most two temporally neighboring blocks (C0 and C1). When there are not enough Merge candidates using both spatial and temporal candidates, combined bidirectional prediction Merge candidates and zero MV candidates are added to the MVP candidate list. Once the number of available Merge candidates reaches the maximum allowed number for signal transmission, the Merge candidate list construction process is terminated.

[0061] In VVC, the process of constructing the Merge pattern is further improved by introducing a history-based MVP (HMVP), where the HMVP incorporates motion information from previously encoded / decoded blocks that can be far removed from the current block. In VVC, HMVP Merge candidates are appended to the Merge list, following the Spatial MVP and TMVP. In this method, motion information from previously encoded / decoded blocks is stored in a table and used as the MVP for the current CU. During the encoding / decoding process, the table with multiple HMVP candidates is maintained using a first-in, first-out (FIFO) strategy. Whenever a non-sub-block inter-encoding / decoding CU is present, the associated motion information is added to the last entry of the table as a new HMVP candidate.

[0062] During the standardization of VVC, non-adjacent MVPs were proposed to facilitate better motion information derivation by utilizing non-adjacent regions. In ECM software, non-adjacent MVPs are inserted between TMVPs and HMVPs, where the distance between the non-adjacent spatial candidate and the current codec block is based on the width and height of the current codec block, such as... Figure 5 As shown.

[0063] 2.2 Affine Motion Compensation Prediction In HEVC, only the translational motion model is applied for motion compensation prediction (MCP). In the real world, there are many types of motion, such as zooming in / out, rotation, perspective motion, and other irregular motions. In VVC, block-based affine transformation motion compensation prediction is applied. Figure 6A and Figure 6B An affine motion model based on control points is shown. For example... Figure 6A and Figure 6B As shown, the affine motion field of a block is described by motion information from two control points (4 parameters) or three control point motion vectors (6 parameters).

[0064] For the 4-parameter affine motion model, the motion vector at the sample point position (x, y) in the block is derived as follows:

[0065] For the 6-parameter affine motion model, the motion vector at the sample point position (x, y) in the block is derived as follows:

[0066] in( mv 0x ,mv 0y ) is the motion vector of the upper left control point, ( mv 1x ,mv 1y ) is the motion vector of the upper right control point, and ( mv 2x ,mv 2y ) is the motion vector of the lower left control point.

[0067] To simplify motion compensation prediction, block-based affine transformation prediction is applied. To derive the motion vector for each 4×4 lumen sub-block, the motion vector of the center sample point of each sub-block is calculated according to the above equation, such as... Figure 7 As shown, the values ​​are rounded to 1 / 16 fractional precision. A motion-compensated interpolation filter is then applied to generate a prediction for each sub-block with a derived motion vector. The sub-block size for the chroma components is also set to 4×4. The MV of the 4×4 chroma sub-block is calculated as the average of the MV of the upper-left luminance sub-block and the lower-right luminance sub-block in the corresponding 8×8 luminance region.

[0068] Similar to the inter-frame prediction for translational motion, there are two other inter-frame prediction modes for affine motion: affine Merge mode and affine AMVP mode.

[0069] 2.2.1 Affine Merge Prediction The Affine Merge pattern can be applied to CUs with a width and height greater than or equal to 8. In this pattern, the CPVM of the current CU is generated based on the motion information of spatially neighboring CUs. There can be up to five CPVM candidates, and a signal transmission index indicates which candidate should be used for the current CU. In VVC, the following three types of CPVM candidates are used to form the Affine Merge candidate list: Inherited affine Merge candidates inferred from the CPMV of neighboring CUs; Affine Merge candidate CPMVP constructed using translational MV derivation of neighboring CUs; Zero MV.

[0070] In VVC, there are at most two inherited affine candidates, which are derived from the affine motion model of neighboring blocks: one from the left neighboring CU and one from the upper neighboring CU. Candidate blocks are as follows: Figure 8 As shown. For the left-hand predicted value, the scan order is A0->A1, and for the upper-hand predicted value, the scan order is B0->B1->B2. Only candidates from the first inheritance on each side are selected. Deduplication checks between candidates from two inheritances are not performed. When a neighboring affine CU is identified, its control point motion vector is used to derive the CPMVP candidate in the affine Merge list of the current CU. Figure 9 As shown, if the adjacent lower-left block A is encoded and decoded using affine mode, the motion vectors of the upper-left, upper-right, and lower-left corners of the CU containing block A are obtained. When block A is encoded and decoded using a 4-parameter affine model, according to Calculate the two CPMVs of the current CU. When block A is encoded and decoded using a 6-parameter affine model, according to... Calculate the three CPMVs of the current CU.

[0071] Figure 8 The location of the inherited affine motion prediction value is shown. Figure 9 The inheritance of control point motion vectors is shown.

[0072] The constructed affine candidate means that the candidate is constructed by combining the translational motion information of the neighbors of each control point. The motion information for the control point is derived from... Figure 10 The derivation is shown among the specified spatial and temporal nearest neighbors. CPMVk (k=1,2,3,4) represents the k-th control point. For CPMV1, the B2->B3->A2 block is checked, and the MV of the first available block is used. For CPMV2, the B1->B0 block is checked, and for CPMV3, the A1->A0 block is checked. If the TMVP is available, it is used as CPMV4.

[0073] After the motion signatures (MVs) of the four control points are obtained, affine merge candidates are constructed based on this motion information. The following combinations of control point MVs are used for sequential construction: {CPMV1, CPMV2, CPMV3}, {CPMV1, CPMV2, CPMV4}, {CPMV1, CPMV3, CPMV4}, {CPMV2, CPMV3, CPMV4}, {CPMV1, CPMV2}, {CPMV1, CPMV3} Combinations of three CPMVs construct a 6-parameter affine merge candidate, and combinations of two CPMVs construct a 4-parameter affine merge candidate. To avoid motion scaling, combinations of control point MVs are discarded if the reference indices of the control points are different. Figure 10 The locations of candidate positions for the constructed affine Merge pattern are shown.

[0074] After the inherited affine Merge candidates and the constructed affine Merge candidates are checked, if the list is still not full, a zero MV is inserted at the end of the list.

[0075] 2.2.2 Affine AMVP Prediction The affine AMVP mode can be applied to CUs with a width and height greater than or equal to 16. An affine flag at the CU level is signaled in the bitstream to indicate whether the affine AMVP mode is used, and another flag is signaled to indicate whether it is a 4-parameter affine or a 6-parameter affine. In this mode, the difference between the current CU's CPVM and its predicted CPMVP is signaled in the bitstream. The affine AMVP candidate list is of size 2 and is generated by sequentially using the following four types of CPVM candidates: Inherited affine AMVP candidates inferred from the CPMV of neighboring CUs; A constructive affine AMVP candidate CPMVP derived using translational MV of neighboring CUs; Translation MV from the neighboring CU; Zero MV.

[0076] The checking order for inherited affine AMVP candidates is the same as that for inherited affine Merge candidates. The only difference is that, for AVMP candidates, only affine CUs with the same reference image as those in the current block are considered. Deduplication is not applied when inserting inherited affine motion predictions into the candidate list.

[0077] The constructed AMVP candidate is from Figure 10 The derivation is based on the specified spatial nearest neighbors. The same checking order as in the affine Merge candidate construction is used. Additionally, the reference picture index of neighboring blocks is also checked. The first block in the checking order to be inter-coded and has the same reference picture as in the current CU is used. The current CU uses 4-parameter affine mode encoding and decoding, and... mv0 and mv1 If all three CPMVs are available, they are added as candidates in the affine AMVP list. If the current CU uses 6-parameter affine mode encoding / decoding and all three CPMVs are available, they are added as candidates in the affine AMVP list. Otherwise, the constructed AMVP candidates are set to unavailable.

[0078] Figure 11A and Figure 11BThe spatial nearest neighbors used to derive affine Merge candidates are shown: Figure 11A Used to derive affine Merge candidates for inheritance, and Figure 11B Used to derive the constructed affine Merge candidate.

[0079] If the number of affine AMVP candidates is still less than 2 after the insertion of valid inherited and constructed AMVP candidates, then when available, mv 0、 mv 1 and mv 2 will be added sequentially as translation MVs to predict all control point MVs for the current CU. Finally, if the affine AMVP list is still not full, zero MVs are used to populate the affine AMVP list.

[0080] 2.2.3 New Affine Candidate Derivation Method In ECM-6.0, three additional affine Merge and AMVP candidate derivation methods are integrated: non-adjacent spatial domain candidates, historical parameter-based candidates, and regression-based affine candidates.

[0081] 2.2.3.1 Non-adjacent airspace candidates In ECM-6.0, non-adjacent airspace nearest neighbors are studied to provide candidates for both affine Merge and affine AMVP. The pattern for obtaining non-adjacent airspace candidates is... Figure 11A and Figure 11B As shown in the diagram, similar to the non-adjacent regular merge candidates, the distance between non-adjacent spatial candidates and the current codec block is also defined based on the width and height of the current CU.

[0082] Figure 11A and Figure 11B Motion information of non-adjacent spatial neighbors is used to generate additional inheritance and construct affine merge candidates. Specifically, to generate inheritance candidates, non-adjacent spatial neighbors are checked based on their distance from the current block (i.e., from nearest to farthest). At a specific distance, only the first available neighbors encoded in affine mode from each side of the current block (e.g., left and top) are included. Figure 11A As shown, the checks of the left and top nearest neighbors are performed from bottom to top and from right to left, respectively. For the constructed candidates, such as... Figure 11B As shown, firstly, the positions of non-adjacent spatial neighbors on the left and top are independently determined; then, the positions of the upper left neighbors can be determined accordingly to form a rectangular virtual block together with the non-adjacent neighbors on the left and top. Figure 12The diagram illustrates the affine Merge candidates from non-nearest neighbors to the build. Motion information from three non-nearest neighbors is used to form a CPMV at the top-left (A), top-right (B), and bottom-left (C) of the virtual block, which is projected onto the current CU to generate the corresponding build candidates, as shown below. Figure 12 As shown.

[0083] 2.2.3.2 Affine Candidates Based on Historical Parameters History-based Affine Model Inheritance (HAMI) allows affine models to be inherited from previously affine-encoded blocks that may not be adjacent to the current block. A History Parameter Table (HPT) is established. Each entry in the HPT stores a set of affine parameters: a, b, c, and d, each represented by a 16-bit signed integer. Entries in the HPT are categorized by reference lists and reference indices. Each reference list in the HPT supports five reference indices. The HPT category (denoted as HPTCat) is calculated in a formulaic manner.

[0084] Here, RefList and RefIdx represent the list of reference images (0 or 1) and the reference index, respectively. A maximum of seven entries can be stored for each category, resulting in a total of 70 entries in the HPT. At the beginning of each CTU line, the number of entries for each category is initialized to zero. After decoding the affine-encoded CU with reference lists RefListcur and RefIdxcur, the affine parameters are used to update the entries in category HPTCat(RefListcur, RefIdxcur) in a manner similar to HMVP table updates.

[0085] Candidates based on historical affine parameters (HAPC) from... Figure 10 The MV of the neighboring 4×4 blocks, denoted as A0, A1, B0, B1, or B2, and a set of affine parameters in the corresponding entries stored in the HPT are derived. The MV of the neighboring 4×4 blocks is used as the base MV. Formulatically, the MV of the current block at position (x, y) is calculated as:

[0086] in( , ) represents the MV of the nearest 4×4 block, (x base y base (x, y) represents the center position of the nearest 4×4 block. (x, y) can be the top left, top right, and bottom left corners of the current block to obtain the corner position MV (CPMV) for the current block, or it can be the center of the current block to obtain the regular MV for the current block.

[0087] Figure 13An example of how to derive an HAPC from block A0 is shown. The affine parameters {a0, b0, c0, d0} are directly obtained from an entry in the category HPTIdx(RefListA0, refIdx0A0) in the HPT. The affine parameters from the HPT (with the center position of A0 as the base position and the MV of block A0 as the base MV) are used together to derive the CPMV for either the affine MergeHAPC or the affine AMVP HAPC. They can also be used to derive the MV located at the center of the current block as a regular Merge candidate. The HAPC can be placed into the sub-block-based Merge candidate list, the affine AMVP candidate list, or the regular Merge candidate list. In response to the introduction of new HAPCs, the size of the sub-block-based Merge candidate list is increased from 5 to 10 and 12 for random access and low-latency B configurations, respectively. Furthermore, for the random access configuration, the size of the regular Merge candidate list is increased from 10 to 11 to accommodate the newly added regular Merge candidates.

[0088] 2.2.3.3 Regression-based Affine Candidates In ECM-6.0, regression-based affine merge candidates are derived and added to the affine merge list. The sub-block motion fields from previously encoded and decoded affine CUs and the motion information of neighboring sub-blocks from the current CU are used as inputs to the regression process to derive the proposed affine candidates.

[0089] Previously encoded and decoded affine CUs can be identified from scans of non-adjacent positions and the affine HMVP table. For example... Figure 14 As shown, the information of the adjacent sub-blocks of the current CU is obtained from the 4x4 sub-blocks represented by the gray area. For each sub-block, given a reference list, the corresponding motion vector and center coordinates of the sub-block can be used.

[0090] For each affine CU, at most two affine candidates can be derived. One has neighboring subblock information, and the other does not. All candidates generated by linear regression are deduplicated and collected into a candidate subgroup. When ARMC is enabled, the ARMC process based on TM cost is applied. Subsequently, when N affine CUs are found, at most N candidates generated by linear regression are added to the affine merge list. Figure 14 This is a schematic diagram of the affine Merge candidate derivation based on regression.

[0091] 2.3 Template Matching Merge / AMVP Pattern in ECM Template Matching (TM) Merge / AMVP mode is a decoder-side MV derivation method used to refine the motion information of the current CU by finding the closest match between a template in the current image (i.e., the top and / or left neighboring blocks of the current CU) and a block in the reference image (i.e., the block of the same size as the template). Figure 15 As shown, within the search range of [-8, +8] pixels, a better MV is searched around the initial motion of the current CU. Figure 15 This demonstrates template matching execution over the search area surrounding the initial MV.

[0092] In AMVP mode, an MVP candidate is determined based on the template matching error, selecting the one that minimizes the difference between the current block and the reference block template. Then, the TM process performs MV refinement only on that specific MVP candidate. The TM refines the MVP candidate using an iterative diamond search, starting with full-pixel MVD precision (or 4 pixels for 4-pixel AMVR mode) within a search range of [-8, +8] pixels. AMVP candidates can also be refined using a cross search with full-pixel MVD precision (or 4 pixels for 4-pixel AMVR mode), followed by half-pixels and quarter-pixels sequentially depending on the AMVR mode. This search process ensures that the MVP candidate maintains the same MV precision as indicated by the Adaptive Motion Vector Resolution (AMVR) mode after the TM process.

[0093] In Merge mode, a similar search method is applied to the Merge candidates indicated by the Merge index. TMMerge can proceed up to 1 / 8 pixel MVD accuracy, or skip those accuracies beyond half-pixel MVD accuracy, depending on whether an alternative interpolation filter is used based on the merged motion information (i.e., used when AMVR is in half-pixel mode). Furthermore, when TM mode is enabled, template matching can operate as a standalone process, or as an additional MV refinement process between a block-based bilateral matching (BM) method and a sub-block-based bilateral matching method, depending on whether BM is enabled according to its enable condition check. When both BM and TM are enabled for CU, the TM search process stops at half-pixel MVD accuracy, and the resulting MV is further refined using the same model-based MVD derivation method as in DMVR.

[0094] 2.4 Adaptive Reordering of Merge Candidates (ARMC) Inspired by the spatial correlation between reconstructed neighboring pixels and the current codec block, we propose Adaptive Reordering of Merge Candidates (ARMC) to refine the order of candidates in a given candidate list. The basic assumption is that candidates with lower template matching costs have a higher probability of being selected through the RDO process and should therefore be placed earlier in the list to reduce signaling costs.

[0095] The reordering method is applied to the regular Merge pattern, Template Matching (TM) Merge pattern, and Affine Merge pattern (excluding SbTMVP candidates). For the TM Merge pattern, Merge candidates are reordered before the refinement process.

[0096] After the Merge candidate list is constructed, the Merge candidates are divided into several subgroups. The subgroup size is set to 5. The Merge candidates in each subgroup are reordered in ascending order based on the cost value of template matching. For simplicity, the Merge candidates in the last subgroup (not the first subgroup) are not reordered.

[0097] Template matching cost is measured by the sum of absolute differences (SAD) between the samples of the current block's template and the samples of its corresponding reference template. For example... Figure 16 As shown, the template includes a set of reconstructed samples adjacent to the current block, while the reference template is located using the same motion information as the current block. When the merge candidate utilizes bidirectional prediction, the reference samples of the merge candidate's template are also generated through bidirectional prediction.

[0098] For sub-block size equal to Wsub Hsub's sub-block-based merge candidate has an upper template comprising several sub-templates of size Wsub × K, and a left template comprising several sub-templates of size K × Hsub. For example... Figure 17 As shown, the motion information of the sub-blocks in the first row and first column of the current block is used to derive the reference sample points of each sub-template.

[0099] 2.5 Sub-block-based temporal motion vector prediction (SbTMVP) VVC supports the Sub-Block-Based Temporal Motion Vector Prediction (SbTMVP) method. Similar to TMVP, SbTMVP leverages motion fields in co-located images to facilitate more accurate MVP derivation. The same co-located image used by TMVP is used for SbTVMP. SbTMVP differs from TMVP primarily in two ways. First, SbTMVP enables motion prediction at the sub-CU level, while TMVP predicts CU-level motion. Second, compared to TMVP, which obtains temporal MVs from co-located blocks in the co-located image (co-located blocks are the lower right or center blocks relative to the current CU), SbTMVP applies motion shifting before obtaining temporal motion information from the co-located image. This motion shifting is achieved by reusing the MV of one of the spatially neighboring blocks from the current CU. Figure 16 The template and the corresponding reference template are shown.

[0100] Figure 18 The derivation of the sub-block level motion field for SbTMVP is shown. Specifically, the motion information of the lower left sub-block A1 is acquired first. If any MV in reference list 0 and list 1 points to the same frame, the corresponding MV will be identified as a motion shift. Otherwise, zero MV will be used as a motion shift.

[0101] Once the motion shift is determined, a designated region within the same frame is used to derive the sub-block level motion field. Assuming... Figure 15 As shown, the motion of A1 is used as motion shift. Then, for each sub-CU, the motion information of its corresponding block (the smallest motion grid covering the center sample point) in the co-location image is obtained to provide motion information, wherein the MV scaling operation is first performed to align the reference frame of the temporal motion vector with the reference frame of the current CU. Figure 17 Templates and reference templates are shown for blocks with sub-block motion that use motion information of the current block's sub-blocks. Figure 18 The derivation of the sub-CU motion field obtained by applying motion displacement based on neighbor motion information is shown.

[0102] In VVC and ECM, in addition to the CU-level MVP candidate list, a sub-CU-level MVP candidate list is also constructed to provide more accurate motion predictions for the current CU. This sub-CU-level MVP candidate list includes the motion field generated by both the SbTMVP and AFFINE methods. Specifically, only one SbTMVP candidate is included, and this SbTMVP candidate is always placed as the first entry in the constructed sub-CU-level MVP candidate list. After performing template matching-based reordering, multiple AFFINE candidates are included in the list, with those having lower costs placed earlier.

[0103] 2.6 Geometric Partitioning (GPM) In VVC, geometric segmentation modes are supported for inter-frame prediction. Geometric segmentation modes are transmitted via signaling using CU-level flags as a merge mode, where other merge modes include regular merge mode, MMVD mode, CIIP mode, and sub-block merge mode. This applies to each possible CU size. (in Excluding 8x64 and 64x8), the geometric segmentation mode supports a total of 64 segments.

[0104] When this mode is used, the CU is divided into two geometric segments by a straight line of geometric positioning. Figure 19 The position of the segmentation line is mathematically derived from the angle and offset parameters of a specific segmentation. Each part of the geometric segmentation in the CU is predicted inter-frame using its own motion; only unidirectional prediction is allowed for each segment, i.e., each part has one motion vector and one reference index. Unidirectional prediction motion constraints are applied to ensure that, as with traditional bidirectional prediction, only two motion-compensated predictions are needed for each CU.

[0105] If the geometric segmentation pattern is used for the current CU, the geometric segmentation index (angle and offset) and two merge indices (one for each segment) are further indicated via signal transmission. The number of maximum GPM candidate dimensions is explicitly transmitted in SPS, and the syntax binarization used for the GPM merge index is specified. After predicting each part of the geometric segment, a blending process with adaptive weights is used to adjust the sample values ​​along the geometric segment edges. This is the prediction signal for the entire CU, and the transformation and quantization processes are applied to the entire CU as in other prediction patterns. Finally, the motion field of the CU predicted using the geometric segmentation pattern is stored.

[0106] 2.6.1 Construction of One-Way Prediction Candidate List The unidirectional prediction candidate list is directly derived from the Merge candidate list constructed according to the extended Merge prediction process. Let n denote the index of the unidirectional prediction motion in the geometric unidirectional prediction candidate list. The LX motion vector of the nth extended Merge candidate (where X equals the parity of n) is used as the nth unidirectional prediction motion vector for the geometric segmentation pattern. These motion vectors are... Figure 20 The vector is marked with "x". If the corresponding LX motion vector of the nth extended Merge candidate does not exist, the L(1-X) motion vector of the same candidate is used as the unidirectional predicted motion vector for the geometric segmentation pattern.

[0107] 2.6.2 Blending along geometric segmentation edges After predicting each segment of the geometric segment using its own motion, a blend is applied to the two predicted signals to derive samples around the segmentation edges. The blend weights for each location of the CU are derived based on the distance between the individual location and the segmentation edge.

[0108] For location The distance to the segmentation edge is derived as follows:

[0109] in It is an index used for the angle and offset of geometric segmentation, which depends on the geometric segmentation index transmitted via signal. The sign depends on the angle index. .

[0110] The weights for each part of the geometric segmentation are derived as follows:

[0111] partIdx depends on the angle index Weight An example in Figure 21 As shown in the image, the Figure 21 The mixed weights using geometric segmentation patterns are shown. An example of generation.

[0112] 2.6.3 Geometric Partitioning Pattern (GPM) with Merge Motion Vector Difference (MMVD) GPM in VVC is extended by applying motion vector refinement on top of the existing unidirectional MV of GPM. First, a flag is transmitted to the GPMCU to specify whether this mode is used. If the mode is used, each geometric segment of the GPM CU can further determine whether MVD is transmitted via signaling. If MVD is transmitted for each geometric segment, the segment's motion is further refined using the transmitted MVD information after selecting a GPM Merge candidate. All other processes remain the same as in GPM.

[0113] Similar to MMVD, MVD is transmitted as a pair of distance and direction via signals. In GPM with MMVD (GPM-MMVD), nine candidate distances (1 / 4 pixel, 1 / 2 pixel, 1 pixel, 2 pixel, 3 pixel, 4 pixel, 6 pixel, 8 pixel, 16 pixel) and eight candidate directions (four horizontal / vertical directions and four diagonal directions) are involved. Additionally, when pic_fpel_mmvd_enabled_flag equals 1, MVD is shifted left by 2, just as in MMVD.

[0114] 2.6.4 Geometric Partitioning Mode with Adaptive Blending (GPM) In VVC, the final predicted samples are generated by weighted averaging of the predictions from two predicted signals. The two integer mixing matrices ( W 0 and W 1) Used. The weights in the GPM mixing matrix are derived from the ramp function based on the displacement from the predicted sample location to the GPM segmentation boundary. The mixing region size is fixed at 2 (2 samples on each side of the GPM segmentation boundary).

[0115] The blending process in ECM is improved by adding four additional blending region sizes (one-quarter, half, twice, and four times the existing region size), such as Figure 22 As shown. The CU-level flags are encoded and decoded to accommodate the selected mixing region size via signal transmission. Furthermore, extended weighted precision is utilized, where the maximum weight value changes from 8 (in VVC) to 32 to accommodate the extended mixing region size. Figure 22 The ramp function for GPM mixing is shown, based on the displacement (d) from the predicted sample location to the GPM segmentation boundary and the size of the mixing region (τ).

[0116] 2.6.5 Geometric Segmentation Pattern (GPM) with Template Matching (TM) Template matching is applied to GPM. When GPM mode is enabled for CU, a CU-level flag is signaled to indicate whether TM is applied to the two geometric segments. Motion information for each geometric segment is refined using TM. When TM is selected, the template is constructed using left, top, or left and top neighbor samples based on the segmentation angle, as shown in Table 1. Then, with the half-pixel interpolation filter disabled, motion is refined by minimizing the difference between the current template and the template in the reference image using the same search pattern as the Merge mode.

[0117] Table 1. Templates for the first geometric segmentation and the second geometric segmentation, where A indicates the use of the top sample point, L indicates the use of the left sample point, and L+A indicates the use of both the left and top samples.

[0118]

[0119] The GPM candidate list is constructed as follows: The staggered List-0 and List-1 MV candidates are derived directly from the regular Merge candidate list, with List-0 MV candidates having a higher priority than List-1 MV candidates. A deduplication method with an adaptive threshold based on the current CU size is applied to remove redundant MV candidates.

[0120] The interleaved List-1 and List-0 MV candidates are also derived directly from the regular Merge candidate list, with List-1 MV candidates having a higher priority than List-0 MV candidates. The same deduplication method with an adaptive threshold is also applied to remove redundant MV candidates.

[0121] Zero MV candidates are filled until the GPM candidate list is full.

[0122] GPM-MMVD and GPM-TM are exclusively enabled to a single GPM CU. This is done first by signaling the GPM-MMVD syntax. When both GPM-MMVD control flags are false (i.e., GPM-MMVD is disabled for both GPM segments), the GPM-TM flag is signaled to indicate whether template matching is applied to both GPM segments. Otherwise (if at least one GPM-MMVD flag is true), the value of the GPM-TM flag is presumed to be false.

[0123] 2.6.6 GPM with inter-frame and intra-frame prediction In GPM with inter-frame and intra-frame prediction, the final prediction samples are generated by weighting inter-frame and intra-frame prediction samples for each GPM separately. Inter-frame prediction samples are derived from inter-frame GPMs, while intra-frame prediction samples are derived from the intra-frame prediction mode (IPM) candidate list and the index from the encoder through signal transmission. The IPM candidate list size is predefined as 3. Available IPM candidates are parallel angle mode (parallel mode) for GPM block boundaries, vertical angle mode (vertical mode) for GPM block boundaries, and planar mode, such as... Figures 23A to 23C As shown. Furthermore, as... Figure 23D The GPM with intra-frame and intra-frame prediction shown is limited to reduce signaling overhead for IPM and avoid increasing the size of intra-frame prediction circuitry on the hardware decoder. Additionally, direct motion vectors and IPM storage on the GPM mixing region are introduced to further improve encoding / decoding performance.

[0124] In IPM derivation based on DIMD and neighboring modes, parallel modes are registered first. Therefore, if no identical IPM candidates exist in the list, up to two IPM candidates derived from the decoder-side intra-frame pattern derivation (DIMD) method and / or neighboring block derivation can be registered. As for neighboring mode derivation, there are up to five locations for available neighboring blocks, but they are limited by the angle of the GPM block boundary, as shown in Table 2. These locations have already been used for GPM with template matching (GPM-TM).

[0125] Table 2. Positions of available neighboring blocks for IPM candidate derivation based on the angle of the GPM block boundary. A and L represent the top and left sides of the predicted block.

[0126]

[0127] GPM-intraframe can be combined with GPM with Merge and Motion Vector Difference (GPM-MMVD). TIMD is used as an IPM candidate for GPM-intraframe to further improve encoding and decoding performance. Parallel mode can be registered first, followed by TIMD, DIMD, and IPM candidates from neighboring blocks.

[0128] 2.6.7 Template Matching-Based Reordering for GPM Partitioning Patterns In template-match-based reordering of GPM partitioning patterns, given the motion information of the current GPM block, the corresponding TM value of the GPM partitioning pattern is calculated. Then, all GPM partitioning patterns are reordered in ascending order based on their TM values. Instead of sending the GPM partitioning patterns, an index indicating where the exact GPM partitioning pattern is located in the reordering list is transmitted via signaling, using Golomb-Rice codes.

[0129] The GPM partitioning pattern reordering method is a two-step process performed after the corresponding reference templates for the two GPM partitions in the encoding / decoding unit are generated, as shown below: The GPM segmentation edge is extended to the reference templates of the two GPM segments, resulting in 64 reference templates, and the corresponding TM cost of each of the 64 reference templates is calculated. The TM generation value based on the GPM partitioning pattern is reordered in ascending order for the GPM partitioning patterns, and the best 32 partitioning patterns are marked as available partitioning patterns.

[0130] like Figure 24 As shown, the edges on the template are extended from the edges of the current CU, but the GPM blending process is not applied to the template regions across the edges.

[0131] After ascending reordering using the TM cost, the index is transmitted via signaling.

[0132] 2.6.8 Motion field storage for geometric segmentation patterns Mv1 from the first part of the geometric segmentation, Mv2 from the second part of the geometric segmentation, and Mv, a combination of Mv1 and Mv2, are stored in the motion field of the CU encoded and decoded by the geometric segmentation pattern.

[0133] The type of motion vector stored for each individual location in the sports field is determined as follows:

[0134] Where motionIdx equals It is recalculated from equation (2-36). partIdx depends on the angle index. .

[0135] If sType equals 0 or 1, then Mv0 or Mv1 is stored in the corresponding motion field; otherwise, if sType equals 2, then the Mv resulting from the combination of Mv0 and Mv2 is stored. The combined Mv is generated using the following process: 1) If Mv1 and Mv2 come from different lists of reference images (one from L0 and the other from L1), then Mv1 and Mv2 are simply combined to form a bidirectional predicted motion vector.

[0136] 2) Otherwise, if Mv1 and Mv2 come from the same list, only the unidirectional predicted motion Mv2 is stored.

[0137] 2.7 Multiple Hypothesis Prediction (MHP) In the multi-hypothesis inter-frame prediction mode (JVET-M0425), in addition to the traditional bidirectional prediction signal, one or more additional motion-compensated prediction signals are transmitted via signal transmission. The resulting overall prediction signal is obtained by weighted superposition of samples. Utilizing the bidirectional prediction signal... and the first additional inter-frame prediction signal / hypothesis The generated prediction signal The following was obtained:

[0138] According to the following mapping, the weighting factor It is defined by the new syntax element add_hyp_weight_idx.

[0139]

[0140] Similar to the above, more than one additional prediction signal can be used. The resulting overall prediction signal is accumulated iteratively using each additional prediction signal.

[0141]

[0142] The resulting overall prediction signal is obtained as the last one. (i.e., has the largest index) Within this EE, a maximum of two additional prediction signals can be used (i.e., Limited to 2).

[0143] The motion parameters for each additional prediction hypothesis can be explicitly transmitted via signaling by specifying a reference index, motion vector prediction value index, and motion vector difference, or implicitly transmitted via signaling by specifying a merge index. A separate multi-hypothesis merge flag distinguishes between these two signaling modes.

[0144] For inter-frame AMVP mode, MHP is only applied if unequal weights are selected in BCW in bidirectional prediction mode.

[0145] Combining MHP and BDOF is possible; however, BDOF is only applied to the bidirectional prediction signal portion of the predicted signal (i.e., the ordinary first two assumptions).

[0146] 2.8 Affine Motion Compensation in Geometric Prediction Mode A sub-block-based motion compensation method was proposed for use in GPM mode.

[0147] a) In one example, sub-block-based motion compensation can be affine motion compensation.

[0148] b) In one example, sub-block-based motion compensation could be sbTMVP motion compensation.

[0149] c) In one example, a prediction of at least one geometric segmentation can be generated using sub-block-based motion compensation, such as affine motion compensation.

[0150] d) In one example, the final prediction can be generated by a weighted sum of two predictions, where at least one prediction is generated using sub-block-based motion compensation (such as affine motion compensation).

[0151] i. In one example, the weighted sum is performed using the weight values ​​defined by GPM.

[0152] e) In one example, the two predictions used in the GPM pattern can be of type A and type B, where type A and type B can be (type A and type B can be the same type): i. Non-affine inter-frame prediction; ii. Affine inter-frame prediction; iii. Intra-frame prediction; iv. Intra-block copy (IBC) prediction; v. sb-TMVP inter-frame prediction; vi. Any combination or generated prediction.

[0153] abbreviation ACT Adaptive Color Transformation ALF Adaptive Loop Filter AMVR Adaptive Motion Vector Resolution APS Adaptive Parameter Set AU Access Unit AUD access unit separator AVC (Advanced Video Coding) (Recommendation ITU-T H.264 | ISO / IEC 14496-10) B Two-way prediction BCW features bidirectional prediction with CU-level weights. BDOF bidirectional optical flow BDPCM is based on block-based incremental pulse coding and decoding modulation. BP cache cycle CABAC Context-Based Adaptive Binary Arithmetic Encoding and Decoding CB codec block CBR constant bit rate CCALF Cross-Component Adaptive Loop Filter CPB encoded / decoded image cache CRA completely random access CRC Cyclic Redundancy Check CTB codec tree block CTU encoding / decoding tree unit CU encoding / decoding unit CVS encoded video sequence DPB decodes image cache DCI decoding capability information DRAP depends on random access points DU decoding unit DUI Decoding Unit Information EG Index Columbus EGk k-th exponent Columbus EOB bitstream ends EOS sequence ends FD Fill Data FIFO (First In First Out) FL fixed length GBR green, blue and red GCI General Constraints Information GDR is being gradually decoded and refreshed. GPM geometric segmentation mode HEVC High-Efficiency Video Codec (Recommendation ITU-T H.265 | ISO / IEC 23008-2) HRD Hypothetical Reference Decoder HSS Hypothesis Flow Scheduler Within I-frame IBC Intra-Block Copying IDR instant decoding refresh ILRP Inter-Frame Layer Reference Image IRAP Intra-Frame Random Access Point LFNST Low-Frequency Inseparable Transform LIC local lighting compensation LPS least likely symbol LSB least significant bit LTRP Long-Term Reference Image LMCS with chroma scaling luminance mapping MIP-based intra-frame prediction MPS most likely symbol MSB most significant bit MTS Multiple Transformation Selection MVP motion vector prediction NAL Network Abstraction Layer OBMC Overlap Block Motion Compensation OLS Output Layer Set OP operation point OPI Operation Point Information P prediction PH image header POC image sequential counting PPS Image Parameter Set PROF refines the prediction using optical flow. PT image timer PU image unit QP quantization parameters RADL random access decodeable front-end (image) RASL random access skipped prerequisites (image) RBSP raw byte sequence payload RGB red, green and blue RPL Reference Image List SAO Sample Adaptive Compensation SAR sample amplitude ratio SEI Supplemental Enhancement Information SH strip head SLI sub-picture level information SODB data bit string SP sequence parameter set STRP Short-Term Reference Image STSA Stepwise Temporal Sublayer Access TR truncated Rice VBR Variable Bit Rate VCL video codec layer VPS Video Parameter Set VSEI Multifunctional Supplemental Enhancement Information (Recommendation ITU-T H.274 | ISO / IEC 23002-7) VUI Video Availability Information VVC Multi-Functional Video Codec (Recommendation ITU-T H.266 | ISO / IEC 23090-3) 3. Problems to be solved In existing technologies, affine motion can also be used to generate predictions in GPM. However, it is currently unclear how to store affine motion information in this case.

[0154] 4. Detailed Solution In this disclosure, we propose to refine the affine CPMV using template matching. For a given affine candidate in the affine candidate list, the CPMV can be further refined using template matching, and the refined affine candidates are then used to derive sub-block or pixel-level affine motion information for the current block.

[0155] The detailed embodiments described below should be considered as examples for explaining general concepts. These embodiments should not be interpreted in a narrow sense. Furthermore, these embodiments can be combined in any way.

[0156] The terms “video unit” or “code-decoder unit” or “block” can refer to code-decoder tree block (CTB), code-decoder tree unit (CTU), code-decoder block (CB), CU, PU, ​​TU, PB, TB.

[0157] The term "affine block" can refer to a block encoded or decoded using affine Merge, affine AMVP, or any other affine variant mode (i.e., affine MMVD, etc.), which can be described by motion information of two control points (4 parameters) or three control point motion vectors (6 parameters). The term "CPMV" can refer to the motion information of an affine block at its top-left, top-right, and / or bottom-left corners.

[0158] The term "template" can refer to a reconstructed region that can be used to refine a CPMV, and can mean either a "separate template" or a "uniform template." Here, a "separate template" can refer to a reconstructed region that can be used to refine a single CPMV (i.e., one of the top-left, top-right, and / or bottom-left corners), while a "uniform template" can refer to a reconstructed region that can be used to refine all or any (multiple) CPMVs for a block. The terms "template matching cost" or "TM cost" can refer to the matching cost of a separate template or the matching cost of a uniform template.

[0159] In this disclosure, with respect to "blocks encoded and decoded in mode N", "mode N" can be a predictive mode (e.g., MODE_INTRA, MODE_INTER, MODE_PLT, MODE_IBC, etc.) or a encoding / decoding technique (e.g., DIMD, TIMD, PDPC, CCLM, CCCM, GLM, intraTMP, AMVP, SMVD, Merge, BDOF, PROF, DMVR, AMVR, TM, affine, CIIP, GPM, spatial GPM, SGPM, GPM inter-inter, GPM intra-intra, GPM inter-intra, MHP, GEO, TPM, MMVD, BCW, HMVP, SbTMVP, LIC, OBMC, ALF, deblocking, SAO, bilateral filter, LMCS and corresponding variants, etc.).

[0160] It should be noted that the following terms are not limited to the specific terms defined in existing standards. Any changes to encoding / decoding tools also apply.

[0161] 1. It is proposed that at least one affine information used in a first block can be stored, wherein the first block is not a block encoded or decoded by affine Merge or affine AMVP.

[0162] a) In one example, the first block can be encoded and decoded in GPM mode.

[0163] i. In one example, affine motion compensation can be applied to generate a GPM prediction for the first block.

[0164] b) In one example, the first block can be encoded and decoded in CIIP mode.

[0165] i. In one example, affine motion compensation can be applied to generate CIIP predictions for the first block.

[0166] c) In one example, the first block can be encoded and decoded in MHP mode.

[0167] i. In one example, affine motion compensation can be applied to generate an MHP prediction for the first block.

[0168] d) In one example, the stored affine model can be utilized by at least one block that is encoded / decoded after the first block.

[0169] e) In one example, affine information may include: i. Inter-frame direction (one-way from list 0, one-way or two-way from list 1); ii. CPMV (such as the CPMVs in the top left, top right, and bottom left corners); iii. Affine parameters (such as those in equations (1) and (2)) a, b, c, d, e, f ) iv. Reference index for list 0; v. Reference index for List 1; vi. Local Lighting Compensation (LIC) sign; vii. Overlapping Block Motion Compensation (OBMC) flag; viii. Utilize bidirectional prediction (BCW) indexing with codec unit-level weights; ix. Affine type (4-parameter affine or 6-parameter affine); x. Merge type (affine or sbTMVP); xi. Image index.

[0170] 2. How affine information is stored after encoding and decoding blocks that have been encoded and decoded by GPM depends on the segmentation method of GPM.

[0171] a) For a sub-block (such as a 4×4 sub-block), if it belongs to a first geometric segment with affine motion compensation having a first set of affine information, the first set of affine information can be stored in the sub-block.

[0172] i. Whether a sub-block belongs to the first geometric segmentation can depend on the segmentation method of GPM.

[0173] ii. Determining whether a sub-block belongs to the first geometric segmentation can depend on the sample point positions of the sub-block.

[0174] 1) If the sample point is located in the first geometric segment, then the sub-block can be determined to belong to the first geometric segment.

[0175] 2) The sample point position can be the upper left, upper right, lower left, or lower right position of the sub-block.

[0176] 3) The sample point can be located at the center of a sub-block.

[0177] 4) Assume the top-left position of a sub-block with dimension W×H is (x, y). a) The sample point location can be (x, y) or (x+W-1, y), or (x, y+H-1), or (x+W-1, y+H-1) or (x+W / 2, y+H / 2) or (x+W / 2-1, y+H / 2) or (x+W / 2, y+H / 2-1) or (x+W / 2-1, y+H / 2-1).

[0178] 3. In one example, the identifier (such as the split index) of the first sub-block in a block encoded by GPM can be stored for the first sub-block (such as a 4×4 sub-block).

[0179] a) For a sub-block, if it belongs to the category represented as a segment K (For example, K A geometric division equal to 0 or 1, then K It can be stored in a sub-block.

[0180] i. Whether a sub-block belongs to the first geometric segmentation can depend on the segmentation method of GPM.

[0181] ii. Determining whether a sub-block belongs to the first geometric segmentation can depend on the sample point positions of the sub-block.

[0182] 1) If the sample point is located in the first geometric segment, then the sub-block can be determined to belong to the first geometric segment.

[0183] 2) The sample point position can be the upper left, upper right, lower left, or lower right position of the sub-block.

[0184] 3) The sample point can be located at the center of a sub-block.

[0185] 4) Assume the top-left position of a sub-block with dimension W×H is (x, y). a) The sample point location can be (x, y) or (x+W-1, y), or (x, y+H-1), or (x+W-1, y+H-1) or (x+W / 2, y+H / 2) or (x+W / 2-1, y+H / 2) or (x+W / 2, y+H / 2-1) or (x+W / 2-1, y+H / 2-1).

[0186] 4. In one example, information regarding the segmentation specifications in GPM can be stored for blocks encoded and decoded by GPM.

[0187] a) The stored information can be categorized by the GPM partition index.

[0188] b) Information may include: i. Segmented encoding / decoding mode.

[0189] 1) For example: IntraFlag[ K Determine whether segmentation utilizes intra-frame prediction for encoding and decoding, where K It is a partitioned index.

[0190] 2) For example: AffineFlag[ K Determine whether the segmentation utilizes affine motion compensation during encoding and decoding, where... K It is a partitioned index.

[0191] ii. Segmented motion information.

[0192] iii. Segmented affine information.

[0193] 1) For example, CPMV [ K [List][CpmvIdx] determines CPMV, where K It is a partition index, List can be 0 or 1, and CpmvIdx can be 0, 1 or 2.

[0194] 2) For example, interDir[ K Determine the inter-frame direction (such as from List 0 or List 1 or bidirectional prediction), where K It is a partitioned index.

[0195] 3) For example, refIdx[ K [List] Determines the reference indices for List 0 and List 1, where K It is a partitioned index.

[0196] 4) For example, AffineType[ K Determine the affine type (such as a 6-parameter affine or a 4-parameter affine).

[0197] 5) For example, LICFlag[ K Determine whether to use Local Illumination Compensation (LIC).

[0198] 6) For example, OBMCFlag[ K Determine whether to use Overlapping Block Motion Compensation (OBMC).

[0199] 7) For example, BCWIdx[K] determines the bidirectional prediction (BCW) index using codec unit-level weights.

[0200] 5. In one example, whether a sub-block is encoded or decoded using affine motion compensation can be determined by the segmentation index stored in the sub-block and the encoding / decoding mode information stored for the segmentation of the block encoded / decoded by GPM.

[0201] a) For example, suppose a partitioned index K If stored in a sub-block, then the AffineFlag is stored in the block encoded / decoded by GPM. K It can be determined whether a sub-block is encoded or decoded using affine motion compensation.

[0202] 6. In one example, the affine information of sub-blocks in a GPM-encoded block can be determined by the segmentation index stored in the sub-block and the affine information stored for the segmentation of the GPM-encoded block.

[0203] a) For example, suppose a partitioned indexK If stored in a sub-block, then the CPMV is stored in the block encoded / decoded by GPM. K [List][CpmvIdx] and / or interDir[ K ] and / or refIdx[ K [List] and / or AffineType[ K This allows us to determine the affine motion information of the sub-blocks.

[0204] 7. Affine information used in the first block encoded / decoded using GPM can be used by blocks encoded / decoded after the first block.

[0205] a) In one example, the affine information of sub-blocks in a block encoded and decoded by GPM can be placed into the History Parameter Table (HPT).

[0206] i. In one example, the HPT can only be updated after encoding / decoding the GPM-encoded block if at least one GPM segment of the block is encoded / decoded using affine motion compensation.

[0207] b) In one example, the affine information of sub-blocks in a block encoded / decoded by GPM can be obtained and placed into the affine Merge / AMVP candidate list of blocks encoded / decoded after the first block.

[0208] c) In one example, the affine information of sub-blocks in a block encoded and decoded by GPM can be obtained and placed into the affine candidate list of a second block encoded and decoded using GPM mode.

[0209] 8. A block or sub-block can be examined to determine whether it was encoded or decoded using GPM mode, and whether the block or sub-block belongs to a segment encoded / decoded using affine motion compensation.

[0210] a) In one example, affine information of a block or sub-block belonging to a segment of a block encoded / decoded using affine motion compensation can be used to derive affine-GPM inheritance candidates for a sub-block Merge list or an affine Merge list or an affine AMVP list or an affine list for GPM.

[0211] 9. In one example, when attempting to add an affine-GPM inheritance candidate to a new candidate list, the affine-GPM inheritance candidate can be compared with at least one candidate already in the new candidate list.

[0212] a) For example, if an affine-GPM inheritance candidate is the same as at least one candidate already in the new candidate list, then the affine-GPM inheritance candidate may not be placed in the new candidate list.

[0213] b) For example, if an affine-GPM inheritance candidate is similar to at least one candidate already in the new candidate list, then the affine-GPM inheritance candidate may not be placed in the new candidate list.

[0214] 10. When constructing an affine list that includes at least one affine candidate, such as a sub-block merge list, an affine merge list, an affine AMVP list, or an affine list for GPM, it covers specific sub-blocks or blocks at specific locations that are adjacent to or not adjacent to the current block (such as...). Figure 5 The A0, A1, B0, B1, and B2 in the model can be examined in a specific order using more than one pattern.

[0215] a) For example, whether a particular sub-block or block is affine-coded is checked first (such as affine Merge or affine AMVP).

[0216] i. For example, if a particular sub-block or block is affine-coded, the affine information stored in or associated with that sub-block or block can be used to generate affine candidates for the affine list.

[0217] b) For example, whether a particular sub-block or block is GPM-affine encoded or decoded can be checked by checking whether the particular sub-block or block belongs to a GPM segment encoded or decoded using affine prediction.

[0218] i. For example, if a particular sub-block or block is GPM-affine encoded or decoded, the affine information stored in or associated with that particular sub-block or block can be used to generate affine candidates for the affine list.

[0219] 11. In one example, when constructing an affine list that includes at least one affine candidate, such as a sub-block merge list, an affine merge list, an affine AMVP list, or an affine list for GPM, at a specific step in the list construction, the first sub-block or block at a specific location adjacent or not adjacent to the current block (such as...) is overwritten. Figure 5 A0, A1, B0, B1, and B2 in the code can be examined to generate candidates from blocks or sub-blocks encoded by GPM-affine coding.

[0220] a) For example, after checking specific inherited affine candidates from neighboring blocks, the first sub-block or block can be checked immediately to generate candidates from the blocks or sub-blocks encoded and decoded by GPM-affine.

[0221] b) For example, after checking specific inherited affine candidates from non-adjacent blocks, the first sub-block or block can be checked immediately to generate candidates from the blocks or sub-blocks encoded and decoded by GPM-affine.

[0222] c) For example, after checking a specific RMVF affine candidate, the first sub-block or block can be checked immediately to generate candidates from the block or sub-block that has been GPM-affine encoded or decoded.

[0223] d) For example, after the constructed affine candidate is checked, the first sub-block or block can be checked immediately to generate candidates from the block or sub-block that has been GPM-affine encoded or decoded.

[0224] e) For example, after checking the affine candidates inherited in the time domain, the first sub-block or block can be checked immediately to generate candidates from the blocks or sub-blocks encoded and decoded by GPM-affine.

[0225] f) For example, after checking history-based affine candidates, the first sub-block or block can be checked immediately to generate candidates from the blocks or sub-blocks encoded and decoded by GPM-affine.

[0226] 12. In one example, whether and / or how affine information is stored and / or used can depend on encoding / decoding information such as block location, block dimension, and encoding / decoding mode.

[0227] a) For example, how affine information is stored and / or used can depend on whether the current block is at the CTU boundary.

[0228] 13. Whether and / or how the methods disclosed above are applied can be determined based on (multiple) syntactic elements.

[0229] a) In one example, whether and / or how affine information is stored and / or used can be transmitted from the encoder to the decoder via a signal.

[0230] b) For example, at least one syntax element is transmitted in a bit stream via a signal.

[0231] c) For example, whether and / or how the disclosed methods can be applied can be transmitted via signaling at the sequence level / picture group level / picture level / strip level / piece group level, such as in the sequence header / picture header / SPS / VPS / DPS / DCI / PPS / APS / strip header / piece group header.

[0232] d) For example, whether and / or how the disclosed methods can be applied to transmit signals at PB / TB / CB / PU / TU / CU / VPDU / CTU / CTU lines / strips / films / sub-images / other types of areas containing more than one sample point or pixel.

[0233] e) For example, whether and / or how to apply the methods disclosed above may depend on the encoded / decoded information, such as block size, color format, single / double tree segmentation, color components, and stripe / image type.

[0234] f) For example, whether a syntax element (i.e., an indicator of whether TM refinement is applied to CPMV) is transmitted via signaling can be determined based on another syntax element.

[0235] Overall 14. Additional operations may be applied to the proposed method or applied together with the proposed method.

[0236] a) The syntax elements disclosed above can be binarized into flags, fixed-length codes, EG(x) codes, unary codes, rounded unary codes, rounded binary codes, etc. These can be signed or unsigned.

[0237] b) The syntax elements disclosed above can be encoded or decoded using at least one context model. Alternatively, they can be encoded or decoded in a bypass manner.

[0238] c) The syntax elements disclosed above can be transmitted conditionally via signals.

[0239] a. SE is transmitted via signal only if the corresponding function applies.

[0240] b. SE is transmitted via signal only if the dimensions of the block (width and / or height) meet the conditions.

[0241] d) The syntax elements disclosed above can be transmitted via signaling at the block level / sequence level / picture group level / picture level / strip level / piece group level, such as in the codec structure of CTU / CU / TU / PU / CTB / CB / TB / PB, or in the sequence header / picture header / SPS / VPS / DPS / DCI / PPS / APS / strip header / piece group header.

[0242] e) Whether and / or how the methods disclosed above can be applied to signal transmission at the block level / sequence level / picture group level / picture level / strip level / piece group level, such as in the codec structure of CTU / CU / TU / PU / CTB / CB / TB / PB, or in the sequence header / picture header / SPS / VPS / DPS / DCI / PPS / APS / strip header / piece group header.

[0243] f) Whether and / or how to apply the methods disclosed above may depend on the encoded / decoded information, such as block size, color format, single / dual tree segmentation, color components, and stripe / image type.

[0244] The methods presented in this document can be used in other codec tools that require chroma blending. Figure 25A flowchart of a method 2500 for video processing according to an embodiment of the present disclosure is shown. Method 2500 is implemented during the conversion between video units of a video and a bitstream of a video.

[0245] At box 2510, for the conversion between video units and the video bitstream, the affine information used in the video unit is determined. For example, the video unit is not a block encoded using affine Merge. Or, the video unit is not a block encoded using affine Advanced Motion Vector Prediction (AMVP).

[0246] At box 2520, affine information used in the video unit is stored. At box 2530, a conversion based on the stored affine information is performed. In some embodiments, the conversion includes encoding the video unit into a bitstream. In some embodiments, the conversion includes decoding the video unit from the bitstream. In this way, affine information can be stored and used, thereby improving encoding / decoding efficiency and performance.

[0247] In some embodiments, the video unit is encoded and decoded in a geometrically segmented mode (GPM). In some embodiments, method 2500 further includes generating a GPM prediction for the video unit by applying affine motion compensation.

[0248] In some embodiments, the video unit is encoded and decoded using an intra-frame / inter-frame joint prediction (CIIP) mode. In some embodiments, method 2500 further includes generating CIIP predictions for the video unit by applying affine motion compensation.

[0249] In some embodiments, the video unit is encoded and decoded using a multiple hypothesis prediction (MHP) mode. In some embodiments, method 2500 further includes generating an MHP prediction for the video unit by applying affine motion compensation.

[0250] In some embodiments, the stored affine information is utilized by at least one block that is encoded / decoded after a video unit. In some embodiments, the affine information includes at least one of the following: inter-frame direction, one or more control point motion vectors (CPMV), one or more affine parameters, reference index of list 0, reference index of list 1, local illumination compensation (LIC) flag, overlapping block motion compensation (OBMC) flag, bidirectional prediction (BCW) index using codec unit-level weights, affine type, merge type, or iso-picture index.

[0251] In some embodiments, the method of storing affine information after encoding and decoding a block encoded and decoded by GPM depends on the segmentation method of GPM. In some embodiments, if a sub-block belongs to a first geometric segmentation that applies affine motion compensation with a first set of affine information, the first set of affine information is stored in the sub-block. For example, the sub-block may be a 4×4 sub-block.

[0252] In some embodiments, the determination of whether a sub-block belongs to the first geometric segmentation depends on the segmentation method of the GPM. In some other embodiments, the determination of whether a sub-block belongs to the first geometric segmentation depends on the sample point position of the sub-block.

[0253] In some embodiments, a sub-block is determined to belong to the first geometric segment if the sample point is located within the first geometric segment. For example, the sample point location is at least one of the following for the sub-block: top-left, top-right, bottom-left, or bottom-right. In some embodiments, the sample point location is the center of the sub-block. In some embodiments, the sample point location is (x, y) or (x+W-1, y), or (x, y+H-1), or (x+W-1, y+H-1), or (x+W / 2, y+H / 2), or (x+W / 2-1, y+H / 2), or (x+W / 2, y+H / 2-1), or (x+W / 2-1, y+H / 2-1), where W represents the width of the sub-block, H represents the height of the sub-block, and the top-left location of the sub-block with dimension W×H is (x, y).

[0254] In some embodiments, a segmentation identifier for the first sub-block within a GPM-encoded block is stored for the first sub-block. For example, the segmentation identifier could be a segmentation index.

[0255] In some embodiments, if a sub-block belongs to a geometric segment with segmentation index K, then K is stored in the sub-block, where K is an integer. In some embodiments, K is 0 or 1.

[0256] In some embodiments, the determination of whether a sub-block belongs to the first geometric segmentation depends on the segmentation method of the GPM. In some other embodiments, the determination of whether a sub-block belongs to the first geometric segmentation depends on the sample point position of the sub-block. In some embodiments, if the sample point position is within the first geometric segmentation, the sub-block is determined to belong to the first geometric segmentation. In some embodiments, the sample point position is at least one of the following for the sub-block: top-left, top-right, bottom-left, or bottom-right. In some other embodiments, the sample point position is the center of the sub-block. In some embodiments, the sample point position is (x, y) or (x+W-1, y), or (x, y+H-1), or (x+W-1, y+H-1), or (x+W / 2, y+H / 2), or (x+W / 2-1, y+H / 2), or (x+W / 2, y+H / 2-1), or (x+W / 2-1, y+H / 2-1), where W represents the width of the sub-block, H represents the height of the sub-block, and the top-left position of the sub-block with dimension W×H is (x, y).

[0257] In some embodiments, information regarding the segmentation specified in the GPM is stored for blocks encoded and decoded by the GPM. In some embodiments, the stored information is categorized by a GPM segmentation index.

[0258] In some embodiments, the information includes at least one of the following: the segmentation encoding / decoding mode, the segmentation motion information, or the segmentation affine information. In some embodiments, the segmentation encoding / decoding mode includes at least one of the following: IntraFlag[K] determines whether the segmentation is encoded / decoded using intra-frame prediction, where K is the GPM segmentation index, or AffineFlag[K] determines whether the segmentation is encoded / decoded using affine motion compensation, where K is the GPM segmentation index.

[0259] In some embodiments, the segmented affine information includes at least one of the following: CPMV[K][List][CpmvIdx] determines CPMV, where K is the GPM segmentation index, List is 0 or 1, and CpmvIdx is 0, 1, or 2; interDir[K] determines the inter-frame direction, where K is the GPM segmentation index; refIdx[K][List] determines the reference indexes of List 0 and List 1, where K is the GPM segmentation index; AffineType[K] determines the affine type; LICFlag[K] determines whether Local Illumination Compensation (LIC) is used; OBMCFlag[K] determines whether Overlapping Block Motion Compensation (OBMC) is used; or BCWIdx[K] determines the Bidirectional Prediction (BCW) index using codec unit-level weights. In some embodiments, the inter-frame direction includes at least one of the following: List 0, List 1, or Bidirectional Prediction.

[0260] In some embodiments, whether a sub-block is encoded or decoded using affine motion compensation is determined by a segmentation index stored in the sub-block and encoding / decoding mode information stored for the segmentation of the GPM-encoded block. In some embodiments, if a segmentation index K is stored in the sub-block, a syntax element stored in the GPM-encoded block determines whether the sub-block is encoded or decoded using affine motion compensation. In some embodiments, the syntax element is AffineFlag[K].

[0261] In some embodiments, the affine information of a sub-block within a GPM-encoded block is determined by a segmentation index stored in the sub-block and affine information stored for the segmentation of the GPM-encoded block. In some embodiments, if a segmentation index K is stored in the sub-block, at least one of the following stored in the GPM-encoded block determines the affine motion information of the sub-block: CPMV[K][List][CpmvIdx], interDir[K], refIdx[K][List], or AffineType[K].

[0262] In some embodiments, the affine information used in a video unit encoded / decoded using GPM is used by one or more blocks encoded / decoded after the video unit. In some embodiments, the affine information of sub-blocks in a GPM-encoded block is placed in a History Parameter Table (HPT). In some embodiments, if at least one GPM segment of a block is encoded / decoded using affine motion compensation, the HPT is updated after encoding / decoding the GPM-encoded block.

[0263] In some embodiments, affine information of sub-blocks in a GPM-encoded block is obtained and placed into at least one of the following: an affine merge candidate list or an AMVP candidate list of blocks encoded / decoded after a video unit. In some other embodiments, affine information of sub-blocks in a GPM-encoded block is obtained and placed into an affine candidate list of a second block encoded / decoded using the GPM mode.

[0264] In some embodiments, a block or sub-block is examined to determine whether it was encoded / decoded using GPM mode, and whether the block or sub-block belongs to a segment encoded / decoded using affine motion compensation. In some embodiments, affine information of a block or sub-block belonging to a segment encoded / decoded using affine motion compensation of a block encoded / decoded using GPM is used to derive affine-GPM inheritance candidates for at least one of the following: a sub-block merge list, an affine merge list, an affine AMVP list, or an affine list for GPM.

[0265] In some embodiments, when determining whether to add an affine-GPM inheritance candidate to the candidate list, the affine-GPM inheritance candidate is compared with at least one candidate already in the candidate list. For example, if the affine-GPM inheritance candidate is identical to at least one candidate already in the candidate list, then the affine-GPM inheritance candidate is not added to the candidate list. Similarly, if the affine-GPM inheritance candidate is similar to at least one candidate already in the candidate list, then the affine-GPM inheritance candidate is not added to the candidate list.

[0266] In some embodiments, when constructing an affine list including at least one affine candidate, specific sub-blocks or blocks covering specific locations adjacent or non-adjacent to the current block are examined in a specific order using more than one pattern. For example, specific locations adjacent or non-adjacent to the current block can be referenced. Figure 8 One of A0, A1, B0, B1, and B2 in the above. In some embodiments, the affine list is one of the following: a sub-block merge list, an affine merge list, an affine AMVP list, or an affine list for GPM.

[0267] In some embodiments, whether a particular sub-block or block is affine-coded is checked first. For example, if a particular sub-block or block is affine-coded, the affine information stored in or associated with the particular sub-block or block is used to generate affine candidates for the affine list.

[0268] In some embodiments, whether a particular sub-block or block is GPM-affine encoded or decoded is checked secondarily. That is, whether a particular sub-block or block belongs to a GPM segment encoded or decoded using affine prediction is checked secondarily. In some embodiments, if a particular sub-block or block is GPM-affine encoded or decoded, affine information stored in or associated with the particular sub-block or block is used to generate affine candidates for the affine list.

[0269] In some embodiments, when constructing an affine list including at least one affine candidate, at a specific step in the list construction, a first sub-block or block covering a specific location adjacent or non-adjacent to the current block is examined to generate candidates from the blocks or sub-blocks encoded and decoded by GPM. In some embodiments, the affine list is one of the following: a sub-block merge list, an affine merge list, an affine AMVP list, or an affine list for GPM. For example, the specific location adjacent or non-adjacent to the current block can be referenced. Figure 8 One of A0, A1, B0, B1, and B2 in the list. In some embodiments, when constructing the affine list, neighboring blocks can be checked to generate candidates. Neighboring blocks can be blocks or sub-blocks encoded using GPM-affine coding.

[0270] In some embodiments, after checking whether a specific inherited affine candidate from a neighboring block has been inserted into the affine list, a first sub-block or block is immediately checked to generate candidates from the GPM-affine encoded block or sub-block. In some other embodiments, after checking whether a specific inherited affine candidate from a non-adjacent block has been inserted into the affine list, a first sub-block or block is immediately checked to generate candidates from the GPM-affine encoded block or sub-block.

[0271] In some embodiments, after checking whether a particular regression-based motion vector field (RMVF) affine candidate has been inserted into the affine list, a first sub-block or block is immediately checked to generate candidates from the GPM-affine encoded block or sub-block. In some other embodiments, after checking whether the constructed affine candidates have been inserted into the affine list, a first sub-block or block is immediately checked to generate candidates from the GPM-affine encoded block or sub-block.

[0272] In some embodiments, after checking whether a temporally inherited affine candidate has been inserted into the affine list, a first sub-block or block is immediately checked to generate a candidate from the GPM-affine encoded block or sub-block. In some other embodiments, after checking whether a history-based affine candidate has been inserted into the affine list, a first sub-block or block is immediately checked to generate a candidate from the GPM-affine encoded block or sub-block.

[0273] In some embodiments, whether and / or how affine information is stored and / or used depends on the codec information. For example, the way affine information is stored and / or used depends on whether the current block is at the boundary of a codec tree unit (CTU).

[0274] In some embodiments, whether and / or how a method is applied is determined based on one or more syntax elements. In some embodiments, whether and / or how affine information is stored and / or used is transmitted from the encoder to the decoder via a signal. For example, at least one syntax element is transmitted in a bitstream via a signal.

[0275] In some embodiments, whether and / or how a method is applied is transmitted via signal at one of the following levels: sequence level, picture group level, picture level, strip level, or slice group level. In some embodiments, whether and / or how a method is applied is transmitted via signal at one of the following levels: sequence header, picture header, sequence parameter set (SPS), video parameter set (VPS), dependency parameter set (DPS), decoding capability information (DCI), picture parameter set (PPS), adaptive parameter set (APS), strip header, or slice group header. In some embodiments, whether and / or how a method is applied is transmitted via signal at one of the following levels: prediction block (PB), transform block (TB), codec block (CB), prediction unit (PU), transform unit (TU), codec unit (CU), codec tree block (CTB), codec tree unit (CTU), CTU row, strip, slice, sub-picture, or region comprising more than one sample or pixel. In some embodiments, the method is applied based on encoded and / or decoded information, which includes at least one of the following: block size, color format, single and / or dual tree segmentation, color components, stripe type, or image type.

[0276] In some embodiments, whether a syntax element is determined by signal transmission based on another syntax element. In some embodiments, a syntax element is binarized into one of the following: a flag, a fixed-length code, an EG(x) code, a unary code, a rounded unary code, or a rounded binary code. In some embodiments, a syntax element may be signed or unsigned.

[0277] In some embodiments, syntax elements are encoded or decoded using at least one context model, or the syntax elements are encoded or decoded in a bypass manner. In some embodiments, syntax elements are transmitted conditionally via signals.

[0278] In some embodiments, syntax elements are transmitted via signals if the corresponding function applies. Alternatively, syntax elements are transmitted via signals if the dimensions of the video unit satisfy a condition. In some embodiments, dimensions include the width and / or height of the video unit.

[0279] In some embodiments, syntax elements are transmitted via signals at one of the following: sequence level, picture group level, picture level, stripe level, or slice group level. In some other embodiments, syntax elements are transmitted via signals at one of the following: sequence header, picture header, sequence parameter set (SPS), video parameter set (VPS), dependency parameter set (DPS), decoding capability information (DCI), picture parameter set (PPS), adaptive parameter set (APS), stripe header, or slice group header. In some embodiments, syntax elements are transmitted via signals at one of the following: prediction block (PB), transform block (TB), codec block (CB), prediction unit (PU), transform unit (TU), codec unit (CU), codec tree block (CTB), codec tree unit (CTU), CTU row, stripe, slice, sub-picture, or a region comprising more than one sample or pixel.

[0280] In some embodiments, whether and / or how affine information is stored is transmitted via signal at one of the following locations: sequence level, picture group level, picture level, stripe level, or slice group level. In some other embodiments, whether and / or how affine information is stored is transmitted via signal at one of the following locations: sequence header, picture header, sequence parameter set (SPS), video parameter set (VPS), dependency parameter set (DPS), decoding capability information (DCI), picture parameter set (PPS), adaptive parameter set (APS), stripe header, or slice group header. In some further embodiments, whether and / or how affine information is stored is transmitted via signal at one of the following locations: prediction block (PB), transform block (TB), codec block (CB), prediction unit (PU), transform unit (TU), codec unit (CU), codec tree block (CTB), codec tree unit (CTU), CTU row, stripe, slice, sub-picture, or region comprising more than one sample or pixel.

[0281] In some embodiments, method 2500 further includes determining whether and / or how to store affine information based on the encoded / decoded information of the video unit. The encoded / decoded information may include at least one of the following: block size, color format, single and / or dual tree segmentation, color components, stripe type, or picture type. In some embodiments, the video unit is encoded / decoded using one or more other encoding / decoding tools that require chroma blending.

[0282] According to another embodiment of this disclosure, a non-transitory computer-readable recording medium is provided. This non-transitory computer-readable recording medium stores a bitstream of video generated by a method performed by an apparatus for video processing. The method includes: determining affine information used in video units of the video, wherein the video units are not blocks encoded or decoded by affine Merge or by affine Advanced Motion Vector Prediction (AMVP); storing the affine information used in the video units; and generating a bitstream based on the stored affine information.

[0283] According to further embodiments of this disclosure, a method for storing a bitstream of video is provided. The method includes: determining affine information used in video units of the video, wherein the video units are not blocks encoded or decoded using affine Merge or affine Advanced Motion Vector Prediction (AMVP); storing the affine information used in the video units; generating a bitstream based on the stored affine information; and storing the bitstream in a non-transitory computer-readable recording medium.

[0284] Embodiments of this disclosure can be described according to the following entries, and its features can be combined in any reasonable manner.

[0285] Item 1. A method for video processing, comprising: a conversion between video units of a video and a bitstream of the video; determining affine information used in the video units, wherein the video units are not blocks encoded or decoded by affine Merge or by affine Advanced Motion Vector Prediction (AMVP); storing the affine information used in the video units; and performing the conversion based on the stored affine information.

[0286] Item 2. The method according to Item 1, wherein the video unit is encoded and decoded in a geometric segmentation mode (GPM).

[0287] Item 3. The method according to Item 2 further includes: generating a GPM prediction for the video unit by applying affine motion compensation.

[0288] Item 4. The method according to Item 1, wherein the video unit is encoded and decoded in an intra-frame / inter-frame joint prediction (CIIP) mode.

[0289] Item 5. The method according to Item 4 further includes: generating CIIP predictions for the video unit by applying affine motion compensation.

[0290] Item 6. The method according to Item 1, wherein the video unit is encoded and decoded using a multiple hypothesis prediction (MHP) mode.

[0291] Item 7. The method according to Item 6 further includes: generating an MHP prediction for the video unit by applying affine motion compensation.

[0292] Item 8. The method according to Item 1, wherein the stored affine information is subsequently utilized by at least one block of the video unit for encoding / decoding.

[0293] Item 9. The method according to Item 1, wherein the affine information includes at least one of the following: inter-frame direction, one or more control point motion vectors (CPMV), one or more affine parameters, reference index of List 0, reference index of List 1, local illumination compensation (LIC) flag, overlapping block motion compensation (OBMC) flag, bidirectional prediction (BCW) index using codec unit level weights, affine type, merge type, or iso-picture index.

[0294] Item 10. The method according to Item 1, wherein the manner in which the affine information is stored after the GPM-encoded block is encoded or decoded depends on the segmentation method of the GPM.

[0295] Item 11. The method according to Item 10, wherein if the sub-block belongs to a first geometric segment with affine motion compensation having a first set of affine information, the first set of affine information is stored in the sub-block.

[0296] Item 12. The method according to Item 11, wherein the determination of whether the sub-block belongs to the first geometric segmentation depends on the segmentation method of the GPM.

[0297] Item 13. The method according to Item 12, wherein the determination of whether the sub-block belongs to the first geometric segmentation depends on the sample point position of the sub-block.

[0298] Item 14. The method according to Item 13, wherein if the sample point is located in the first geometric segment, the sub-block is determined to belong to the first geometric segment.

[0299] Item 15. The method according to Item 13, wherein the sample point position is at least one of the following of the sub-block: upper left position, upper right position, lower left position, or lower right position.

[0300] Item 16. The method according to Item 13, wherein the sample point location is the center of the sub-block.

[0301] Item 17. The method according to Item 13, wherein the sample point position is (x, y) or (x+W-1, y), or (x, y+H-1), or (x+W-1, y+H-1) or (x+W / 2, y+H / 2) or (x+W / 2-1, y+H / 2) or (x+W / 2, y+H / 2-1) or (x+W / 2-1, y+H / 2-1), where W represents the width of the sub-block, H represents the height of the sub-block, and the upper left position of the sub-block with dimension W×H is (x, y).

[0302] Item 18. The method according to Item 1, wherein the identifier of the segmentation of the first sub-block in the GPM-encoded block is stored for the first sub-block.

[0303] Item 19. The method according to Item 18, wherein if a sub-block belongs to a geometric segmentation with a segmentation index of K, then K is stored in the sub-block, where K is an integer.

[0304] Item 20. The method according to Item 19, wherein K is 0 or 1.

[0305] Item 21. The method according to Item 19, wherein the determination of whether the sub-block belongs to the first geometric segmentation depends on the segmentation method of the GPM.

[0306] Item 22. The method according to Item 19, wherein the determination of whether the sub-block belongs to the first geometric segment depends on the sample point position of the sub-block.

[0307] Item 23. The method according to Item 22, wherein if the sample point is located in the first geometric segment, the sub-block is determined to belong to the first geometric segment.

[0308] Item 24. The method according to Item 22, wherein the sample point position is at least one of the following of the sub-block: upper left position, upper right position, lower left position, or lower right position.

[0309] Item 25. The method according to Item 22, wherein the sample point location is the center of the sub-block.

[0310] Item 26. The method according to Item 22, wherein the sample point position is (x, y) or (x+W-1, y), or (x, y+H-1), or (x+W-1, y+H-1) or (x+W / 2, y+H / 2) or (x+W / 2-1, y+H / 2) or (x+W / 2, y+H / 2-1) or (x+W / 2-1, y+H / 2-1), where W represents the width of the sub-block, H represents the height of the sub-block, and the upper left position of the sub-block with dimension W×H is (x, y).

[0311] Item 27. The method according to Item 1, wherein information specified for segmentation in the GPM is stored for blocks encoded and decoded by the GPM.

[0312] Item 28. The method described in Item 27, wherein the stored information is categorized by a GPM segmentation index.

[0313] Item 29. The method according to Item 27, wherein the information includes at least one of the following: the encoding / decoding mode of the segment, the motion information of the segment, or the affine information of the segment.

[0314] Item 30. The method according to Item 29, wherein the encoding / decoding mode of the segmentation includes at least one of the following: IntraFlag[K] determines whether the segmentation is encoded / decoded using intra-frame prediction, where K is the GPM segmentation index, or AffineFlag[K] determines whether the segmentation is encoded / decoded using affine motion compensation, where K is the GPM segmentation index.

[0315] Item 31. The method according to Item 29, wherein the segmented affine information includes at least one of the following: CPMV[K][List][CpmvIdx] determines CPMV, where K is the GPM segmentation index, List is 0 or 1, and CpmvIdx is 0, 1, or 2; interDir[K] determines the inter-frame direction, where K is the GPM segmentation index; refIdx[K][List] determines the reference indexes of List 0 and List 1, where K is the GPM segmentation index; AffineType[K] determines the affine type; LICFlag[K] determines whether Local Illumination Compensation (LIC) is used; OBMCFlag[K] determines whether Overlap Block Motion Compensation (OBMC) is used; or BCWIdx[K] determines the Bidirectional Prediction (BCW) index using codec unit-level weights.

[0316] Item 32. The method according to Item 31, wherein the inter-frame direction includes at least one of the following: list 0, list 1, or bidirectional prediction.

[0317] Item 33. The method according to Item 1, wherein whether a sub-block is encoded or decoded using affine motion compensation is determined by a segmentation index stored in the sub-block and encoding / decoding mode information stored for the segmentation of the block encoded / decoded by GPM.

[0318] Item 34. The method according to Item 33, wherein if the segmentation index K is stored in the sub-block, the syntax elements stored in the GPM-encoded block determine whether the sub-block is encoded or decoded using affine motion compensation.

[0319] Item 35. The method described in Item 34, wherein the syntax element is AffineFlag[K].

[0320] Item 36. The method according to Item 1, wherein the affine information of a sub-block in a GPM-encoded block is determined by a segmentation index stored in the sub-block and affine information stored for the segmentation of the GPM-encoded block.

[0321] Item 37. The method according to Item 36, wherein if the segmentation index K is stored in the sub-block, the affine motion information of the sub-block is determined by at least one of the following stored in the GPM-encoded block: CPMV[K][List][CpmvIdx], interDir[K], refIdx[K][List], or AffineType[K].

[0322] Item 38. The method according to Item 1, wherein the affine information used in the video unit encoded / decoded using GPM is used by one or more blocks encoded / decoded after the video unit.

[0323] Item 39. The method according to Item 38, wherein the affine information of sub-blocks in a block encoded and decoded by GPM is placed in a history parameter table (HPT).

[0324] Item 40. The method according to Item 39, wherein if at least one GPM segment of the block is encoded / decoded using affine motion compensation, the HPT is updated after encoding / decoding the GPM-encoded block.

[0325] Item 41. The method according to Item 38, wherein the affine information of the sub-blocks in the GPM-encoded block is obtained and placed into at least one of the following: an affine Merge candidate list or an AMVP candidate list of blocks subsequently encoded or decoded by the video unit.

[0326] Item 42. The method according to Item 38, wherein the affine information of the sub-blocks in the GPM-encoded block is obtained and placed into the affine candidate list of the second block encoded using the GPM mode.

[0327] Item 43. The method according to Item 1, wherein a block or sub-block is examined to determine whether it is encoded or decoded using GPM mode, and said block or sub-block belongs to a segment encoded / decoded using affine motion compensation.

[0328] Item 44. The method according to Item 43, wherein affine information of the block or sub-block belonging to the segmentation of the block encoded / decoded with affine motion compensation using the block encoded / decoded with GPM is used to derive an affine-GPM inheritance candidate for at least one of the following: a sub-block merge list, an affine merge list, an affine AMVP list, or an affine list for GPM.

[0329] Item 45. The method according to Item 1, wherein when determining whether to add an affine-GPM inheritance candidate to the candidate list, the affine-GPM inheritance candidate is compared with at least one candidate already in the candidate list.

[0330] Item 46. The method according to Item 45, wherein if the affine-GPM inheritance candidate is the same as the at least one candidate already in the candidate list, then the affine-GPM inheritance candidate is not placed in the candidate list.

[0331] Item 47. The method according to Item 45, wherein if the affine-GPM inheritance candidate is similar to the at least one candidate already in the candidate list, then the affine-GPM inheritance candidate is not placed in the candidate list.

[0332] Item 48. The method according to Item 1, wherein when constructing an affine list including at least one affine candidate, specific sub-blocks or blocks covering specific locations adjacent or not adjacent to the current block are examined in a specific order using more than one pattern.

[0333] Item 49. The method according to Item 48, wherein the affine list is one of the following: a sub-block merge list, an affine merge list, an affine AMVP list, or an affine list for GPM.

[0334] Item 50. According to the method described in Item 48, whether the particular sub-block or block is affine-coded is checked first.

[0335] Item 51. The method according to Item 50, wherein if the particular sub-block or block is affine encoded or decoded, affine information stored in or associated with the particular sub-block or block is used to generate affine candidates for the affine list.

[0336] Item 52. The method according to Item 48, wherein whether the particular sub-block or block is GPM-affine encoded or decoded is subsequently checked.

[0337] Item 53. According to the method described in Item 52, whether the particular sub-block or block belongs to a GPM segment encoded and decoded using affine prediction is subsequently checked.

[0338] Item 54. The method according to Item 53, wherein if the particular sub-block or block is GPM-affine encoded or decoded, the affine information stored in or associated with the particular sub-block or block is used to generate affine candidates for the affine list.

[0339] Item 55. The method according to Item 1, wherein when constructing an affine list including at least one affine candidate, at a specific step in the list construction, a first sub-block or block covering a specific location adjacent or not adjacent to the current block is examined to generate a candidate from the GPM-affine encoded block or the GPM-affine encoded sub-block.

[0340] Item 56. The method according to Item 55, wherein the affine list is one of the following: a sub-block merge list, an affine merge list, an affine AMVP list, or an affine list for GPM.

[0341] Item 57. The method according to Item 55, wherein after checking whether a specific inherited affine candidate from a neighboring block has been inserted into the affine list, the first sub-block or block is immediately checked to generate candidates from the GPM-affine encoded block or the GPM-affine encoded sub-block.

[0342] Item 58. The method according to Item 55, wherein after checking whether a specific inherited affine candidate from a non-adjacent block is inserted into the affine list, the first sub-block or block is immediately checked to generate candidates from the GPM-affine encoded block or the GPM-affine encoded sub-block.

[0343] Item 59. The method according to Item 55, wherein after checking whether a particular regression-based motion vector field (RMVF) affine candidate is inserted into the affine list, the first sub-block or block is immediately checked to generate candidates from the GPM-affine encoded block or the GPM-affine encoded sub-block.

[0344] Item 60. The method according to Item 55, wherein after checking whether the constructed affine candidate is inserted into the affine list, the first sub-block or block is immediately checked to generate a candidate from the GPM-affine encoded block or the GPM-affine encoded sub-block.

[0345] Item 61. The method according to Item 55, wherein after checking whether a temporal inheritance affine candidate is inserted into the affine list, the first sub-block or block is immediately checked to generate a candidate from the GPM-affine encoded block or the GPM-affine encoded sub-block.

[0346] Item 62. The method according to Item 55, wherein after checking whether a history-based affine candidate has been inserted into the affine list, the first sub-block or block is immediately checked to generate a candidate from the GPM-affine encoded block or the GPM-affine encoded sub-block.

[0347] Item 63. The method according to Item 1, wherein whether and / or using affine information, and / or the manner in which affine information is stored and / or used, depends on the encoding / decoding information.

[0348] Item 64. The method of Item 1, wherein the manner in which affine information is stored and / or used depends on whether the current block is at the boundary of a codec tree unit (CTU).

[0349] Item 65. The method according to any one of items 1-64, wherein whether and / or how the method is applied is determined based on one or more syntactic elements.

[0350] Item 66. The method according to Item 65, wherein whether and / or how affine information is stored and / or used is transmitted from the encoder to the decoder via a signal.

[0351] Item 67. The method according to Item 65, wherein at least one syntax element is transmitted in the bit stream via a signal.

[0352] Item 68. The method according to Item 65, wherein whether and / or how the method is applied is transmitted by signal in one of the following: sequence level, picture group level, picture level, strip level, or slice group level.

[0353] Item 69. The method according to Item 65, wherein whether and / or how the method is applied is transmitted via a signal in one of the following: sequence header, picture header, sequence parameter set (SPS), video parameter set (VPS), dependency parameter set (DPS), decoding capability information (DCI), picture parameter set (PPS), adaptive parameter set (APS), strip header, or slice header.

[0354] Item 70. The method according to Item 65, wherein whether and / or how the method is applied is transmitted by signal at one of the following locations: prediction block (PB), transform block (TB), codec block (CB), prediction unit (PU), transform unit (TU), codec unit (CU), codec tree block (CTB), codec tree unit (CTU), CTU row, strip, slice, sub-picture, or region comprising more than one sample point or pixel.

[0355] Item 71. The method according to Item 65, wherein the method is applied based on the encoded information, the encoded information including at least one of the following: block size, color format, single and / or dual tree segmentation, color components, stripe type, or picture type.

[0356] Item 72. The method according to Item 65, wherein whether a syntax element is determined by signal transmission based on another syntax element.

[0357] Item 73. The method according to any one of items 1-72, wherein the syntax element is binarized into one of the following: a flag, a fixed-length code, an EG(x) code, a unary code, a rounded unary code, or a rounded binary code.

[0358] Item 74. The method according to Item 73, wherein the syntax element is signed or unsigned.

[0359] Item 75. The method according to any one of items 1-72, wherein the syntax element is encoded or decoded using at least one context model, or wherein the syntax element is encoded or decoded by bypass.

[0360] Item 76. The method according to any one of items 1-75, wherein the syntax element is transmitted via signal in a conditional manner.

[0361] Item 77. The method according to Item 76, wherein the syntax element is transmitted via signaling if the corresponding function is applicable, or wherein the syntax element is transmitted via signaling if the dimension of the video unit satisfies a condition.

[0362] Item 78. The method according to Item 77, wherein the dimension includes the width and / or height of the video unit.

[0363] Item 79. The method according to any one of items 1-78, wherein the syntax element is transmitted by signal at one of the following: sequence level, picture group level, picture level, strip level, or slice group level.

[0364] Item 80. The method according to any one of Items 1-78, wherein the syntax elements are transmitted by signal at one of the following locations: sequence header, picture header, sequence parameter set (SPS), video parameter set (VPS), dependency parameter set (DPS), decoding capability information (DCI), picture parameter set (PPS), adaptive parameter set (APS), strip header, or slice header.

[0365] Item 81. The method according to any one of Items 1-78, wherein the syntax element is transmitted by signal at one of the following locations: prediction block (PB), transform block (TB), codec block (CB), prediction unit (PU), transform unit (TU), codec unit (CU), codec tree block (CTB), codec tree unit (CTU), CTU line, strip, slice, sub-picture, or region comprising more than one sample point or pixel.

[0366] Item 82. The method according to any one of items 1-81, wherein whether and / or how the affine information is stored is transmitted by signal at one of the following: sequence level, picture group level, picture level, strip level, or slice group level.

[0367] Item 83. The method according to any one of items 1-81, wherein the affine information is transmitted by signal at one of the following locations: sequence header, picture header, sequence parameter set (SPS), video parameter set (VPS), dependency parameter set (DPS), decoding capability information (DCI), picture parameter set (PPS), adaptive parameter set (APS), strip header, or slice header.

[0368] Item 84. The method according to any one of items 1-81, wherein the affine information is stored by signal transmission at one of the following locations: prediction block (PB), transform block (TB), codec block (CB), prediction unit (PU), transform unit (TU), codec unit (CU), codec tree block (CTB), codec tree unit (CTU), CTU row, strip, slice, sub-picture, or region comprising more than one sample point or pixel.

[0369] Item 85. The method according to any one of items 1-81 further comprises: determining, based on the encoded and decoded information of the video unit, whether and / or how to store the affine information, the encoded and decoded information including at least one of the following: block size, color format, single and / or dual tree segmentation, color components, stripe type, or picture type.

[0370] Item 86. The method according to any one of items 1-85, wherein the video unit is encoded or decoded using one or more other encoding / decoding tools that require chroma blending.

[0371] Item 87. The method according to any one of items 1-86, wherein the conversion includes encoding the video unit into the bitstream.

[0372] Item 88. The method according to any one of items 1-86, wherein the conversion comprises decoding the video unit from the bitstream.

[0373] Item 89. An apparatus for video processing, comprising a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to perform the method according to any one of items 1-88.

[0374] Item 90. A non-transitory computer-readable storage medium storing instructions that cause a processor to execute the method according to any one of items 1-88.

[0375] Item 91. A non-transitory computer-readable recording medium storing a bitstream of video generated by a method performed by means of a video processing apparatus, wherein the method comprises: determining affine information used in video units of the video, wherein the video units are not blocks encoded or decoded by affine Merge or by affine Advanced Motion Vector Prediction (AMVP); storing the affine information used in the video units; and generating the bitstream based on the stored affine information.

[0376] Item 92. A method for storing a bitstream of video, comprising: determining affine information used in video units of the video, wherein the video units are not blocks encoded or decoded by affine Merge or by affine Advanced Motion Vector Prediction (AMVP); storing the affine information used in the video units; generating the bitstream based on the stored affine information; and storing the bitstream in a non-transitory computer-readable recording medium.

[0377] Example device Figure 26 A block diagram of a computing device 2600 in which various embodiments of the present disclosure may be implemented is shown. The computing device 2600 may be implemented as a source device 110 (or video encoder 114 or 200) or a destination device 120 (or video decoder 124 or 300), or may be included in a source device 110 (or video encoder 114 or 200) or a destination device 120 (or video decoder 124 or 300).

[0378] It should be understood that, Figure 26 The computing device 2600 shown is for illustrative purposes only and is not intended to imply any limitation on the functionality and scope of the embodiments of this disclosure.

[0379] like Figure 26 As shown, computing device 2600 includes general-purpose computing device 2600. Computing device 2600 may include at least one or more processors or processing units 2610, memory 2620, storage unit 2630, one or more communication units 2640, one or more input devices 2650, and one or more output devices 2660.

[0380] In some embodiments, the computing device 2600 can be implemented as any user terminal or server terminal with computing capabilities. The server terminal can be a server provided by a service provider, a large computing device, etc. The user terminal can be, for example, any type of mobile terminal, fixed terminal, or portable terminal, including mobile phones, stations, units, devices, multimedia computers, multimedia tablet computers, internet nodes, communicators, desktop computers, laptop computers, notebook computers, netbook computers, tablet computers, personal communication system (PCS) devices, personal navigation devices, personal digital assistants (PDAs), audio / video players, digital cameras / camcorders, positioning devices, television receivers, radio receivers, e-book devices, gaming devices, or any combination thereof, including accessories and peripherals of these devices, or any combination thereof. It is conceivable that the computing device 2600 can support any type of interface to the user (such as "wearable" circuitry devices, etc.).

[0381] Processing unit 2610 can be a physical processor or a virtual processor, and can perform various processes based on programs stored in memory 2620. In a multiprocessor system, multiple processing units execute computer-executable instructions in parallel to improve the parallel processing capability of computing device 2600. Processing unit 2610 may also be referred to as a central processing unit (CPU), microprocessor, controller, or microcontroller.

[0382] Computing device 2600 typically includes various computer storage media. Such media can be any media accessible by computing device 2600, including but not limited to volatile and non-volatile media, or removable and non-removable media. Memory 2620 can be volatile memory (e.g., registers, cache, random access memory (RAM)), non-volatile memory (such as read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), or flash memory) or any combination thereof. Storage cell 2630 can be any removable or non-removable media and may include machine-readable media, such as memory, flash drives, disks, or other media that can be used to store information and / or data and can be accessed within computing device 2600.

[0383] The computing device 2600 may also include additional removable / non-removable storage media, volatile / non-volatile storage media. Although in Figure 26Not shown, but a disk drive for reading from and / or writing to a removable non-volatile disk, and an optical disc drive for reading from and / or writing to a removable non-volatile optical disc may be provided. In this case, each drive may be connected to a bus (not shown) via one or more data media interfaces.

[0384] Communication unit 2640 communicates with another computing device via a communication medium. Furthermore, the functionality of components in computing device 2600 can be achieved by a single computing cluster or by multiple computing machines that can communicate via communication connections. Therefore, computing device 2600 can operate in a networked environment using logical connections to one or more other servers, networked personal computers (PCs), or other general-purpose network nodes.

[0385] Input device 2650 can be one or more of a variety of input devices, such as a mouse, keyboard, trackball, voice input device, etc. Output device 2660 can be one or more of a variety of output devices, such as a monitor, speaker, printer, etc. With the aid of communication unit 2640, computing device 2600 can also communicate with one or more external devices (not shown), such as storage devices and display devices. Computing device 2600 can also communicate with one or more devices that enable a user to interact with computing device 2600, or, if needed, with any device (e.g., network card, modem, etc.) that enables computing device 2600 to communicate with one or more other computing devices. Such communication can be performed via an input / output (I / O) interface (not shown).

[0386] In some embodiments, some or all of the components of computing device 2600 may be arranged in a cloud computing architecture, rather than being integrated into a single device. In a cloud computing architecture, components may be remotely provided and work together to achieve the functionality described herein. In some embodiments, cloud computing provides computing, software, data access, and storage services without requiring end users to know the physical location or configuration of the systems or hardware providing these services. In various embodiments, cloud computing provides services via a wide area network (WAN), such as the Internet, using suitable protocols. For example, a cloud computing provider provides applications via a WAN that can be accessed through a web browser or any other computing component. The software or components of the cloud computing architecture, along with the corresponding data, may be stored on servers at remote locations. Computing resources in a cloud computing environment may be consolidated or distributed across locations in remote data centers. Cloud computing infrastructure may provide services through shared data centers, although they appear as a single access point to users. Therefore, a cloud computing architecture can be used to provide the components and functionality described herein from service providers at remote locations. Alternatively, the components and functionality described herein may be provided by conventional servers or installed directly or otherwise on client devices.

[0387] In embodiments of this disclosure, computing device 2600 may be used to implement video encoding / decoding. Memory 2620 may include one or more video encoding / decoding modules 2625 having one or more program instructions. These modules are accessible and executed by processing unit 2610 to perform the functions of the various embodiments described herein.

[0388] In an example embodiment of performing video encoding, input device 2650 may receive video data as input 2670 to be encoded. The video data may be processed, for example, by video codec module 2625 to generate an encoded bitstream. The encoded bitstream may be provided as output via output device 2660.

[0389] In an example embodiment of performing video decoding, input device 2650 may receive an encoded bitstream as input 2670. The encoded bitstream may be processed, for example, by video codec module 2625 to generate decoded video data. The decoded video data may be provided as output 2680 via output device 2660.

[0390] While this disclosure has been specifically shown and described with reference to preferred embodiments, those skilled in the art will understand that various changes in form and detail may be made without departing from the spirit and scope of this application as defined by the appended claims. These variations are intended to be covered by the scope of this application. Therefore, the foregoing description of embodiments of this application is not intended to be limiting.

Claims

1. A video processing method, comprising: For the conversion between video units and the bitstream of the video, determine the affine information used in the video units, wherein the video units are not blocks encoded or decoded by affine Merge or blocks encoded or decoded by affine Advanced Motion Vector Prediction (AMVP). The affine information stored in the video unit for use; as well as The transformation is performed based on the stored affine information.

2. The method of claim 1, wherein the video unit is encoded and decoded in geometric segmentation mode (GPM).

3. The method according to claim 2, further comprising: GPM predictions for the video unit are generated by applying affine motion compensation.

4. The method of claim 1, wherein the video unit is encoded and decoded using an intra-frame / inter-frame joint prediction (CIIP) mode.

5. The method according to claim 4, further comprising: CIIP predictions for the video unit are generated by applying affine motion compensation.

6. The method of claim 1, wherein the video unit is encoded and decoded using a multiple hypothesis prediction (MHP) mode.

7. The method according to claim 6, further comprising: MHP predictions for the video unit are generated by applying affine motion compensation.

8. The method of claim 1, wherein the stored affine information is utilized by at least one block that is encoded / decoded after the video unit.

9. The method of claim 1, wherein the affine information comprises at least one of the following: Inter-frame direction, One or more control point motion vectors (CPMV). One or more affine parameters, Reference index of list 0 Reference index for List 1 Local Lighting Compensation (LIC) sign, Overlapping Block Motion Compensation (OBMC) flag, Using bidirectional prediction (BCW) indexes with weights at the codec unit level, Affine type, Merge type, or Image index.

10. The method of claim 1, wherein the manner in which the affine information is stored after encoding and decoding the GPM-encoded block depends on the segmentation method of the GPM.

11. The method of claim 10, wherein if the sub-block belongs to a first geometric segment with affine motion compensation having a first set of affine information, the first set of affine information is stored in the sub-block.

12. The method of claim 11, wherein the determination of whether the sub-block belongs to the first geometric segmentation depends on the segmentation method of the GPM.

13. The method of claim 12, wherein the determination of whether the sub-block belongs to the first geometric segmentation depends on the sample point position of the sub-block.

14. The method of claim 13, wherein if the sample point is located in the first geometric segment, the sub-block is determined to belong to the first geometric segment.

15. The method of claim 13, wherein the sample point position is at least one of the following of the sub-block: upper left position, upper right position, lower left position, or lower right position.

16. The method of claim 13, wherein the sample point location is the center of the sub-block.

17. The method according to claim 13, wherein the sample point position is (x, y) or (x+W-1, y), or (x, y+H-1), or (x+W-1, y+H-1) or (x+W / 2, y+H / 2) or (x+W / 2-1, y+H / 2) or (x+W / 2, y+H / 2-1) or (x+W / 2-1, y+H / 2-1), where W represents the width of the sub-block, H represents the height of the sub-block, and the upper left position of the sub-block with dimension W×H is (x, y).

18. The method of claim 1, wherein the identifier for the segmentation of the first sub-block in the GPM-encoded block is stored for the first sub-block.

19. The method of claim 18, wherein if the sub-block belongs to a geometric segment with segmentation index K, then K is stored in the sub-block, where K is an integer.

20. The method of claim 19, wherein K is 0 or 1.

21. The method of claim 19, wherein the determination of whether the sub-block belongs to the first geometric segmentation depends on the segmentation method of the GPM.

22. The method of claim 19, wherein the determination of whether the sub-block belongs to the first geometric segment depends on the sample point position of the sub-block.

23. The method of claim 22, wherein if the sample point is located in the first geometric segment, the sub-block is determined to belong to the first geometric segment.

24. The method of claim 22, wherein the sample point position is at least one of the following of the sub-block: upper left position, upper right position, lower left position, or lower right position.

25. The method of claim 22, wherein the sample point location is the center of the sub-block.

26. The method according to claim 22, wherein the sample point position is (x, y) or (x+W-1, y), or (x, y+H-1), or (x+W-1, y+H-1) or (x+W / 2, y+H / 2) or (x+W / 2-1, y+H / 2) or (x+W / 2, y+H / 2-1) or (x+W / 2-1, y+H / 2-1), where W represents the width of the sub-block, H represents the height of the sub-block, and the upper left position of the sub-block with dimension W×H is (x, y).

27. The method of claim 1, wherein the information specified for the segmentation in the GPM is stored for blocks encoded and decoded by the GPM.

28. The method of claim 27, wherein the stored information is classified by a GPM segmentation index.

29. The method of claim 27, wherein the information comprises at least one of the following: The segmentation encoding / decoding mode, The segmented motion information, or The segmented affine information.

30. The method of claim 29, wherein the segmented encoding / decoding mode comprises at least one of the following: IntraFlag[K] determines whether the segmentation was encoded / decoded using intra-frame prediction, where K is the GPM segmentation index, or AffineFlag[K] determines whether the segmentation is encoded or decoded using affine motion compensation, where K is the GPM segmentation index.

31. The method of claim 29, wherein the segmented affine information comprises at least one of the following: CPMV[K][List][CpmvIdx] determines CPMV, where K is the GPM split index, List is 0 or 1, and CpmvIdx is 0, 1, or 2. interDir[K] determines the inter-frame direction, where K is the GPM segmentation index. refIdx[K][List] determines the reference indexes for List 0 and List 1, where K is the GPM partition index. AffineType[K] determines the affine type. LICFlag[K] determines whether to use Local Illumination Compensation (LIC). OBMCFlag[K] determines whether to use Overlapping Block Motion Compensation (OBMC), or BCWIdx[K] determines the bidirectional prediction (BCW) index using codec unit-level weights.

32. The method of claim 31, wherein the inter-frame direction includes at least one of the following: list 0, list 1, or bidirectional prediction.

33. The method of claim 1, wherein whether a sub-block is encoded or decoded using affine motion compensation is determined by a segmentation index stored in the sub-block and encoding / decoding mode information stored for the segmentation of the block encoded / decoded by GPM.

34. The method of claim 33, wherein if the segmentation index K is stored in the sub-block, the syntax elements stored in the GPM-encoded block determine whether the sub-block is encoded or decoded using affine motion compensation.

35. The method of claim 34, wherein the syntax element is AffineFlag[ K ].

36. The method of claim 1, wherein the affine information of a sub-block in a GPM-encoded block is determined by a segmentation index stored in the sub-block and affine information stored for the segmentation of the GPM-encoded block.

37. The method of claim 36, wherein if the segmentation index K is stored in the sub-block, the affine motion information of the sub-block is determined by at least one of the following stored in the GPM-encoded block: CPMV[K][List][CpmvIdx], interDir[K], refIdx[K][List], or AffineType[K].

38. The method of claim 1, wherein the affine information used in the video unit encoded / decoded using GPM is used by one or more blocks encoded / decoded after the video unit.

39. The method of claim 38, wherein the affine information of the sub-blocks in the GPM-encoded block is placed in a history parameter table (HPT).

40. The method of claim 39, wherein if at least one GPM segment of the block is encoded / decoded using affine motion compensation, the HPT is updated after encoding / decoding the GPM-encoded block.

41. The method of claim 38, wherein the affine information of the sub-blocks in the GPM-encoded block is obtained and placed into at least one of: an affine Merge candidate list or an AMVP candidate list of blocks subsequently encoded or decoded by the video unit.

42. The method of claim 38, wherein the affine information of the sub-blocks in the GPM-encoded block is obtained and placed into the affine candidate list of the second block encoded using the GPM mode.

43. The method of claim 1, wherein a block or sub-block is examined to determine whether it is encoded or decoded using GPM mode, and the block or sub-block belongs to a segment encoded / decoded using affine motion compensation.

44. The method of claim 43, wherein affine information of the block or sub-block that is segmented by affine motion compensation encoding / decoding of the block encoded / decoded using GPM is used to derive an affine-GPM inheritance candidate for at least one of: a sub-block merge list, an affine merge list, an affine AMVP list, or an affine list for GPM.

45. The method of claim 1, wherein when determining whether to add an affine-GPM inheritance candidate to the candidate list, the affine-GPM inheritance candidate is compared with at least one candidate already in the candidate list.

46. ​​The method of claim 45, wherein if the affine-GPM inheritance candidate is the same as the at least one candidate already in the candidate list, then the affine-GPM inheritance candidate is not placed in the candidate list.

47. The method of claim 45, wherein if the affine-GPM inheritance candidate is similar to the at least one candidate already in the candidate list, then the affine-GPM inheritance candidate is not placed in the candidate list.

48. The method of claim 1, wherein when constructing an affine list including at least one affine candidate, specific sub-blocks or blocks covering specific locations adjacent or not adjacent to the current block are examined in a specific order using more than one pattern.

49. The method of claim 48, wherein the affine list is one of the following: a sub-block merge list, an affine merge list, an affine AMVP list, or an affine list for GPM.

50. The method of claim 48, wherein whether the particular sub-block or block is affine-coded is checked first.

51. The method of claim 50, wherein if the particular sub-block or block is affine-coded, the affine information stored in or associated with the particular sub-block or block is used to generate affine candidates for the affine list.

52. The method of claim 48, wherein whether the particular sub-block or block is GPM-affine encoded or decoded is subsequently checked.

53. The method of claim 52, wherein whether the particular sub-block or block belongs to a GPM segment encoded and decoded using affine prediction is subsequently checked.

54. The method of claim 53, wherein if the particular sub-block or block is GPM-affine encoded or decoded, the affine information stored in or associated with the particular sub-block or block is used to generate affine candidates for the affine list.

55. The method of claim 1, wherein when constructing an affine list including at least one affine candidate, at a specific step in the list construction, a first sub-block or block covering a specific location adjacent or not adjacent to the current block is examined to generate a candidate from the block or sub-block encoded and decoded by GPM.

56. The method of claim 55, wherein the affine list is one of the following: a sub-block merge list, an affine merge list, an affine AMVP list, or an affine list for GPM.

57. The method of claim 55, wherein after checking whether a specific inherited affine candidate from a neighboring block has been inserted into the affine list, the first sub-block or block is immediately checked to generate a candidate from the block or sub-block encoded by GPM-affine.

58. The method of claim 55, wherein after checking whether a specific inherited affine candidate from a non-adjacent block is inserted into the affine list, the first sub-block or block is immediately checked to generate a candidate from the block or sub-block encoded by GPM-affine.

59. The method of claim 55, wherein after checking whether a particular regression-based motion vector field (RMVF) affine candidate is inserted into the affine list, the first sub-block or block is immediately checked to generate candidates from the block or sub-block encoded by GPM-affine coding.

60. The method of claim 55, wherein after checking whether the constructed affine candidate is inserted into the affine list, the first sub-block or block is immediately checked to generate a candidate from the block or sub-block encoded and decoded by GPM-affine.

61. The method of claim 55, wherein after checking whether a temporal inheritance affine candidate is inserted into the affine list, the first sub-block or block is immediately checked to generate a candidate from the block or sub-block encoded by GPM-affine.

62. The method of claim 55, wherein after checking whether a history-based affine candidate has been inserted into the affine list, the first sub-block or block is immediately checked to generate a candidate from the block or sub-block encoded by GPM-affine.

63. The method of claim 1, wherein whether to store and / or use affine information, and / or the manner in which to store and / or use affine information, depends on the encoding / decoding information.

64. The method of claim 1, wherein the manner in which affine information is stored and / or used depends on whether the current block is at the boundary of a codec tree unit (CTU).

65. The method according to any one of claims 1-64, wherein whether and / or how the method is applied is determined based on one or more syntactic elements.

66. The method of claim 65, wherein whether and / or how affine information is stored and / or used is transmitted from the encoder to the decoder via a signal.

67. The method of claim 65, wherein at least one syntax element is transmitted in the bit stream via a signal.

68. The method of claim 65, wherein whether and / or how the method is applied is transmitted via a signal in one of the following ways: sequence level, Image group level, Image quality, strip level, or Film series level.

69. The method of claim 65, wherein whether and / or how the method is applied is transmitted via a signal in one of the following ways: Sequence header, Image header, Sequence Parameter Set (SPS) Video Parameter Set (VPS) Dependency Parameter Set (DPS) Decoding Capability Information (DCI) Image Parameter Set (PPS) Adaptive Parameter Set (APS) strip head, or The beginning of the film.

70. The method of claim 65, wherein whether and / or how the method is applied is transmitted via a signal at one of the following locations: Predicted blocks (PB). Transform block (TB) Code block (CB) Prediction Unit (PU) Transformer Unit (TU) Codec Unit (CU) Code-decode tree block (CTB). Code-decode tree unit (CTU) CTU line, strip, piece, Sub-images, or This includes regions containing more than one sample point or pixel.

71. The method of claim 65, wherein the method is applied based on the encoded / decoded information, the encoded / decoded information comprising at least one of the following: Block size, Color format, Single and / or dual tree partitioning Color components, Strip type, or Image type.

72. The method of claim 65, wherein whether a syntax element is determined based on another syntax element via signal transmission.

73. The method according to any one of claims 1-72, wherein the syntax element is binarized into one of the following: a flag, a fixed-length code, an EG(x) code, a unary code, a rounded unary code, or a rounded binary code.

74. The method of claim 73, wherein the syntax element is signed or unsigned.

75. The method according to any one of claims 1-72, wherein the syntax elements are encoded and decoded using at least one context model, or The syntax elements are bypassed and decoded.

76. The method according to any one of claims 1-75, wherein the syntax element is transmitted via signal in a conditional manner.

77. The method of claim 76, wherein if the corresponding function is applicable, the syntax element is transmitted via a signal, or If the dimension of the video unit meets the condition, the syntax element is transmitted via signal.

78. The method of claim 77, wherein the dimension includes the width and / or height of the video unit.

79. The method according to any one of claims 1-78, wherein the syntax element is transmitted by signal at one of the following locations: sequence level, Image group level, Image quality, strip level, or Film series level.

80. The method according to any one of claims 1-78, wherein the syntax element is transmitted by a signal at one of the following locations: Sequence header, Image header, Sequence Parameter Set (SPS) Video Parameter Set (VPS) Dependency Parameter Set (DPS) Decoding Capability Information (DCI) Image Parameter Set (PPS) Adaptive Parameter Set (APS) strip head, or The beginning of the film.

81. The method according to any one of claims 1-78, wherein the syntax element is transmitted by signal at one of the following locations: Predicted blocks (PB). Transform block (TB) Code block (CB) Prediction Unit (PU) Transformer Unit (TU) Codec Unit (CU) Code-decode tree block (CTB). Code-decode tree unit (CTU) CTU line, strip, piece, Sub-images, or This includes regions containing more than one sample point or pixel.

82. The method according to any one of claims 1-81, wherein whether and / or how the affine information is stored is transmitted by signal at one of the following locations: sequence level, Image group level, Image quality, strip level, or Film series level.

83. The method according to any one of claims 1-81, wherein whether and / or how the affine information is stored is transmitted by a signal at one of the following locations: Sequence header, Image header, Sequence Parameter Set (SPS) Video Parameter Set (VPS) Dependency Parameter Set (DPS) Decoding Capability Information (DCI) Image Parameter Set (PPS) Adaptive Parameter Set (APS) strip head, or The beginning of the film.

84. The method according to any one of claims 1-81, wherein whether and / or how the affine information is stored is transmitted by signal at one of the following locations: Predicted blocks (PB). Transform block (TB) Code block (CB) Prediction Unit (PU) Transformer Unit (TU) Codec Unit (CU) Code-decode tree block (CTB). Code-decode tree unit (CTU) CTU line, strip, piece, Sub-images, or This includes regions containing more than one sample point or pixel.

85. The method according to any one of claims 1-81, further comprising: Based on the encoded and decoded information of the video unit, determine whether and / or how to store the affine information, wherein the encoded and decoded information includes at least one of the following: Block size, Color format, Single and / or dual tree partitioning Color components, Strip type, or Image type.

86. The method according to any one of claims 1-85, wherein the video unit is encoded or decoded using one or more other encoding / decoding tools that require chroma blending.

87. The method according to any one of claims 1-86, wherein the conversion comprises encoding the video unit into the bitstream.

88. The method according to any one of claims 1-86, wherein the conversion comprises decoding the video unit from the bitstream.

89. An apparatus for video processing, comprising a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to perform the method according to any one of claims 1-88.

90. A non-transitory computer-readable storage medium storing instructions that cause a processor to execute the method according to any one of claims 1-88.

91. A non-transitory computer-readable recording medium storing a bitstream of video generated by a method performed by means of a video processing apparatus, wherein the method comprises: Determine the affine information used in video units of the video, wherein the video units are not blocks encoded by affine Merge or blocks encoded by affine Advanced Motion Vector Prediction (AMVP). The affine information stored in the video unit for use; as well as The bit stream is generated based on the stored affine information.

92. A method for storing a bitstream of video, comprising: Determine the affine information used in video units of the video, wherein the video units are not blocks encoded by affine Merge or blocks encoded by affine Advanced Motion Vector Prediction (AMVP). The affine information stored in the video unit for use; The bit stream is generated based on the stored affine information; as well as The bitstream is stored in a non-transitory computer-readable recording medium.

Images