Method, apparatus and medium for video processing

Abstract

The embodiments of the present disclosure provide a solution for video processing. A method for video processing is proposed. The method includes: during conversion between a video unit of a video and a bitstream of the video unit, deriving a first motion candidate for the video unit based on a first position of a first block of the video, and deriving a second motion candidate for the video unit based on a second position of a second block of the video, where the first position and the second position satisfy a position condition, and performing conversion based on the first and second motion candidates.

Description

[Technical field]

[0001] FIELD OF THE DISCLOSURE Embodiments of the present disclosure relate generally to video processing techniques, and more particularly, to history-based affine model inheritance. [Background technology]

[0002] Nowadays, digital video functions are applied in various aspects of people's lives. Several kinds of video compression technologies, such as MPEG-2, MPEG-4, ITU-TH.263, ITU-TH.264 / MPEG-4 Part10 Advanced Video Coding (AVC), ITU-TH.265 high efficiency video coding (HEVC) standard, and versatile video coding (VVC) standard, have been proposed for video encoding / decoding. However, in general, the coding efficiency of video coding technologies is expected to be further improved. Summary of the Invention

[0003] SUMMARY OF THE DISCLOSURE Embodiments of the present disclosure provide a solution for video processing.

[0004] In a first aspect, a method for video processing is proposed. The method includes: deriving a first motion candidate for a video unit based on a first position of a first block of the video and a second position of a second block of the video during conversion between a video unit of a video and a bitstream of the video unit, where the first position and the second position satisfy a position condition, and performing conversion based on the first and second motion candidates. The method according to the first aspect of the present disclosure improves coding efficiency and performance by selecting the position of the block based on a specific rule, rather than checking each block.

[0005] In a second aspect, an apparatus for video processing is proposed, the apparatus comprising a processor and a non-transitory memory having instructions thereon, the instructions, when executed by the processor, causing the processor to perform a method according to the first aspect of the present disclosure.

[0006] In a third aspect, a non-transitory computer-readable storage medium is proposed. The non-transitory computer-readable storage medium stores instructions for causing a processor to perform the method according to the first aspect of the present disclosure.

[0007] In a fourth aspect, another non-transitory computer-readable recording medium is proposed, which stores a bitstream of a video generated by a method executed by an apparatus for video processing, the method including: deriving a first motion candidate for a video unit of the video based on a first position of a first block of the video and a second position of a second block of the video, where the first position and the second position satisfy a position condition; and generating a bitstream based on the first motion candidate and the second motion candidate.

[0008] In a fifth aspect, a method for storing a bitstream of a video is proposed, the method including: deriving a first motion candidate for a video unit of the video based on a first position of a first block of the video and a second position of a second block of the video, where the first position and the second position satisfy a position condition; generating a bitstream based on the first motion candidate and the second motion candidate; and storing the bitstream in a non-transitory computer-readable recording medium.

[0009] This Summary is provided to introduce in a simplified form some concepts that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. [Brief description of the drawings]

[0010] The above and other objects, features, and advantages of exemplary embodiments of the present disclosure will become more apparent through the following detailed description taken in conjunction with the accompanying drawings, in which like reference numerals generally refer to like components.

[0011] [Figure 1] FIG. 1 shows a block diagram illustrating an example video coding system according to some embodiments of the present disclosure.

[0012] [Diagram 2] 1 shows a block diagram illustrating a first exemplary video encoder according to some embodiments of the present disclosure.

[0013] [Diagram 3] FIG. 2 shows a block diagram illustrating an exemplary video decoder according to some embodiments of the present disclosure.

[0014] [Figure 4] 1 illustrates sub-block based prediction.

[0015] [Figure 5a] A simplified affine motion model is shown, and a four-parameter affine model is shown. [Figure 5b] A simplified affine motion model is shown, with a six-parameter affine model being shown.

[0016] [Figure 6] The affine MVF for each subblock is shown.

[0017] [Figure 7a] Indicates candidates for AF_MERGE. [Figure 7b] Indicates candidates for AF_MERGE.

[0018] [Figure 8]Shows candidate positions for affine merge mode.

[0019] [Figure 9] Shows candidate positions for affine merge mode.

[0020] [Figure 10a] An example of dividing a CU into two triangular prediction units (two division patterns) is shown, illustrating a 135-degree partition type. [Figure 10b] An example of dividing a CU into two triangular prediction units (two division patterns) is shown, with a 45-degree division pattern being shown.

[0021] [Figure 11] Indicates the location of nearby blocks.

[0022] [Figure 12] 13 shows an example of a CU to which a first set of weighting coefficients is applied.

[0023] [Figure 13] 4 shows an example of storing motion vectors.

[0024] [Figure 14] 1 shows a decoding flowchart according to the proposed HMVP method.

[0025] [Figure 15] An example of updating a table in the proposed HMVP method is given below.

[0026] [Figure 16] The UMVE exploration process is shown.

[0027] [Figure 17] UMVE search points are shown.

[0028] [Figure 18] 1 shows a mapping between distance index and distance offset.

[0029] [Figure 19] An example of deriving the CPMV from the MVs of neighboring blocks and a parameter set stored in a buffer is shown below.

[0030] [Figure 20] 1 shows examples of possible positions for collocated unit blocks.

[0031] [Figure 21] Indicates the position in a 4x4 basic block.

[0032] [Figure 22] The sub-blocks at the right and bottom boundaries are shown shaded.

[0033] [Figure 23a] Possible positions for deriving the MVs stored in the sub-blocks at the right and bottom boundaries are shown. [Figure 23b] Possible positions for deriving the MVs stored in the sub-blocks at the right and bottom boundaries are shown. [Figure 23c] Possible positions for deriving the MVs stored in the sub-blocks at the right and bottom boundaries are shown. [Figure 23d] Possible positions for deriving the MVs stored in the sub-blocks at the right and bottom boundaries are shown.

[0034] [Figure 24] 1 shows possible positions for deriving the MV prediction.

[0035] [Figure 25a] 13 shows the spatial neighborhood for deriving inherited affine merge candidates. [Figure 25b] 1 shows the spatial neighborhood for deriving constructed affine merge candidates.

[0036] [Figure 26] 1 shows a schematic diagram of affine merge candidates constructed from non-adjacent neighborhoods.

[0037] [Figure 27a] 13 illustrates example block locations according to some embodiments of the present disclosure. [Figure 27b] 13 illustrates example block locations according to some embodiments of the present disclosure.

[0038] [Figure 28] 13 illustrates example block locations according to some embodiments of the present disclosure.

[0039] [Figure 29] 1 illustrates an example of a HPAC according to an exemplary embodiment of the present disclosure.

[0040] [Diagram 30] 1 shows a flowchart of a method for video processing according to an embodiment of the present disclosure.

[0041] [Diagram 31] FIG. 1 illustrates a block diagram of a computing device in which various embodiments of the present disclosure may be implemented.

[0042] Throughout the drawings, the same or similar reference numbers typically refer to the same or similar elements. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0043] Next, the principles of the present disclosure will be described with reference to some embodiments. It should be understood that these embodiments are described for illustrative purposes only, to help those skilled in the art understand and practice the present disclosure, and do not imply any limitations on the scope of the present disclosure. The disclosure described herein can be implemented in various forms other than those described below.

[0044] In the following description and claims, unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs.

[0045] References in this disclosure to "one embodiment," "embodiment," "exemplary embodiment," and the like indicate that the described embodiment may include a particular feature, structure, or characteristic, but not all embodiments need to include the particular feature, structure, or characteristic. Moreover, such phrases do not necessarily refer to the same embodiment. Furthermore, when a particular feature, structure, or characteristic is described in connection with an exemplary embodiment, it is submitted that it is within the knowledge of one of ordinary skill in the art to affect such feature, structure, or characteristic in connection with other embodiments, whether or not explicitly described.

[0046] In this specification, terms such as "first" and "second" may be used to describe various elements, but it should be understood that these elements should not be limited by these terms. These terms are only used to distinguish one element from another element. For example, a first element can be referred to as a second element, and similarly, a second element can be referred to as a first element, without departing from the scope of the exemplary embodiments. As used herein, the term "and / or" includes any and all combinations of one or more of the listed terms.

[0047] The terms used herein are for the purpose of describing particular embodiments only and are not intended to limit the exemplary embodiments. As used herein, the singular forms "a," "an," and "the" are intended to include the plural forms unless the context clearly indicates otherwise. It is further understood that as used herein, the terms "comprises," "comprising," "has," "having," "includes," and / or "including" specify the presence of stated features, elements, and / or components, etc., but do not exclude the presence or addition of one or more other features, elements, components, and / or combinations thereof.

[0048] Example Environment 1 is a block diagram illustrating an example video coding system 100 that may utilize techniques of this disclosure. As shown, video coding system 100 may include a source device 110 and a destination device 120. Source device 110 may also be referred to as a video encoding device, and destination device 120 may also be referred to as a video decoding device. In operation, source device 110 may be configured to generate encoded video data, and destination device 120 may be configured to decode the encoded video data generated by source device 110. Source device 110 may include a video source 112, a video encoder 114, and an input / output (I / O) interface 116.

[0049] Video source 112 may include sources such as video capture devices, examples of which include, but are not limited to, an interface for receiving video data from a video content provider, a computer graphics system for generating video data, and / or combinations thereof.

[0050] The video data may include one or more pictures. The video encoder 114 encodes the video data from the video source 112 to generate a bitstream. The bitstream may include a bit sequence that forms a coded representation of the video data. The bitstream may include coded pictures and associated data. A coded picture is a coded representation of a picture. The associated data may include a sequence parameter set, a picture parameter set, and other syntax structures. The I / O interface 116 may include a modulator / demodulator and / or a transmitter. The coded video data may be transmitted directly to the destination device 120 via the I / O interface 116 through the network 130A. The coded video data may also be stored in a storage medium / server 130B for access by the destination device 120.

[0051] Destination device 120 may include an I / O interface 126, a video decoder 124, and a display device 122. I / O interface 126 may include a receiver and / or a modem. I / O interface 126 may obtain encoded video data from source device 110 or storage medium / server 130B. Video decoder 124 may decode the encoded video data. Display device 122 may display the decoded video data to a user. Display device 122 may be integrated with destination device 120 or may be external to destination device 120 configured to interface with an external display device.

[0052] The video encoder 114 and the video decoder 124 may operate according to a video compression standard, such as the High Efficiency Video Coding (HEVC) standard, the Versatile Video Coding (VVC) standard, and other current and / or future standards.

[0053] FIG. 2 is a block diagram illustrating an example of a video encoder 200, which may be an example of the video encoder 114 in the system 100 shown in FIG. 1, according to some embodiments of the disclosure.

[0054] Video encoder 200 may be configured to implement any or all of the techniques of this disclosure. In the example of FIG. 2, video encoder 200 includes multiple functional components. The techniques described in this disclosure may be shared among various components of video encoder 200. In some examples, a processor may be configured to perform any or all of the techniques described in this disclosure.

[0055] In some embodiments, the video encoder 200 may include a division unit 201, a prediction unit 202, which may include a mode selection unit 203, a motion estimation unit 204, a motion compensation unit 205, and an intra prediction unit 206, a residual generation unit 207, a transform unit 208, a quantization unit 209, an inverse quantization unit 210, an inverse transform unit 211, a reconstruction unit 212, a buffer 213, and an entropy coding unit 214.

[0056] In other examples, video encoder 200 may include more, fewer, or different functional components. In one example, prediction unit 202 may include an intra block copy (IBC) unit. The IBC unit may perform prediction in an IBC mode, where at least one reference picture is the picture in which the current video block is located.

[0057] Furthermore, some components, such as the motion estimation unit 204 and the motion compensation unit 205, may be integrated, but are depicted separately in the example of FIG. 2 for ease of explanation.

[0058] Division unit 201 may divide a picture into one or more video blocks. Video encoder 200 and video decoder 300 may support a variety of video block sizes.

[0059] The mode selection unit 203 may select one of the coding modes (intra or inter), for example based on the error result, and provide the generated intra-coded or inter-coded blocks to the residual generation unit 207 to generate residual block data and to the reconstruction unit 212 to reconstruct coded blocks for use as reference pictures. In some examples, the mode selection unit 203 may select a combination of intra and inter prediction (CIIP) mode, in which prediction is performed based on an inter prediction signal and an intra prediction signal. The mode selection unit 203 may also select a resolution of the motion vectors of the blocks (e.g., sub-pixel accuracy or integer pixel accuracy) in the case of inter prediction.

[0060] To perform inter prediction on the current video block, motion estimation unit 204 may generate motion information for the current video block by comparing one or more reference frames from buffer 213 to the current video block. Motion compensation unit 205 may determine a predicted video block for the current video block based on the motion information and decoded samples of pictures from buffer 213 other than the picture associated with the current video block.

[0061] Motion estimation unit 204 and motion compensation unit 205 may perform different operations on a current video block depending on whether the current video block is in an I slice, a P slice, or a B slice, for example. As used herein, an "I slice" may refer to a portion of a picture that is composed of macroblocks, all of which are based on macroblocks in the same picture. Additionally, as used herein, in some aspects, a "P slice" and a "B slice" may refer to a portion of a picture that is composed of macroblocks that are independent of macroblocks in the same picture.

[0062] In some examples, motion estimation unit 204 may perform unidirectional prediction on the current video block, and motion estimation unit 204 may search reference pictures of list 0 or list 1 for reference video blocks for the current video block. Motion estimation unit 204 may then generate a reference index indicating a reference picture in list 0 or list 1 that contains the reference video block, and a motion vector indicating a spatial displacement between the current video block and the reference video block. Motion estimation unit 204 may output the reference index, the prediction direction indicator, and the motion vector as motion information for the current video block. Motion compensation unit 205 may generate a predicted video block for the current video block based on the reference video block indicated by the motion information of the current video block.

[0063] Alternatively, in other examples, motion estimation unit 204 may perform bidirectional prediction for the current video block. Motion estimation unit 204 may search reference pictures in list 0 for a reference video block for the current video block and also search reference pictures in list 1 for another reference video block for the current video block. Motion estimation unit 204 may then generate reference indexes indicating reference pictures in list 0 and list 1 that contain reference video blocks and motion vectors indicating spatial displacements between the reference video blocks and the current video block. Motion estimation unit 204 may output the reference index and the motion vector for the current video block as motion information for the current video block. Motion compensation unit 205 may generate a predicted video block for the current video block based on the reference video block indicated by the motion information of the current video block.

[0064] In some examples, motion estimation unit 204 may output a full set of motion information for a decoder's decoding process. Alternatively, in some embodiments, motion estimation unit 204 may signal the motion information of the current video block by reference to motion information of another video block. For example, motion estimation unit 204 may determine that the motion information of the current video block is sufficiently similar to the motion information of a neighboring video block.

[0065] In one example, motion estimation unit 204 may indicate, in a syntax structure associated with the current video block, a value that indicates to video decoder 300 that the current video block has the same motion information as another video block.

[0066] In another example, motion estimation unit 204 may identify another video block and a motion vector difference (MVD) in a syntax structure associated with the current video block. The motion vector difference indicates a difference between the motion vector of the current video block and the motion vector of the pointed-to video block. Video decoder 300 may use the motion vector of the pointed-to video block and the motion vector difference to determine the motion vector of the current video block.

[0067] As mentioned above, video encoder 200 may predictively signal motion vectors. Two examples of predictive signaling techniques that may be implemented by video encoder 200 include advanced motion vector prediction (AMVP) and merge mode signaling.

[0068] Intra prediction unit 206 may perform intra prediction on the current video block. When intra prediction unit 206 performs intra prediction on the current video block, it may generate predictive data for the current video block based on decoded samples of other video blocks in the same picture. The predictive data for the current video block may include a predicted video block and various syntax elements.

[0069] Residual generation unit 207 may generate residual data for the current video block by subtracting (e.g., as indicated by a minus sign) the predicted video block(s) of the current video block from the current video block. The residual data for the current video block may include residual video blocks that correspond to different sample components of the samples in the current video block.

[0070] In other examples, for the current video block, such as in skip mode, residual data for the current video block may not be present and residual generation unit 207 may not perform a subtraction operation.

[0071] Transform processing unit 208 may generate one or more transform coefficient video blocks for the current video block by applying one or more transforms to a residual video block related to the current video block.

[0072] After transform processing unit 208 generates a transform coefficient video block associated with the current video block, quantization unit 209 may quantize the transform coefficient video block associated with the current video block based on one or more quantization parameter (QP) values ​​associated with the current video block.

[0073] Inverse quantization unit 210 and inverse transform unit 211 may apply inverse quantization and inverse transform, respectively, to the transform coefficient video block to reconstruct a residual video block from the transform coefficient video block. Reconstruction unit 212 may add the reconstructed residual video block to corresponding samples from one or more predicted video blocks generated by prediction unit 202 to generate a reconstructed video block related to the current video block for storage in buffer 213.

[0074] After reconstruction unit 212 reconstructs the video blocks, a loop filtering operation may be performed to reduce video blocking artifacts in the video blocks.

[0075] The entropy encoding unit 214 may receive data from other functional components of the video encoder 200. Upon receiving the data, the entropy encoding unit 214 may perform one or more entropy encoding operations to generate entropy-coded data, and output a bitstream that includes the entropy-coded data.

[0076] FIG. 3 is a block diagram illustrating an example of a video decoder 300, which may be an example of the video decoder 124 in the system 100 shown in FIG. 1, according to some embodiments of the disclosure.

[0077] The video decoder 300 may be configured to perform any or all of the techniques described in this disclosure. In the example of FIG. 3, the video decoder 300 includes multiple functional components. The techniques described in this disclosure may be shared among various components of the video decoder 300. In some examples, a processor may be configured to perform any or all of the techniques described in this disclosure.

[0078] 3, the video decoder 300 includes an entropy decoding unit 301, a motion compensation unit 302, an intra prediction unit 303, an inverse quantization unit 304, an inverse transform unit 305, a reconstruction unit 306, and a buffer 307. The video decoder 300 may, in some examples, perform a decoding path that is generally the inverse of the encoding path described with respect to the video encoder 200.

[0079] The entropy decoding unit 301 may retrieve an encoded bitstream. The encoded bitstream may include entropy coded video data (e.g., coded blocks of video data). The entropy decoding unit 301 may decode the entropy coded video data, and from the entropy decoded video data, the motion compensation unit 302 may determine motion information including motion vectors, motion vector precision, reference picture list indexes, and other motion information. The motion compensation unit 302 may determine such information, for example, by performing AMVP and merge mode. AMVP includes deriving multiple most probable candidates based on data from neighboring PB and reference pictures and is used. The motion information typically includes horizontal and vertical motion vector displacement values, one or two reference picture indexes, and an identification of which reference picture list is associated with each index in case of a prediction region in a B slice. As used herein, in some aspects, "merge mode" may refer to deriving motion information from spatially or temporally neighboring blocks.

[0080] The motion compensation unit 302 generates motion compensated blocks and may optionally perform interpolation based on an interpolation filter. An identifier for the interpolation filter used with sub-pixel precision may be included in a syntax element.

[0081] The motion compensation unit 302 may calculate sub-integer pixel interpolated values ​​of the reference block using an interpolation filter used by the video encoder 200 during encoding of the video block. The motion compensation unit 302 may determine the interpolation filter used by the video encoder 200 according to received syntax information and generate the predictive block using the interpolation filter.

[0082] The motion compensation unit 302 may use at least a portion of the syntax information to determine the size of blocks used to code the frame(s) and / or slice(s) of the coded video sequence, partition information describing how each macroblock of a picture of the coded video sequence is partitioned, a mode indicating how each partition is coded, one or more reference frames (and reference frame lists) for each inter-coded block, and other information for decoding the coded video sequence. As used herein, in some aspects, a "slice" may refer to a data structure that can be decoded independently from other slices of the same picture in terms of entropy coding, signal prediction, and residual signal reconstruction. A slice may be either an entire picture or a region of a picture.

[0083] The intra prediction unit 303 may form a prediction block from spatially adjacent blocks, e.g., using an intra prediction mode received in the bitstream. The inverse quantization unit 304 inverse quantizes, or dequantizes, the quantized video block coefficients provided in the bitstream and decoded by the entropy decoding unit 301. The inverse transform unit 305 applies an inverse transform.

[0084] Reconstruction unit 306 may obtain a decoded block, for example, by summing the residual block with a corresponding prediction block generated by motion compensation unit 302 or intra prediction unit 303. If desired, a deblocking filter may be applied to filter the decoded block to remove blockiness artifacts. The decoded video block is stored in buffer 307 to provide reference blocks for subsequent motion compensation / intra prediction, and to generate decoded video for presentation on a display device.

[0085] Some exemplary embodiments of the present disclosure are described in detail below. It should be understood that section headings are used herein for ease of understanding and are not intended to limit the embodiments disclosed in a section to that section alone. Furthermore, although certain embodiments are described with reference to Versatile Video Coding or other specific video codecs, the disclosed techniques are applicable to other video coding techniques. Furthermore, although some embodiments describe video coding steps in detail, it should be understood that the corresponding step of decoding, which undoes the coding, is performed by a decoder. Furthermore, the term video processing encompasses video coding or compression, video decoding or decompression, and video transcoding, which represents video pixels from one compressed format to another compressed format or at a different compressed bit rate. 1. Overview The present disclosure relates to video / image coding techniques. In particular, the present disclosure relates to affine prediction in video / image coding. The present disclosure may be applied to existing video coding standards such as HEVC and VVC, and may also be applied to future video / image coding standards and video / image codecs. 2. Introduction Video coding standards have evolved primarily through the development of well-known ITU-T and ISO / IEC standards. ITU-T developed H.261 and H.263, ISO / IEC developed MPEG-1 and MPEG-4 Visual, and the two organizations jointly developed H.262 / MPEG-2 Video, H.264 / MPEG-4 AVC (Advanced Video Coding), and H.265 / HEVC (H.265 / HEVC, https: / / www.itu.int / rec / T-REC-H.265) standards. Video coding standards after H.262 are based on a hybrid video coding structure in which temporal prediction + transform coding is employed. In order to explore future video coding technologies beyond HEVC, VCEG and MPEG jointly established the Joint Video Exploration Team (JVET) in 2015. Since then, many new methods have been adopted by JVET and put into reference software named JEM (Joint Exploration Model) (JEM-7.0: https: / / jvet.hhi.fraunhofer.de / svn / svn_HMJEMSoftware / tags / HM-16.6-JEM-7.0) (VTM-2.0.1: https: / / vcgit.hhi.fraunhofer.de / jvet / VVCSoftware_VTM / tags / VTM-2.0.1.) In April 2018, JVET (Joint Video Expert Team) was established between VCEG (Q6 / 16) and ISO / IEC JTC1 SC29 / WG11 (MPEG) to work on the VVC standard, which targets a 50% bitrate reduction compared to HEVC. The latest version of the VVC draft, Versatile Video Coding (Draft 2), can be found at: http: / / phenix.it-sudparis.eu / jvet / doc_end_user / documents / 11_Ljubljana / wg11 / JVET-K1001-v7.zip. The latest reference software for VVC, named VTM, can be found at: https: / / vcgit.hhi.fraunhofer.de / jvet / VVCSoftware_VTM / tags / VTM-2.1. Sub-block-based prediction was first introduced into video coding standards by HEVC Annex I (3D-HEVC) (H.265 / HEVC, https: / / www.itu.int / rec / T-REC-H.265). In sub-block-based prediction, a block such as a coding unit (CU) or a prediction unit (PU) is divided into several non-overlapping sub-blocks. Different sub-blocks are assigned different motion information such as reference indexes or motion vectors (MV), and motion compensation (MC) is performed for each sub-block separately. Figure 4 shows the concept of sub-block-based prediction. To explore future video coding technologies beyond HEVC, VCEG and MPEG jointly established the Joint Video Exploration Team (JVET) in 2015. Since then, many new methods (J. Chen, E. Alshina, GJ Sullivan, J.-R. Ohm, J. Boyce, “Algorithm description of Joint Exploration Test Model 7 (JEM7) ,” JVET-G1001, Aug. 2017) have been adopted by JVET and put into reference software named Joint Exploration Model (JEM) (JEM-7.0: https: / / jvet.hhi.fraunhofer.de / svn / svn_HMJEMSoftware / tags / HM-16.6-JEM-7.0). In JEM, sub-block-based prediction is adopted in several coding tools, such as affine prediction, ATMVP (Alternative temporal motion vector prediction), STMVP (Spatial-Temporal motion vector prediction), BIO (Bi-directional Optical flow), and FRUC (Frame-Rate Up Conversion). Affine prediction is also adopted in VVC. 2.1 Affine Prediction In HEVC, only a translational motion model is applied for motion compensation prediction (MCP). Meanwhile, in the real world, various motions exist, such as zoom-in / zoom-out, rotation, perspective motion, and other irregular motions. In VVC, a simplified affine transformation motion compensation prediction is applied. As shown in Fig. 5a-5b, the affine motion field of a block is described by two (for the four-parameter affine model) or three (for the six-parameter affine model) control point motion vectors. The motion vector field (MVF) of a block is described by the four-parameter affine model in equation (1) (where the four parameters are defined as the variables a, b, e, f) and the six-parameter affine model in equation (2) (where the four parameters are defined as the variables a, b, c, d, e, f), respectively, by the following equations:

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

[0086] As used herein, the terms "video unit" or "coding unit" or "block" may refer to one or more of the following: a color component, a sub-picture, a slice, a tile, a coding tree unit (CTU), a CTU row, a CTU group, a coding unit (CU), a prediction unit (PU), a transform unit (TU), a coding tree block (CTB), a coding block (CB), a prediction block (PB), a transform block (TB), a block, a sub-block of a block, a sub-region within a block, or a region containing two or more samples or pixels.

[0087] In this disclosure, for a "mode N coded block," the term "mode N" may be a prediction mode (e.g., MODE_INTRA, MODE_INTER, MODE_PLT, MODE_IBC, etc.) or a coding technique (e.g., AMVP, Merge, SMVD, BDOF, PROF, DMVR, AMVR, TM, Affine, CIIP, GPM, MMVD, BCW, HMVP, SbTMVP, etc.).

[0088] Please note that the terms described below are not limited to those specifically defined in existing standards, and any variation of coding tools is also applicable.

[0089] 30 shows a flowchart of a method 3000 for video processing according to an embodiment of the present disclosure. The method 3000 is performed during conversion between video units of a video and a bitstream of a video.

[0090] In block 3010, a first motion candidate is derived for the video unit based on a first position of a first block of the video and a second position of a second block of the video during conversion between the video unit of the video and the bitstream of the video unit. In some embodiments, the first motion candidate is one of a non-adjacent affine inheritance candidate or a non-adjacent affine construction candidate. In some embodiments, the first position is one of a top-left position or a center position of the first block of the video. Alternatively, the first position may be any other position.

[0091] The first position and the second position satisfy a position condition. In some embodiments, the second motion candidate is one of a non-adjacent affine inheritance candidate or a non-adjacent affine construction candidate. In some embodiments, the second position is one of a top-left position or a center position of the second block of the video. Alternatively, the second position may be any other position.

[0092] At block 3030, a transformation is performed based on the first and second motion candidates. In some embodiments, the transformation may include encoding the video unit into a bitstream. Alternatively or additionally, the transformation may include decoding the video unit from the bitstream.

[0093] According to the method 3000, instead of checking each block, the positions of blocks used to derive motion candidates can be selected based on certain rules, which can improve coding efficiency and performance.

[0094] In some embodiments, the first position is represented as (x0, y0) and the second position is represented as (x1, y1). For example, as shown in Figures 27a and 27b, the first position 110 of the first block 1110 is represented as (x0, y0) and the second position 120 of the second block 1120 is represented as (x1, y1).

[0095] In some embodiments, as shown in FIG. 27a, the position condition is y1=y0 and x1-x0=2 k Alternatively or additionally, as shown in Fig. 27b, the position condition may include x1 = x0 and y1 - y0 = 2k, where k is an integer not less than 0, x0 is not greater than x1, and y0 is not greater than y1.

[0096] In some other embodiments, the position conditions are y1=y0 and x1-x0=W*2 k Alternatively or additionally, the position condition may include x1=x0 and y1-y0=H*2 k where k is an integer not less than 0, M represents the width of the current block in video units, H represents the height of the current block, x0 is not greater than x1, and y0 is not greater than y1.

[0097] In some embodiments, a first motion candidate for the video unit is derived based on a third position of a third block of the video, where in some embodiments the first position is represented as (x0, y0), the second position is represented as (x1, y1), and the third position is represented as (x2, y2).

[0098] In some embodiments, the positional conditions are y1=y0 and x1-x0=2. k Alternatively or additionally, the position condition may include x2=x0 and y2-y0=2. m In this case, k is an integer not less than 0, m is an integer not less than 0, x0 is not greater than x1 or x2, and y0 is not greater than y1 or y2.

[0099] In some other embodiments, the positional conditions are: y1=y0 and x1-x0=W*2 k Alternatively or additionally, the position condition may include x2=x0 and y2-y0=H*2 mwhere k is an integer not less than 0, m is an integer not less than 0, M represents the width of the current block in video units, H represents the height of the current block, x0 is not greater than x1 or x2, and y0 is not greater than y1 or y2.

[0100] In some embodiments, the first block is a current block for the video unit and the second block is a block to the lower left of the current block. The first position may be a top-left position of the current block and is represented as (x, y). The second position is a bottom-left position of the bottom-left block and is represented as (xLB, yLB).

[0101] In some embodiments, the second position is derived as follows: yLB=y+H-1, and xLB=x+WW*2. k In this case, k is an integer, H represents the height of the current block, and W represents the width of the current block. In some embodiments, as shown in FIG. 28, when k is equal to 1, xLB is equal to (xW), and when k is equal to 2, xLB is equal to (x-3*W).

[0102] In some embodiments, the first block is a current block for the video unit and the second block is a block to the upper right of the current block, the first position may be a top-left position of the current block and is represented as (x, y), and the second position may be a top-right position of the bottom-left block and is represented as (xRT, yRT).

[0103] In some embodiments, the second position may be derived as follows: xRT=x+W-1, and yRT=y+HH*2. k In this case, k is an integer, H represents the height of the current block, and W represents the width of the current block. In some embodiments, as shown in FIG. 28, when k is equal to 1, yRT is equal to (yH), and when k is equal to 2, yRT is equal to (y-3*H).

[0104] In some embodiments, the second block is the bottom-left block or the top-right block. In this case, the second block is shifted to an adjacent block. In one example, the bottom-left block and / or the top-right block used to derive the non-adjacent affine construction candidates may be shifted to an adjacent block, such as LB1 / LB2 / RT1 / RT2 shown in FIG. 28.

[0105] In some embodiments, the top left block used to derive the motion candidates is positioned by at least one of the bottom left block or the top right block. In one example, the top left block used to derive the non-adjacent affine construction candidates may be positioned by the bottom left block and / or the top right block used to derive the non-adjacent affine construction candidates, such as the dashed line LT shown in FIG.

[0106] According to a further embodiment of the present disclosure, a non-transitory computer-readable recording medium is provided, which stores a bitstream of a video generated by a method performed by an apparatus for video processing, the method including deriving a first motion candidate for a video unit of the video based on a first position of a first block of the video, deriving a second motion candidate for the video unit based on a second position of a second block of the video, where the first position and the second position satisfy a position condition, and generating a bitstream based on the first and second motion candidates.

[0107] According to yet another embodiment of the present disclosure, a method for storing a bitstream of a video is provided, the method including: deriving a first motion candidate for a video unit of the video based on a first position of a first block of the video and deriving a second motion candidate for the video unit based on a second position of a second block of the video, where the first position and the second position satisfy a position condition, generating a bitstream based on the first and second motion candidates, and storing the bitstream in a non-transitory computer-readable recording medium.

[0108] Implementations of the present disclosure can be described in terms of the following clauses, the features of which can be combined in any reasonable manner.

[0109] Clause 1. A method for video processing, comprising: during conversion between a video unit of a video and a bitstream of the video unit, deriving a first motion candidate for the video unit based on a first position of a first block and a second position of a second block of the video, wherein the first position and the second position satisfy a position condition; and performing the conversion based on the first motion candidate and the second motion candidate.

[0110] Clause 2. The method of clause 1, wherein the first motion candidate is one of a non-adjacent affine inheritance candidate or a non-adjacent affine construction candidate.

[0111] Item 3. The first position is represented as (x0, y0), the second position is represented as (x1, y1), and the position condition includes at least one of the following: y1=y0 and x1-x0=2. k Or x1=x0 and y1-y0=2 k ,where k is an integer not less than 0, x0 is not greater than x1, and y0 is not greater than y1.

[0112] Item 4. The first position is represented as (x0, y0), the second position is represented as (x1, y1), and the position condition includes at least one of the following: y1=y0 and x1-x0=W*2 k Or x1 = x0 and y1-y0 = H*2 k 13. The method of claim 1, wherein k is an integer not less than 0, M represents a width of a current block for the video unit, H represents a height of the current block, x0 is not greater than x1, and y0 is not greater than y1.

[0113] Clause 5. The method of clause 1, further comprising deriving the first motion candidate for the video unit based on a third position of a third block of the video.

[0114] Clause 6. The first position is represented as (x0, y0), the second position is represented as (x1, y1), and the third position is represented as (x2, y2), and the position condition includes at least one of the following: y1=y0 and x1-x0=2. k Or x2=x0 and y2-y0=2 m 6. The method of claim 5, wherein k is an integer not less than 0, m is an integer not less than 0, x0 is not greater than x1 or x2, and y0 is not greater than y1 or y2.

[0115] Clause 7. The first position is represented as (x0, y0), the second position is represented as (x1, y1), the third position is represented as (x2, y2), and the position condition includes at least one of the following: y1=y0 and x1-x0=W*2 k Or x2 = x0 and y2 - y0 = H*2 m 6. The method of claim 5, wherein k is an integer not less than 0, m is an integer not less than 0, M represents a width of a current block for the video unit, H represents a height of the current block, x0 is not greater than x1 or x2, and y0 is not greater than y1 or y2.

[0116] Clause 8. A method according to any one of clauses 1 to 7, wherein the first position is one of a top left or center position of the first block of the video, or the second position is one of a top left or center position of the second block of the video.

[0117] Clause 9. The method of clause 1, wherein the first block is a current block for the video unit, the second block is a bottom-left block of the current block, the first position is a top-left position of the current block and is represented as (x, y), and the second position is a bottom-left position of the bottom-left block and is represented as (xLB, yLB).

[0118] Item 10. The second position is derived as follows: yLB=y+H-1, and xLB=x+WW*2. k 10. The method of claim 9, wherein k is an integer, H represents the height of the current block, and W represents the width of the current block.

[0119] Clause 11. The method of clause 10, wherein xLB is equal to (xW) when k is equal to 1, and xLB is equal to (x-3*W) when k is equal to 2.

[0120] Clause 12. The method of clause 1, wherein the first block is a current block for the video unit, the second block is a block to the top right of the current block, the first position is a top left position of the current block and is represented as (x, y), and the second position is a top right position of the bottom left block and is represented as (xRT, yRT).

[0121] Item 13. The second position is derived as follows: xRT=x+W-1, and yRT=y+HH*2. k 13. The method of claim 12, wherein k is an integer, H represents the height of the current block, and W represents the width of the current block.

[0122] Item 14. The method of item 13, wherein yRT is equal to (yH) when k is equal to 1, and yRT is equal to (y-3*H) when k is equal to 2.

[0123] Clause 15. The method of clause 1, wherein the second block is a bottom-left block or a top-right block, and the second block is shifted to an adjacent block.

[0124] Clause 16. The method of clause 1, wherein the top-left block used to derive motion candidates is positioned by at least one of the bottom-left block or the top-right block.

[0125] Clause 17. The method of any one of clauses 1 to 16, wherein the conversion includes encoding the video units into the bitstream.

[0126] Clause 18. The method of any one of clauses 1 to 16, wherein the converting includes decoding the video unit from the bitstream.

[0127] Clause 19. An apparatus for video processing comprising a processor and a non-transitory memory having instructions thereon, the instructions, when executed by the processor, causing the processor to perform a method according to any one of clauses 1 to 18.

[0128] Clause 20. A non-transitory computer-readable storage medium storing instructions that cause a processor to perform the method of any one of clauses 1 to 18.

[0129] Clause 21. A non-transitory computer-readable recording medium storing a bitstream of video generated by a method performed by an apparatus for video processing, the method including: deriving a first motion candidate for a video unit of the video based on a first position of a first block of the video; and deriving a second motion candidate for the video unit based on a second position of a second block of the video, the first position and the second position satisfying a position condition; and generating the bitstream based on the first and second motion candidates.

[0130] Clause 22. A method for storing a bitstream of a video, comprising: deriving a first motion candidate for a video unit of the video based on a first position of a first block of the video; and deriving a second motion candidate for the video unit based on a second position of a second block of the video, wherein the first position and the second position satisfy a position condition; generating the bitstream based on the first and second motion candidates; and storing the bitstream in a non-transitory computer-readable recording medium.

[0131] Exemplary Devices 31 illustrates a block diagram of a computing device 3100 in which various embodiments of the disclosure may be implemented. The computing device 3100 may be implemented as or included in the source device 110 (or video encoder 114 or 200) or the destination device 120 (or video decoder 124 or 300).

[0132] It should be understood that the computing device 3100 shown in FIG. 31 is for illustrative purposes only and is not intended to suggest any limitation on the functionality and scope of the embodiments of the present disclosure.

[0133] 31, the computing device 3100 includes a general-purpose computing device 3100. The computing device 3100 may include at least one or more processors or processing units 3110, a memory 3120, a storage unit 3130, one or more communication units 3140, one or more input devices 3150, and one or more output devices 3160.

[0134] In some embodiments, the computing device 3100 may be implemented as any user terminal or server terminal having computing capabilities. The server terminal may be a server provided by a service provider, a large-scale computing device, etc. The user terminal may be, for example, any type of mobile, fixed, or portable terminal, including a mobile phone, a station, a unit, a device, a multimedia computer, a multimedia tablet, an Internet node, a communicator, a desktop computer, a laptop computer, a notebook computer, a netbook computer, a tablet computer, a personal communication system (PCS) device, a personal navigation device, a personal digital assistant (PDA), an audio / video player, a digital camera / camcorder, a positioning device, a television receiver, a radio broadcast receiver, an electronic book device, a gaming device, or any combination thereof, further including accessories and peripherals of these devices, or any combination thereof. It is contemplated that the computing device 3100 may support any type of interface to a user, such as "wearable" circuitry.

[0135] The processing unit 3110 may be a physical processor or a virtual processor and may perform various processes based on programs stored in the memory 3120. In a multiprocessor system, multiple processing units execute computer-executable instructions in parallel to increase the parallel processing capabilities of the computing device 3100. The processing unit 3110 may also be referred to as a central processing unit (CPU), a microprocessor, a controller, or a microcontroller.

[0136] The computing device 3100 typically includes a variety of computer storage media. Such media may be any media accessible by the computing device 3100, including but not limited to volatile and non-volatile media, or removable and non-removable media. The memory 3120 may be volatile memory (e.g., registers, cache, Random Access Memory (RAM)), non-volatile memory (Read-Only Memory (ROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), or flash memory, etc.), or any combination thereof. The storage unit 3130 may be any removable or non-removable media and may include machine-readable media such as memory, flash memory drives, magnetic disks, or other media that may be used to store information and / or data and that are accessible within the computing device 3100.

[0137] The computing device 3100 may further include other removable / non-removable, volatile / non-volatile memory media. Although not shown in Figure 31, it is possible to provide a magnetic disk drive for reading from and / or writing to a removable, non-volatile magnetic disk, and an optical disk drive for reading from and / or writing to a removable, non-volatile optical disk. In such a case, each drive may be connected to a bus (not shown) via one or more data media interfaces.

[0138] The communication unit 3140 communicates with further computing devices via a communication medium. Furthermore, the functionality of the components in the computing device 3100 may be implemented by a single computing cluster or multiple computing machines that can communicate via a communication connection. Thus, the computing device 3100 may operate in a networked environment using logical connections with one or more other servers, networked personal computers (PCs), or even more general network nodes.

[0139] The input device 3150 may be one or more of a variety of input devices such as a mouse, a keyboard, a tracking ball, a voice input device, etc. The output device 3160 may be one or more of a variety of output devices such as a display, a speaker, a printer, etc. The communication unit 3140 allows the computing device 3100 to further communicate with one or more external devices (not shown), such as a storage device and a display device, with one or more devices that allow a user to interact with the computing device 3100, or with any device (such as a network card, a modem, etc.) that allows the computing device 3100 to communicate with one or more other computing devices as needed. Such communication may be performed via an input / output (I / O) interface (not shown).

[0140] In some embodiments, instead of being integrated into a single device, some or all of the components of the computing device 3100 may be located in a cloud computing architecture. In a cloud computing architecture, the components may be provided remotely and may cooperate to implement the functionality described in this disclosure. In some embodiments, cloud computing provides computing, software, data access and storage services without the end user needing to be aware of the physical location or configuration of the systems or hardware providing these services. In various embodiments, cloud computing provides services over a wide area network (such as the Internet) using appropriate protocols. For example, a cloud computing provider provides applications accessible through a web browser or any other computing component over the wide area network. The software or components of the cloud computing architecture and corresponding data may be stored on a server located at a remote location. The computing resources of the cloud computing environment may be consolidated or distributed across locations in a remote data center. The cloud computing infrastructure may act as a single access point for users, but provide services through a shared data center. Thus, a cloud computing architecture may be used to provide the components and functionality described herein from a remote service provider. Alternatively, they may be provided from a traditional server or installed directly or otherwise on a client device.

[0141] The computing device 3100 may be used to perform video encoding / decoding in embodiments of the present disclosure. The memory 3120 may include one or more video coding modules 3125 having one or more program instructions. These modules are accessible and executable by the processing unit 3110 to perform the functions of the various embodiments described herein.

[0142] In an example embodiment performing video encoding, the input device 3150 may receive video data as input 3170 to be encoded. The video data may be processed, for example, by the video coding module 3125 to generate an encoded bitstream. The encoded bitstream may be provided as output 3180 via the output device 3160.

[0143] In an example embodiment performing video decoding, the input device 3150 may receive an encoded bitstream as an input 3170. The encoded bitstream may be processed, for example, by the video coding module 3125 to generate decoded video data. The decoded video data may be provided as output 3180 via the output device 3160.

[0144] Although the present disclosure has been particularly shown and described with reference to preferred embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present application as defined by the appended claims. Such variations are intended to be covered by the scope of the present application. Thus, the foregoing description of the embodiments of the present application is not intended to be limiting.

Claims

1. 1. A method for video processing, comprising: deriving a first motion candidate for the video unit based on a first position of a first block and a second position of a second block of the video during conversion between a video unit of a video and a bitstream of the video unit, the first position and the second position satisfying a position condition; performing the transformation based on the first motion candidate and a second motion candidate.

2. The method of claim 1 , wherein the first motion candidate is one of a non-adjacent affine inheritance candidate or a non-adjacent affine construction candidate.

3. The first position is represented as (x0, y0), the second position is represented as (x1, y1), and the position condition is y1=y0 and x1-x0=2 ｋ or x1=x0 and y1-y0=2 ｋ where k is an integer not less than 0, x0 is not greater than x1, and y0 is not greater than y1.

4. The first position is represented as (x0, y0), the second position is represented as (x1, y1), and the position condition is y1=y0 and x1-x0=W*2 ｋ or x1=x0 and y1-y0=H*2 ｋ where k is an integer greater than or equal to 0, M represents a width of a current block for the video unit, H represents a height of the current block, x0 is not greater than x1, and y0 is not greater than y1.

5. The method of claim 1 , further comprising deriving the first motion candidate for the video unit based on a third position of a third block of the video.

6. The first position is represented as (x0, y0), the second position is represented as (x1, y1), the third position is represented as (x2, y2), and the position condition is: y1=y0 and x1-x0=2 ｋ or x2=x0 and y2-y0=2 ｍ wherein k is an integer not less than 0, m is an integer not less than 0, x0 is not greater than x1 or x2, and y0 is not greater than y1 or y2.

7. The first position is represented as (x0, y0), the second position is represented as (x1, y1), the third position is represented as (x2, y2), and the position condition is: y1=y0 and x1-x0=W*2 ｋ or x2=x0 and y2-y0=H*2 ｍ where k is an integer not less than 0, m is an integer not less than 0, M represents a width of a current block for the video unit, H represents a height of the current block, x0 is not greater than x1 or x2, and y0 is not greater than y1 or y2.

8. the first location is one of a top left or a center location of the first block of the video; or The method of any one of claims 1 to 7, wherein the second location is one of a top left or a central location of the second block of the video.

9. the first block is a current block for the video unit and the second block is a bottom-left block of the current block; and 2. The method of claim 1, wherein the first location is a top-left location of the current block and is represented as (x, y), and the second location is a bottom-left location of the bottom-left block and is represented as (xLB, yLB).

10. The second position is yLB=y+H-1, and xLB=x+W-W*2 ｋ 10. The method of claim 9, wherein k is an integer, H represents the height of the current block, and W represents the width of the current block.

11. If k is equal to 1, then xLB is equal to (x-W); 11. The method of claim 10, wherein when k is equal to 2, xLB is equal to (x-3*W).

12. the first block is a current block for the video unit and the second block is an upper right block of the current block; and 2. The method of claim 1, wherein the first location is a top-left location of the current block and is represented as (x, y), and the second location is a top-right location of the bottom-left block and is represented as (xRT, yRT).

13. The second position is xRT=x+W−1, and yRT=y+H-H*2 ｋ 13. The method of claim 12, wherein k is an integer, H represents the height of the current block, and W represents the width of the current block.

14. If k is equal to 1, then y is equal to (y−H); 14. The method of claim 13, wherein when k is equal to 2, yRT is equal to (y-3*H).

15. The second block is the lower left block or the upper right block, and The method of claim 1 , wherein the second block is shifted to an adjacent block.

16. The method of claim 1 , wherein the top-left block used to derive motion candidates is positioned by at least one of the bottom-left block or the top-right block.

17. The method of any one of claims 1 to 16, wherein the converting comprises encoding the video units into the bitstream.

18. The method of any one of claims 1 to 16, wherein the converting comprises decoding the video units from the bitstream.

19. An apparatus for video processing comprising a processor and a non-transitory memory having instructions thereon, the instructions, when executed by the processor, causing the processor to perform a method according to any one of claims 1 to 18.

20. A non-transitory computer readable storage medium storing instructions for causing a processor to perform the method of any one of claims 1 to 18.

21. 1. A non-transitory computer-readable recording medium storing a bitstream of video generated by a method performed by an apparatus for video processing, the method comprising: deriving a first motion candidate for a video unit of the video based on a first position of a first block and a second position of a second block of the video, the first position and the second position satisfying a position condition; generating the bitstream based on the first motion candidate and a second motion candidate; A non-transitory computer-readable recording medium comprising:

22. 1. A method for storing a video bitstream, comprising: deriving a first motion candidate for a video unit of the video based on a first position of a first block and a second position of a second block of the video, the first position and the second position satisfying a position condition; generating the bitstream based on the first motion candidate and a second motion candidate; storing the bitstream on a non-transitory computer-readable recording medium.

Images