Method, apparatus, and system for determining predictive weights for merge mode

Abstract

The present invention provides a system, apparatus, and method for determining predictive weights for merge mode. [Solution] One exemplary method includes: determining weights for at least one of inherited affine merge candidates, constructed affine merge candidates, or zero motion vectors of a coding unit; and bidirectionally predicting the coding unit based on the determined weights. The weights of the current coding unit may be determined by explicitly signaling a weighting index in the bitstream or by implicit derivation on the decoder side.

Description

[Technical Field]

[0001] Cross-references to related applications

[0001] This application claims priority and convenience of priority of U.S. Provisional Patent Application No. 62 / 816,879 filed March 11, 2019, which is incorporated herein by reference.

[0002] Technical field

[0002] This disclosure generally relates to video data processing, and more specifically to methods, apparatus and systems for determining predictive weightings for merge mode. [Background technology]

[0003] background

[0003] Video coding is often used to convert and compress digital video signals, for example, to reduce the storage space consumed or to reduce the transmission bandwidth consumption associated with such signals.

[0004]

[0004] Video coding systems can implement various tools or techniques to solve various problems involved in the coding process. For example, in video coding, input video data can be divided into blocks of various sizes and processed block by block. In prediction processing, blocks can be predicted using various prediction modes, such as intra-prediction mode and inter-prediction mode. Various prediction modes can be applied based on the size of the blocks. Various techniques have been proposed to improve the efficiency and accuracy of video coding and to reduce the computational complexity involved.

[0005]

[0005] However, the application of various techniques can depend on various conditions. For example, some techniques may require that the coding unit satisfies various conditions or that the coding unit is coded in a particular mode. As another example, in weighted prediction processing, some techniques may be applicable to processing only some motion vector candidates, but not others. It may not be feasible to combine various techniques to process all blocks or all subblocks. Therefore, it is desirable to correctly combine or select various techniques by considering the applicability conditions of each. [Overview of the project] [Means for solving the problem]

[0006] Summary of Disclosure

[0006] Embodiments of the present disclosure provide methods, apparatus and systems for determining prediction weights in video data processing. According to some exemplary embodiments, the method performed by a video data decoder includes: determining weights for at least one of inherited affine merge candidates, constructed affine merge candidates or zero motion vectors of a coding unit; and predicting the coding unit bidirectionally based on the determined weights. In some embodiments, the weights may be determined based on a weighting index signaled in the bitstream.

[0007]

[0007] In some embodiments, methods are provided that can be performed by a decoder to obtain weights for a constructed affine merge candidate having multiple control points. One exemplary method includes: determining the weights for a constructed affine merge candidate based on weights associated with a top-left control point or a top-right control point, depending on whether the multiple control points associated with the constructed affine merge candidate have one or more weights.

[0008]

[0008] In some embodiments, methods are provided that can be performed by the decoder to obtain weights for a constructed affine merge candidate having a plurality of control points. One exemplary method includes: determining the weights of a plurality of control points as the weight of the constructed affine merge candidate, depending on whether the plurality of control points associated with the constructed affine merge candidate have the same weight; and determining default values ​​as the weight of the constructed affine merge candidate, depending on whether the plurality of control points have different weights.

[0009]

[0009] In some embodiments, methods are provided for determining predictive weights in video data processing. One exemplary method includes: determining the weights of a first subblock within a coding unit based on the weights of the control points of the coding unit from among the weights of a plurality of control points; generating a motion predictor for the first subblock based on the determined weights; and processing the coding unit based on the motion predictor.

[0010]

[0010] In some embodiments, devices are provided for determining prediction weights in video data processing. One exemplary video processing device includes a memory for storing instructions and a processor, the processor configured to execute the instructions such that the device: determines weights for at least one of inherited affine merge candidates, constructed affine merge candidates, or zero motion vectors of a coding unit; and predicts a coding unit in both directions based on the determined weights.

[0011]

[0011] In some embodiments, devices are provided for determining predictive weights in video data processing. One exemplary video processing device includes: determining the weights of a first subblock within a coding unit based on the weights of the control points of the coding unit; generating motion predictors for the first subblock based on the determined weights; and processing the coding unit based on the motion predictors.

[0012]

[0012] In some embodiments, non-temporary computer-readable media are provided. One exemplary non-temporary computer-readable medium stores a set of instructions which are executable by one or more processors of an image processing device to cause the image processing device to perform a method which includes determining weights for at least one of inherited affine merge candidates, constructed affine merge candidates, or zero motion vectors of a coding unit; and predicting a coding unit bidirectionally based on the determined weights.

[0013]

[0013] In some embodiments, non-temporary computer-readable media are provided. One exemplary non-temporary computer-readable medium stores a set of instructions which are executable by one or more processors of an image processing device to cause the image processing device to perform a method which includes determining the weighting of a first subblock within the coding unit based on the weighting of the control points of the coding unit; generating a motion predictor for the first subblock based on the determined weighting; and processing the coding unit based on the motion predictor.

[0014] Brief explanation of the drawing

[0001] Some embodiments and various aspects of the present disclosure are shown in the following detailed description and accompanying drawings. Various features shown in the accompanying drawings are not depicted to actual size. [Brief explanation of the drawing]

[0015] [Figure 1]

[0002] This is a schematic diagram showing an exemplary video coding and decoding system consistent with some embodiments of the present disclosure. [Figure 2]

[0003] This is a schematic diagram showing an exemplary video encoder, which may be part of the exemplary system of Figure 1, consistent with some embodiments of the present disclosure. [Figure 3]

[0004] This is a schematic diagram showing an exemplary video decoder, which may be part of the exemplary system of Figure 1, consistent with some embodiments of the present disclosure. [Figure 4A]

[0005] This is a schematic diagram of an exemplary two-control-point affine motion model according to some embodiments of the present disclosure. [Figure 4B]

[0006] This is a schematic diagram of an exemplary three-control-point affine motion model according to some embodiments of the present disclosure. [Figure 5]

[0007] This is a schematic diagram showing an exemplary affine motion vector field (MVF) per subblock according to some embodiments of the present disclosure. [Figure 6]

[0008] This is a schematic diagram showing exemplary locations of inherited affine motion predictors according to some embodiments of the present disclosure. [Figure 7]

[0009] This is a schematic diagram illustrating an example of control point motion vector inheritance according to some embodiments of the present disclosure. [Figure 8]

[0010] This is a schematic diagram showing exemplary candidate locations of affine merge modes constructed according to some embodiments of the present disclosure. [Figure 9A]

[0011] This is a schematic diagram showing a horizontal subblock division of an encoding unit having two control points according to some embodiments of the present disclosure. [Figure 9B]

[0012] This is a schematic diagram showing a vertical subblock division of an encoding unit having two control points according to some embodiments of the present disclosure. [Figure 9C]

[0013] This is a schematic diagram showing subblock partitioning of an encoding unit having two control points into four subblocks according to some embodiments of the present disclosure. [Figure 9D]

[0014] This is a schematic diagram showing the subblock division of an encoding unit having two control points into four subblocks according to some embodiments of the present disclosure. [Figure 10A]

[0015] This is a schematic diagram showing a vertical subblock partition of an encoding unit having three control points according to some embodiments of the present disclosure. [Figure 10B]

[0016] This is a schematic diagram showing a horizontal subblock partition of an encoding unit having three control points according to some embodiments of the present disclosure. [Figure 10C]

[0017] This is a schematic diagram showing a horizontal subblock partition of an encoding unit having three control points according to some embodiments of the present disclosure. [Figure 10D]

[0018] This is a schematic diagram showing a vertical subblock partition of an encoding unit having three control points according to some embodiments of the present disclosure. [Figure 10E]

[0019] This is a schematic diagram illustrating the subblock partitioning of an encoding unit having three control points into four subblocks according to some embodiments of the present disclosure. [Figure 10F]

[0020] This is a schematic diagram illustrating the subblock partitioning of an encoding unit having three control points into four subblocks according to some embodiments of the present disclosure. [Figure 10G]

[0021] This is a schematic diagram illustrating the subblock partitioning of an encoding unit having three control points into four subblocks according to some embodiments of the present disclosure. [Figure 10H]

[0022] This is a schematic diagram illustrating the subblock partitioning of an encoding unit having three control points into four subblocks according to some embodiments of the present disclosure. [Figure 11]

[0023] This is a schematic diagram illustrating the subblock partitioning of an encoding unit having four control points into four subblocks according to some embodiments of the present disclosure. [Modes for carrying out the invention]

[0016] Detailed explanation

[0024] The following description refers in detail to exemplary embodiments illustrated in the accompanying drawings. Unless otherwise noted, the following description refers to the accompanying drawings where the same numbers in various drawings represent the same or similar elements. The embodiments described in the following description of exemplary embodiments do not necessarily represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatus and methods consistent with aspects relating to the present invention as described in the accompanying claims.

[0017]

[0025] Figure 1 is a block diagram showing an exemplary video coding and decoding system 100 that may utilize techniques compliant with various video coding standards such as HEVC / H.265 and VVC / H.266. As shown in Figure 1, the system 100 includes a source device 120 that provides coded video data to be decoded at some point by a destination device 140. In accordance with some embodiments of the present disclosure, each of the source device 120 and the destination device 140 may include any of a wide range of devices such as desktop computers, notebook (e.g., laptop) computers, tablet computers, set-top boxes, mobile phones, televisions, cameras, wearable devices (e.g., smartwatches or wearable cameras), display devices, digital media players, video game consoles, and video streaming devices. The source device 120 and the destination device 140 may be equipped to be suitable for wireless or wired communication.

[0018]

[0026] Referring to Figure 1, the source device 120 may include a video source 122, a video encoder 124, and an output interface 126. The destination device 140 may include an input interface 142, a video decoder 144, and a display device 146. In some embodiments, the source and destination devices may include other components or arrangements. For example, the source device 120 may receive video data from an external video source (not shown), such as an external camera. Similarly, the destination device 140 may interface with an external display device rather than including an integrated display device.

[0019]

[0027] In the following description, some techniques are described as being performed by a video encoding device, but these techniques can also be performed by a video encoder / decoder (commonly referred to as a “CODEC”). Furthermore, the techniques of this disclosure can also be performed by a video preprocessor. Source device 120 and destination device 140 are merely examples of such encoding devices, where source device 120 generates encoded video data for transmission to destination device 140. In some embodiments, source device 120 and destination device 140 may operate in a substantially symmetrical manner, such that each of source device 120 and destination device 140 includes video encoding and decoding components. Thus, system 100 can support one-way or two-way video transmission between source device 120 and destination device 140 for, for example, video streaming, video playback, video broadcasting, or video phone.

[0020]

[0028] The video source 122 of the source device 120 may include a video capture device such as a video camera, a video archive containing previously captured video data, or a video distribution interface that receives video from a video content provider. Alternatively, the video source 122 may generate computer graphics-based data as source video or as a combination of live video, archived video, and computer-generated video. Captured, previously captured, or computer-generated video can be encoded by the video encoder 124. The encoded video information can then be output onto the communication medium 160 via the output interface 126.

[0021]

[0029] The output interface 126 may include any type of medium or device capable of transmitting encoded video data from the source device 120 to the destination device 140. For example, the output interface 126 may include a transmitter or transceiver configured to transmit encoded video data directly from the source device 120 to the destination device 140 in real time. The encoded video data may be modulated according to a communication standard, such as a wireless communication protocol, and transmitted to the destination device 140.

[0022]

[0030] The communication medium 160 may include a temporary medium such as wireless broadcast communication or wired network transmission. For example, the communication medium 160 may include a radio frequency (RF) spectrum or one or more physical transmission lines (e.g., cables). The communication medium 160 may form part of a packet-based network such as a local area network, a wide area network, or a global network such as the Internet. In some embodiments, the communication medium 160 may include routers, switches, base stations, or any other equipment that may be useful for facilitating communication from the source device 120 to the destination device 140. For example, a network server (not shown) may receive encoded video data from the source device 120 and provide such encoded video data to the destination device 140 (e.g., via network transmission).

[0023]

[0031] The communication medium 160 may also be in the form of a storage medium (e.g., a non-temporary storage medium), such as a hard disk, flash drive, compact disc, digital video disc, Blu-ray disc, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded video data. In some embodiments, a computer device in a media production facility, such as a disc stamping facility, may receive encoded video data from the source device 120 and generate a disc containing such encoded video data.

[0024]

[0032] The input interface 142 of the destination device 140 receives information from the communication medium 160. The received information may include syntactic information, which includes syntactic elements describing the characteristics and processing of blocks and other coding units. The syntactic information is defined by the video encoder 124 and used by the video decoder 144. The display device 146 displays the decoded video data to the user and may include any one of a variety of display devices, such as a cathode ray tube (CRT), liquid crystal display (LCD), plasma display, organic light-emitting diode (OLED) display, or another type of display device.

[0025]

[0033] As another example, the encoded video generated by the source device 120 may be stored on a file server or storage device. The input interface 142 can access the stored video data from the file server or storage device via streaming or download. The file server or storage device may be any type of computer device that stores the encoded video data and can transmit such encoded video data to the destination device 140. Examples of file servers include web servers that support websites, file transfer protocol (FTP) servers, network attached storage (NAS) devices, or local disk drives. Transmission of the encoded video data from the storage device may be via streaming, download, or a combination thereof.

[0026]

[0034] The video encoder 124 and the video decoder 144 can each be implemented as any of a variety of suitable encoder circuit systems, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or any combination thereof. When these technologies are partially implemented in software form, one device may store instructions for the software in a suitable non-temporary computer-readable medium and execute these instructions in hardware using one or more processors to perform the technologies of this disclosure. Each of the video encoder 124 and the video decoder 144 may be included in one or more encoders or decoders, any of which may be integrated as part of a combined encoder / decoder (CODEC) within each device.

[0027]

[0035] The video encoder 124 and video decoder 144 may operate according to any video encoding standard, such as the General-Purpose Video Coding (VVC / H.266) standard, the High Efficiency Video Coding (HEVC / H.265) standard, or the ITU-T H.264 (also known as MPEG-4) standard. Although not shown in Figure 1, in some embodiments, the video encoder 124 and video decoder 144 may be integrated with an audio encoder and decoder, respectively, and may include a suitable MUX-DEMUX unit or other hardware and software to handle the encoding of both audio and video in a common data stream or separate data streams.

[0028]

[0036] Figure 2 is a schematic diagram showing an exemplary video encoder 200 consistent with the disclosed embodiments. For example, video encoder 200 may be used as video encoder 124 in system 100 (Figure 1). Video encoder 200 may perform intra-encoding or inter-encoding of blocks within a video frame, including video blocks or divisions or subdivisions of video blocks. Intra-encoding may rely on spatial prediction to reduce or eliminate spatial redundancy in the video within a given video frame. Inter-encoding may rely on temporal prediction to reduce or eliminate temporal redundancy in the video within adjacent frames of a video sequence. Intra-mode may refer to many space-based compression modes. Inter-mode (such as unidirectional or bidirectional prediction) may refer to many time-based compression modes.

[0029]

[0037] Referring to Figure 2, the input video signal 202 can be processed block by block. For example, a video block unit may be a 16x16 pixel block (e.g., a macroblock (MB)). The size of the video block unit can vary depending on the encoding technique used and the required precision and efficiency. In HEVC, extended block sizes (e.g., coding tree units (CTUs)) can be used to compress video signals with resolutions of 1080p or higher, for example. In HEVC, a CTU may contain up to 64x64 chromaticity samples corresponding to luminance samples and associated syntactic elements. In VVC, the size of the CTU can be further increased to include 128x128 luminance samples, corresponding chromaticity samples, and associated syntactic elements. A CTU can be further divided into coding units (CUs) using, for example, a quadtree, binary tree, or ternary tree. A CU can be further divided into prediction units (PUs) to which a separate prediction method can be applied. Each input video block can be processed using a spatial prediction unit 260 or a temporal prediction unit 262.

[0030]

[0038] The spatial prediction unit 260 performs spatial prediction (e.g., intra-prediction) for the current block / CU using information about the same picture / slice, including the current block. Spatial prediction may use pixels from already encoded adjacent blocks within the same picture picture frame / slice to predict the current video block. Spatial prediction may reduce spatial redundancy inherent in the video signal.

[0031]

[0039] The temporal prediction unit 262 performs a temporal prediction (e.g., interpretation) for the current block using information from a picture / slice different from the picture / slice containing the current block. Temporal predictions for a video block can be signaled by one or more motion vectors. In unit-directional temporal prediction, only one motion vector pointing to a single reference picture is used to generate the prediction signal for the current block. On the other hand, in bidirectional temporal prediction, two motion vectors, each pointing to its own reference picture, may be used to generate the prediction signal for the current block. The motion vectors may indicate the amount and direction of movement between the current block and one or more associated blocks in the reference coordinate system. If multiple reference pictures are supported, one or more reference picture indices for the video block may be transmitted. One or more reference indices may be used to identify which reference picture in the reference picture storage or decoded picture buffer (DPB) 264 can generate the temporal prediction signal.

[0032]

[0040] The mode determination and encoder control unit 280 within the encoder may select a prediction mode (for example, based on velocity distortion optimization). Based on the determined prediction mode, a prediction block may be acquired. The prediction block may be subtracted from the current image block in the adder 216. The prediction residual may be transformed by the transformation unit 204 and quantized by the quantization unit 206. The quantized residual coefficients may be inversely quantized in the inverse quantization unit 210 and inversely transformed in the inverse transformation unit 212 to form a reconstructed residual. The reconstructed residual may be added to the prediction block in the adder 226 to form a reconstructed image block. The reconstructed image block before loop filtering may be used to provide a reference sample for intra-prediction.

[0033]

[0041] The reconstructed video block may pass through loop filtering in the loop filter 266. For example, loop filtering such as a deblocking filter, sample adaptive offset (SAO), and adaptive loop filter (ALF) may be applied. The reconstructed block after loop filtering may be stored in the reference picture storage 264 and used to provide inter-predictive reference samples for encoding other video blocks. To form the output video bitstream 220, the encoding mode (e.g., inter or intra), predictive mode information, motion information, and quantized residual coefficients may be sent to the entropy encoding unit 208 to further reduce the bitrate before the data is compressed and packed to form the bitstream 220.

[0034]

[0042] Figure 3 is a schematic diagram showing a video decoder 300 consistent with the disclosed embodiment. For example, the video decoder 300 may be used as video decoder 144 in system 100 (Figure 1). Referring to Figure 3, the video bitstream 302 may be unpacked or entropy-decoded in the entropy decoding unit 308. Encoding mode information may be used to determine whether to select the spatial prediction unit 360 or the temporal prediction unit 362. Prediction mode information may be transmitted to the corresponding prediction unit to generate prediction blocks. For example, motion-compensated predictions may be applied by the temporal prediction unit 362 to form a temporal prediction block.

[0035]

[0043] The residual coefficients may be sent to the inverse quantization unit 310 and the inverse transform unit 312 to obtain the reconstructed residual. The predicted block and the reconstructed residual may be added in 326 to form a reconstructed block before loop filtering. The reconstructed block may then pass through loop filtering in the loop filter 366. For example, loop filtering such as a deblocking filter, SAO, and ALF may be applied. The reconstructed block after loop filtering may then be stored in the reference picture storage 364. The reconstructed data in the reference picture storage 364 may be used to obtain the decoded image 320 or to predict future image blocks. The decoded image 320 may be displayed on a display device such as the display device 146 as described in System 100 (Figure 1).

[0036]

[0044] In some embodiments, affine motion prediction can be applied. In HEVC, for example, a translational motion model for motion compensation prediction (MCP) is applied. In the real world, there are many types of motion, such as zoom in / out, rotation, visual field motion, and other irregular motions. In VVC, block-based affine transform motion compensation prediction is applied. As shown in Figures 4A and 4B, the affine motion field of a block can be described by motion information of a 2-control point motion vector (4-parameter affine motion model: see Figure 4A) or a 3-control point motion vector (6-parameter affine motion model: see Figure 4B).

[0037]

[0045] As shown in Figure 4A, v0 represents the control point motion vector of the upper left corner control point (CP) 402, and v1 represents the control point motion vector of the upper right corner CP 404. For a four-parameter affine motion model, the motion vector at the sample position (x,y) within the block can be derived as follows:

number

[0038]

[0046] As shown in FIG. 4B, in the 6-parameter affine motion model, v0 represents the control point motion vector of the upper-left corner control point (CP) 406, v1 represents the control point motion vector of the upper-right corner CP408, and v2 represents the control point motion vector of the lower-left corner CP410. For the 6-parameter affine motion model, the motion vector at the sample position (x, y) within the block can be derived as follows:

Equation

[0039]

[0047] In some embodiments, to simplify the motion compensation prediction process, block-based affine transform prediction may be applied. To derive the motion vector of each 4×4 luminance sub-block, as shown in FIG. 5, the motion vector of the central sample of each sub-block is calculated according to the above equations (1) and (2), and can be rounded to 1 / 16 fractional precision. The arrows in FIG. 5 indicate the motion vectors corresponding to each sub-block. Next, a motion compensation interpolation filter may be applied to generate a prediction for each sub-block based on the derived motion vector. The sub-block size of the chrominance component may also be set to 4×4. The motion vector of the 4×4 chrominance sub-block can be calculated as the average of the motion vectors of the four corresponding 4×4 luminance sub-blocks.

[0040]

[0048] As with translational motion interpretation, there are also two affine motion interpretation modes: the affine merge mode and the Advanced Motion Vector Prediction (AMVP) mode. The affine merge mode can be applied to CUs having both width and height of 8 or more. In AMVP mode, the control point motion vector (CPMV) of the current CU can be generated based on motion information of spatially adjacent CUs. Up to five CPMV candidates can exist. An index can be signaled to indicate one candidate to be used for the current CU. The following three types of CPMV candidates are used to form the affine merge candidate list: (1) inherited affine merge candidates extrapolated from the CPMVs of neighboring CUs; ​​(2) constructed affine merge candidate CPMVs derived using the translational motion vectors (MVs) of neighboring CUs; ​​and (3) zero MVs. The three types of candidates are described further below.

[0041]

[0049] In VVC, there can be up to two inherited affine merge candidates. The two inherited affine merge candidates are derived from the affine motion models of neighboring blocks (one from the left neighboring CU and one from the upper neighboring CU). Candidate blocks A0, A1, B0, B1, and B2 are shown in Figure 6. With respect to the left motion vector predictor, neighboring blocks may be checked in the following order: A0 → A1. With respect to the upper motion vector predictor, neighboring blocks may be checked in the following order: B0 → B1 → B2. A first inherited candidate may be selected from each side (left and upper). That is, on each side, the first neighboring block encoded in affine mode in the order it is checked is selected. Furthermore, a pruning check may not be performed between the two inherited candidates.

[0042]

[0050] Once a neighboring affine CU is identified, its control point motion vector can be used to derive CPMV candidates in the affine merge list of the current CU. As shown in Figure 7, if the neighboring lower-left block A is encoded in affine mode, the motion vectors v2, v3, and v4 of the upper-left, upper-right, and lower-left corners of CU704 containing block A can be obtained. If block A is encoded by a 4-parameter affine model, the two CPMVs of the current CU702 can be calculated according to v2 and v3. Alternatively, if block A is encoded by a 6-parameter affine model, the three CPMVs of the current CU702 can be calculated according to v2, v3, and v4.

[0043]

[0051] The constructed affine candidate means that the candidate is constructed by combining the neighbor motion information of each control point. As shown in Figure 8, the motion information of the four control points 802, 804, 806, and 808 can be derived from the defined spatial and temporal neighbors. CPMV k (k=1,2,3,4) is the Kth control point CP k It can be used to represent the motion vector. For CPMV1 corresponding to CP801, neighboring blocks B2, B3 and A2 can be checked in the following order: B2 → B3 → A2. The MV of the first available block can be used. For CPMV2 corresponding to CP804, neighboring blocks B1 and B0 can be checked in the following order: B1 → B0. For CPMV3 corresponding to CP806, neighboring blocks A1 and A0 can be checked in the following order: A1 → A0. For CPMV4 corresponding to CP808, the temporal motion vector predictor (TMVP) can be used as CPMV4 if available.

[0044]

[0052] After the motion video (MV) data for the four control points 802-808 is obtained, affine merge candidates can be constructed based on the motion information. The combination of control point MVs can be used to construct candidates in the following order: {CPMV1,CPMV2,CPMV3}, {CPMV1,CPMV2,CPMV4}, {CPMV1,CPMV3,CPMV4}, {CPMV2,CPMV3,CPMV4}, {CPMV1,CPMV2}, {CPMV1,CPMV3}

[0045]

[0053] A combination of three CPMVs can construct a 6-parameter affine merge candidate, and a combination of two CPMVs can construct a 4-parameter affine merge candidate. In some embodiments, to avoid motion scaling, corresponding combinations of control point MVs may be discarded if the reference indices of the control points are different.

[0046]

[0054] After the inherited and constructed affine merge candidates have been checked, if the affine merge candidate list is not yet full, a zero MV may be inserted at the end of the list.

[0047]

[0055] The affine AMVP mode can be applied to CUs having both width and height greater than 16. An affine flag at the CU level may be signaled in the bitstream to indicate whether the affine AMVP mode is used. If the affine AMVP mode is used, another flag may be signaled to indicate whether a 4-parameter affine or a 6-parameter affine is used. The difference between the current CU's CPMV and its predictor (CPMVP) may be signaled in the bitstream. The affine AMVP candidate list size is 2. The candidate list can be generated by using the following four types of CPMVP candidates in the following order: (1) inherited affine AMVP candidates extrapolated from the CPMV of neighboring CUs; ​​(2) constructed affine AMVP candidate CPMVP derived using the translational MV of neighboring CUs; ​​(3) translational MV from neighboring CUs; ​​and (4) zero MV.

[0048]

[0056] The order in which inherited affine AMVP candidates are checked can be the same as the order in which inherited affine merge candidates are checked, as described above. The only difference is that for AMVP candidates, only affine CUs that have the same reference picture as the current block are considered. No pruning is applied when inserting inherited affine motion predictors into the candidate list.

[0049]

[0057] The constructed AMVP candidates can be derived from a defined spatial neighborhood, as shown in Figure 8. The same check order used in constructing affine merge candidates may be used. In addition, the reference picture index of neighboring blocks is also checked. The first block in the check order, which is intercoded and has the same reference picture as the current CU, is used. If the current CU is encoded in a 4-parameter affine mode and both mv0 and mv1 are available, then mv0 and mv1 may be added as one candidate in the affine AMVP list. If the current CU is encoded in a 6-parameter affine mode and all three CPMVs are available, then all three CPMVs are added as one candidate in the affine AMVP list. Otherwise, the constructed AMVP candidates may be set to unavailable.

[0050]

[0058] After the inherited affine AMVP candidates and constructed AMVP candidates have been checked, the affine AMVP list candidates may still be less than 2. In this case, mv0, mv1, and mv2 may be added in order as translational MVs to predict all control point MVs of the current CU, if available. Finally, if the affine AMVP candidate list is still not full, zero MVs may be used to fill the list.

[0051]

[0059] In some embodiments, bi-prediction with weighted averaging (BWA) may be applied. Traditionally, bi-prediction signals are generated simply by averaging two prediction signals obtained from two different reference pictures and / or using two different motion vectors. In VVC, the bi-prediction mode is extended beyond simple averaging to allow weighted averaging of two prediction signals, as shown in the following equation: P bi-pred =((8-w)*P0+w*P1+4)≫3 (3)

[0052]

[0060] Five weights are allowed in weighted-averaged bidirectional prediction, w ∈ {-2, 3, 4, 5, 10}. For each bidirectional prediction CU, the weight w can be determined in one of two ways: 1) for non-merged CUs, the weight index can be signaled after the motion vector difference; 2) for merged CUs, the weight index can be estimated from neighboring blocks based on the merge candidate index. Weighted-averaged bidirectional prediction with 256 or more luminance samples can be applied to a CU, meaning that the CU height × CU width is 256 or greater. For low-latency pictures, all five weights can be used. For non-low-latency pictures, only three weights (w ∈ {3, 4, 5}) are used.

[0053]

[0061] On the encoder side, high-speed search algorithms can be applied to discover weighting metrics without significantly increasing encoder complexity. The application of these algorithms can be summarized as follows: When combined with adaptive motion vector resolution (AMVR), if the current picture is a low-latency picture, only unequal weightings are conditionally checked with respect to 1-pixel and 4-pixel motion vector accuracy. When combined with affine motion compensation, affine motion estimation (ME) can be performed with respect to unequal weightings only if and only if the affine mode is selected as the best mode currently available. If the two reference pictures in bidirectional prediction are the same, only unequal weightings are conditionally checked.

[0054]

[0062] In some embodiments, unequal weightings are not searched if certain conditions are met. Various factors may be considered, such as the picture order count (POC) distance between the current picture and its reference picture, the coding quantization parameter (QP), and the time level.

[0055]

[0063] As noted above, when BWA is combined with merge prediction, the weighting index can be estimated from neighboring blocks based on the merge candidate index. However, this may not always be the case for affine merge prediction. As noted above, with respect to affine merge prediction, there can be three types of CPMV candidates in the affine merge candidate list: 1) inherited affine merge candidates extrapolated from the CPMV of neighboring CUs; ​​2) constructed affine merge candidate CPMVs derived using the translational motion MV of neighboring CUs; ​​and 3) zero MVs. If the first type of candidate (inherited affine merge candidate) is selected, the BWA weighting index of the neighboring CU is also inherited. Thus, the weighting index of the current CU used in bidirectional prediction is the same as the weighting index of the neighboring CU from which the CPMV is inherited. However, if the second type of candidate (constructed affine merge candidate) or the third type of candidate (zero motion vector) is selected, the BWA weighting index of the neighboring CU is not inherited. Alternatively, in some techniques, equal weighting may be used for the current CU by setting the current CU weighting index as the default value.

[0056]

[0064] The above design has at least the following drawbacks: Based on the above design, there may be a mismatch in BWA weighting index inheritance between inherited affine merge candidates and constructed affine merge candidates, which are two types of CPMV candidates within the same candidate list. Therefore, the encoder and decoder must distinguish between these two types of candidates and use different logic to generate the BWA weighting for the current CU. Constructed affine merge candidates are fixed, while inherited affine merge candidates are inherited from neighboring CUs. This thus increases processing complexity.

[0057]

[0065] Furthermore, the current CU weighting can be fixed when a constructed affine merge candidate is selected, thus losing flexibility in weight selection. BWA is adapted to the VC standard because it increases coding performance by offering more options for averaging weights in interpretation. However, using fixed, equal weighting can reduce the benefits of BWA and thus impair coding performance.

[0058]

[0066] Furthermore, zero MVs are inserted into the affine merge candidate list if the candidate list is not full after inserting inherited and constructed affine merge candidates. However, fixing the BWA weighting for all zero MVs inserted into the candidate list means that the same MV candidate will be duplicated in the list, which introduces redundancy in candidate signaling. In addition, as mentioned above, neighbor CU weighting is not used for constructed affine merge candidates. This deviates from the design philosophy that, when BWA is combined with merge mode, the weighting index is estimated from neighboring blocks based on the merge candidate index.

[0059]

[0067] To solve the above-mentioned problems, the following technologies are provided in this disclosure.

[0060]

[0068] One technique to address the above problem is through explicit weighting signaling. With explicit signaling, the weighting of the current CU can be selected by the encoder, and the index of the selected weighting can be explicitly signaled in the bitstream. This changes the concept that the weighting index is estimated from neighboring blocks based on the merge candidate index. With respect to affine merge prediction, the weighting of the current CU is no longer estimated from neighboring CUs. The encoder can select any value from the allowed candidate weights and signal the selected value in the bitstream.

[0061]

[0069] In some embodiments of this disclosure, the weights of all types of affine merge candidates, including inherited affine merge candidates, constructed affine merge candidates, and zero motion vectors, may all be selected by an encoder and signaled in a bitstream. In some embodiments, the weights of some but not all affine merge candidates may be selected by an encoder and signaled in a bitstream. For example, with respect to constructed affine merge candidates only, the weights may be selected and signaled; and with respect to other candidates (e.g., inherited affine merge candidates), the weights may be estimated from neighboring CUs or fixed to default values.

[0062]

[0070] In some embodiments of the present disclosure, the weighting may be signaled by transmitting the weighting index in a bitstream.

[0063]

[0071] Another technique to address the existing problem is implicit weighting derivation. With this technique, the weights are not fixed with respect to the CU (Common Unit) in which the constructed affine merge candidate is used. Instead, the weights can be estimated from neighboring CUs. For each constructed affine merge candidate, the weight or weighting index can be derived according to the CP (Common Unit) of this constructed affine merge candidate. If this constructed affine merge candidate is used for a CU, the derived weight or weighting index can also be used for the CU.

[0064]

[0072] For example, CP k(k=1,2,3,4) can be used to represent the Kth CP. For each CP, neighboring CUs can be used to derive the corresponding CPMV. Neighboring CUs can also be used to derive the weights or weighting indices of the CPs. Generally speaking, there are four control points. Therefore, a constructed affine merge candidate may have at most four different weights or weighting indices. Thus, rules can be prescribed for deriving one weight or weighting indice of a constructed affine merge candidate from at most four weights or weighting indices. For example, multiple CPs, including the top-left CP and / or top-right CP of a constructed affine merge candidate, have one or more weights. The weights of a constructed affine merge candidate may be determined based on the weights associated with the top-left CP and / or the weights associated with the top-right CP. Similarly, in some embodiments, the bottom-left CP and / or bottom-right CP of a constructed affine merge candidate have one or more weights. The weighting of the constructed affine merge candidates may be determined based on weightings related to the lower-left CP and / or weightings related to the lower-right CP. An example of the derivation process is further described below. In some embodiments of this disclosure, if all CPs have the same weight or weighting index, this weight or weighting index may be used for the constructed affine merge candidates. If all CPs do not have the same weight or weighting index, a default value corresponding to equal weights may be used.

[0065]

[0073] For example, two CPs (CP a ,CP b Regarding a constructed affine merge candidate having ), if (CP a Weighting == CP b (Weighting) Affine merge candidate weighting = CP a weighting Otherwise Affine merge candidate weighting = Default value

[0066]

[0074] Furthermore, there are three CPs (CP a ,CPb ,CP c Regarding a constructed affine merge candidate having ), if (CP a Weighting == CP b Weighting &&CP b Weighting == CP c (Weighting) Affine merge candidate weighting = CP a weighting Otherwise Affine merge candidate weighting = Default value

[0067]

[0075] Similarly, the above rules may also be applied to constructed affine merge candidates having four or more CPs, and the above operations may also be applied to weighting metrics.

[0068]

[0076] In some embodiments of this disclosure, the weighting or weighting index of the constructed affine merge candidate may be a maximum likelihood weighting or weighting index. The maximum likelihood weighting or weighting index may correspond to the weighting or weighting index used by most of the CPs of the constructed affine merge candidate. For example, two CPs (CP a ,CP b For constructed affine merge candidates having ), the weighting can be determined based on the following: if (CP a Weighting == CP b (Weighting) Affine merge candidate weighting = CP a weighting Otherwise Affine merge candidate weighting = Default value

[0069]

[0077] 3 CP (CP a ,CP b ,CP c For constructed affine merge candidates having ), the weighting can be determined based on the following: if (CP a Weighting == CP b (Weighting) Affine merge candidate weighting = CP a weighting Otherwise, if(CP b Weighting == CP c (Weighting) Affine merge candidate weighting = CP b weighting Otherwise, if(CP a Weighting == CP c (Weighting) Affine merge candidate weighting = CP c weighting

[0070]

[0078] The above rules may also be applied to constructed affine merge candidates having four or more CPs and to the determination of weighting metrics.

[0071]

[0079] In some embodiments of this disclosure, the weight having the smallest difference from equal weights among all the weights of the constructed affine merge candidate CP may be used as the weight or weighting index of the constructed affine merge candidate. For example, two CPs (CP a ,CP b For constructed affine merge candidates having ), the weighting can be determined based on the following: Diff a =|CP a Weighting - Equal weighting | Diff b =|CP b Weighting - Equal weighting | if( Diff a ≤ Diff b ) Affine merge candidate weighting = CP a weighting Otherwise Affine merge candidate weighting = CP b weighting

[0072]

[0080] 3 CP (CP a ,CP b ,CP cFor constructed affine merge candidates having ), the weighting can be determined based on the following: Diff a =|CP a Weighting - Equal weighting | Diff b =|CP b Weighting - Equal weighting | Diff c =|CP c Weighting - Equal weighting | if( Diff a ≤ Diff b && Diff a ≤ Diff c ) Affine merge candidate weighting = CP a weighting Otherwise, if( Diff b ≤ Diff a && Diff b ≤ Diff c ) Affine merge candidate weighting = CP b weighting Otherwise Affine merge candidate weighting = CP c weighting

[0073]

[0081] Alternatively, two CPs (CP a ,CP b For constructed affine merge candidates having ), the weighting can be determined based on the following: if (CP a Weighting ≥ Equal weighting &&CP b (Weighting ≥ Equal weighting) Affine merge candidate weighting = min(CP) a Weighting, CP b (Weighting) Otherwise, if(CP a Weighting ≤ Equal Weighting &&CP b (Weighting ≤ Equal weighting) Affine merge candidate weighting = max(CP a Weighting, CP b (Weighting)

[0074]

[0082] And for a constructed affine merge candidate having three CPs (CP a , CP b , CP c ), the weighting can be determined based on the following: if (CP a weighting ≥ equal weighting && CP b weighting ≥ equal weighting && CP c weighting ≥ equal weighting) Affine merge candidate weighting = min(CP a weighting, CP b weighting, CP c weighting) Otherwise, if (CP a weighting ≤ equal weighting && CP b weighting ≤ equal weighting && CP c weighting ≤ equal weighting) Affine merge candidate weighting = max(CP a weighting, CP b weighting, CP c weighting) Otherwise Affine merge candidate weighting = equal weighting, or the closer weighting of the two weightings on the same side of equal weighting

[0075]

[0083] Similarly, the above rules can also be applied to constructed affine merge candidates having four or more CPs and to the determination of the weighting index.

[0076]

[0084] In some embodiments, the average of the CP weightings or the weighting index can be used as the weighting of the constructed affine merge candidate. For example, for a constructed affine merge candidate having two CPs (CP a , CP b ), the weighting can be determined based on the following: weighting = integer part of (CP a [[ID=6......]] b weighting) / 2 res = (CP​a Weighting + CP b Weighting) %2 if(res==0) Affine merge candidate weighting = weighting Otherwise Affine merge candidate weighting = (CP a Weighting + CP b Rounding by equal weighting direction (weighting) / 2

[0077]

[0085] Furthermore, there are three CPs (CP a ,CP b ,CP c For constructed affine merge candidates having ), the weighting can be determined based on the following: Weighting = (CP a Weighting + CP b Weighting + CP c The integer part of (weighting) / 3 res=(CP a Weighting + CP b Weighting + CP c Weighting) %3 if(res==0) Affine merge candidate weighting = weighting Otherwise Affine merge candidate weighting = (CP a Weighting + CP b Weighting + CP c Rounding by equal weighting (weighting) / 3 in the direction of weighting

[0078]

[0086] Similarly, the above rules may also be applied to constructed affine merge candidates having four or more CPs and to the determination of weighting metrics.

[0079]

[0087] According to some embodiments of the present disclosure, a method for using BWA weighting with respect to zero MV candidates is also provided. As noted above, after the inherited affine merge candidates and constructed affine merge candidates have been inserted into the affine merge candidate list (if the candidate list is not full), the candidate list will be filled with zero MVs until it is full. Consistent with some embodiments of the present disclosure, the candidate list may be filled with zero MVs having different BWA weightings such that the filled zero MV candidates may provide different predictors.

[0080]

[0088] For example, the weighting order of zero MVs filling the candidate list could be {4,4} / 8, {5,3} / 8, {3,5} / 8, {-2,10} / 8, {10,-2} / 8. After the inherited affine merge candidates and constructed affine merge candidates have been derived, if the candidate list is not full, a zero MV with equal weighting {4,4} / 8 may be inserted first. If the list is still not full, a zero MV with weighting {5,3} / 8 may be inserted. Subsequently, if the list is still not full, zero MVs with weighting {3,5} / 8, {-2,10} / 8, and {10,-2} / 8 may be inserted until the list is full. After all zero MVs with weightings have been inserted, if the candidate list is still not full, zero MVs with different criterion metrics may be inserted.

[0081]

[0089] In VVC, several constructed affine merge candidates may exist. The order of the candidate control point sets / combinations is as follows: {CP1,CP2,CP3}, {CP1,CP2,CP4}, {CP1,CP3,CP4}, {CP2,CP3,CP4}, {CP1,CP2}, {CP1,CP3}. The encoder and decoder can check from the first combination to the last. If available, a combination can be added to the candidate list. If a combination is not available, the next combination can be checked. The indicators of the candidates selected within the candidate list may be represented by bitstreams with variable-length codes. Smaller indicators can be encoded by shorter-length codes. Therefore, the order of candidates in the list can affect encoding efficiency. The fixed order of constructed affine merge candidates may not be optimized for each video sequence.

[0082]

[0090] To address the aforementioned issues, the order of merge candidates can be adapted. For example, the weighting of CPs within a combination can be checked. Based on the check results, the order of each combination can be determined.

[0083]

[0091] In some embodiments of this disclosure, combinations in which all CPs have the same weight or weighting index are assigned the highest priority and placed first in the list. Combinations in which CPs have the most diverse weights or weighting indexes are assigned the lowest priority and placed later in the list. In some embodiments of this disclosure, CPs may be classified into different classes according to their CP weights. Combinations in which all CPs belong to the same class may be assigned the highest priority and placed first in the list. Combinations in which CPs belong to the most diverse classes may be assigned the lowest priority and placed later in the list.

[0084]

[0092] Based on the above, the more consistent the weighting of the CPs within a combination, the higher the priority the combination has. For example, with respect to the combinations {CPMV1,CPMV2,CPMV3} and {CPMV1,CPMV2,CPMV4}, if CP1, CP2, and CP3 have the same weight or weighting index or belong to the same class, and CP4 has a different weight or weighting index or belongs to a different class, then {CPMV1,CPMV2,CPMV3} may be assigned a higher priority. That is, in the list, {CPMV1,CPMV2,CPMV3} comes before {CPMV1,CPMV2,CPMV4}. Alternatively, if the weights or weighting index or corresponding class of CP1, CP2, and CP3 are all different, but the weight or weighting index or corresponding class of CP4 is the same as CP1, then {CPMV1,CPMV2,CPMV4} is not as diverse and may be assigned a higher priority. In other words, within the list, {CPMV1,CPMV2,CPMV4} comes before {CPMV1,CPMV2,CPMV3}. If two combinations have the same level of diversity in terms of weighting, weighting metrics, or classes, the default order may be assigned to these two combinations.

[0085]

[0093] In VVC, a constructed affine candidate refers to a candidate constructed by combining the neighbor motion information of each control point. The control points of the following combinations are used to sequentially construct affine merge candidates: {CP1,CP2,CP3}, {CP1,CP2,CP4}, {CP1,CP3,CP4}, {CP2,CP3,CP4}, {CP1,CP2}, {CP1,CP3}. The encoder and decoder can check the availability of each combination. If available, the constructed affine merge candidate can be added to the candidate list. If a combination is not available, the availability of the next combination can be checked. In existing techniques, the availability condition focuses on whether neighboring CUs are intercoded or not. BWA weighting is not considered.

[0086]

[0094] New availability conditions may be applied in accordance with some embodiments of this disclosure. The weighting or weighting index of each CP in the combination may be checked. The availability of the combination may be determined according to the check results.

[0087]

[0095] In some embodiments, a combination may be set as unavailable if the weights of two CPs in the combination have different signs. For example, in the combination {CP1, CP2, CP3) if the weight of CP1 is (-2, 10) and the weight of CP2 is (10, -2), this combination may be set as unavailable. As another example, in the combination {CP1, CP2, CP3) if the weight of CP1 is (-2, 10) and the weight of CP2 is (4, 4), this combination may be set as unavailable.

[0088]

[0096] In VVC, BWA weighting is applied at the CU or PU level. This means that each pixel within a single CU or PU has the same weighting in the prediction process. However, in affine motion prediction, subblock-based motion compensation may be applied. The motion vector of each subblock within the current CU can be derived from the motion vector of the control point of the current CU. Then, a motion compensation interpolation filter is applied to generate predictors for each subblock having the derived motion vector. Subblock-level weighting can be derived according to some embodiments of this disclosure to align the motion vector with the BWA weighting.

[0089]

[0097] In some embodiments, the weighting of each subblock within the current CU or PU may be derived from the weighting of the control points of the current CU or PU. Thus, each subblock may have a different weighting.

[0090]

[0098] In some embodiments, the weighting of each pixel w x,yThe CP can be interpolated from the weights wcp0, wcp1, and wcp2, where (x,y) represents the coordinates of each pixel. Various interpolation filters, not limited to those specified herein, can be used. To simplify the interpolation process, the weights can be derived at the subblock level. For example, (x,y) could be the center coordinates of each subblock. Furthermore, the subblock size can be the same as that used in affine motion prediction.

[0091]

[0099] In some embodiments of this disclosure, a CU or PU may be divided into, for example, two or four subblocks. For each subblock, a weighting of the control points contained in the subblock may be used for the subblock. For subblocks that do not have control points in the control point combination of the currently constructed affine merge candidate, a predetermined rule may be used to derive a weighting value. For example, as shown in Figure 9A, with respect to a 2CP combination {CP1, CP3}, the CU or PU 910 may be divided horizontally into two subblocks 911, 912. The weighting of CP1, W1 may be used for the upper subblock 911. The weighting of CP3, W3 may be used for the lower subblock 912. As another example, as shown in Figure 9B, with respect to a 2CP combination {CP1, CP2}, the CU or PU 920 may be divided vertically into two subblocks 921, 922. The weighting of CP1 may be used for the left subblock 921. The weighting of CP2 may be used for the right subblock 922.

[0092]

[0100] Alternatively, for a 2CP combination {CP1, CP3}, as shown in Figure 9C, the CU or PU 930 can be divided into four subblocks 931, 932, 933, and 934. The weighting of CP1 can be used for the upper left subblock 931. The weighting of CP3 can be used for the lower left subblock 932. The weighting of the upper right subblock 933 can be derived from the weighting of CP1 or set as a default value. The weighting of the lower right subblock 934 can be derived from the weighting of CP3 or set as a default value. Similarly, for a 2CP combination {CP1, CP2}, as shown in Figure 9D, the CU or PU 940 can be divided into four subblocks 941, 942, 943, and 944. The weighting of CP1 can be used for the upper left subblock 941. The weighting of CP2 can be used for the upper right subblock 942. The weighting of the lower left subblock 943 may be derived from the weighting of CP1, or it may be set as a default value. The weighting of the lower right subblock 944 may be derived from the weighting of CP2, or it may be set as a default value.

[0093]

[0101] Further illustrative partitions for three CPs are provided below. As shown in Figure 10A, for a 3CP combination {CP1, CP2, CP4}, the CU or PU 1010 can be vertically divided into two subblocks 1011 and 1012. The weighting of CP1 can be used for the left subblock 1011. The weighting of CP2 can be used for the right subblock 1012. For a 3CP combination {CP1, CP3, CP4}, as shown in Figure 10B, the CU or PU 1020 can be horizontally divided into two subblocks 1021 and 1022. The weighting of CP1 can be used for the upper subblock 1021. The weighting of CP3 can be used for the lower subblock 1022. For a 3CP combination {CP2, CP3 (CP4)}, as shown in Figure 10C, the CU or PU 1030 can be horizontally divided into subblocks 1031 and 1032. The weighting of CP2 may be used for the upper subblock 1031. The weighting of CP3 may be used for the lower subblock 1032. Alternatively, as shown in Figure 10D, the CU or PU 1040 may be divided vertically into subblocks 1041 and 1042. The weighting of CP2 may be used for the right subblock 1042. The weighting of CP3 may be used for the left subblock 1041.

[0094]

[0102] A CU or PU can be further divided into four subblocks. As shown in Figure 10E, with respect to a 3CP combination {CP1, CP2, CP3}, a CU or PU 1050 can be divided into four subblocks 1051, 1052, 1053, and 1054. The weighting of CP1 can be used for the upper left subblock 1051. The weighting of CP2 can be used for the upper right subblock 1052. The weighting of CP3 can be used for the lower left subblock 1053. The weighting of the lower right subblock 1054 may be set to one of the following: the average of the three weights CP1, CP2, and CP3; the average of the weights of CP2 and CP3; the median of the three weights CP1, CP2, and CP3; one of the three weights CP1, CP2, and CP3 that has the smallest difference from equal weights; one of the weights CP2 and CP3 that has a smaller difference from equal weights; equal weights; or another default value.

[0095]

[0103] As shown in Figure 10F, for a 3CP combination {CP1, CP2, CP4}, CU or PU 1060 can be divided into four subblocks 1061, 1062, 1063, and 1064. The weight of CP1 can be used for the upper left subblock 1061. The weight of CP2 can be used for the upper right subblock 1062. The weight of CP4 can be used for the lower right subblock 1064. The weight of the lower left subblock 1063 can be set to one of the following: the average of all three weights; the average of the weights of CP1 and CP4; the median of the three weights; one of the three weights of CP1, CP2, and CP4 that has the smallest difference from equal weights; one of the weights of CP1 and CP4 that has a smaller difference from equal weights; equal weights; or another default value.

[0096]

[0104] As shown in Figure 10G, with respect to a 3CP combination {CP1, CP3, CP4}, CU or PU 1080 can be divided into four subblocks 1071, 1072, 1073, and 1074. The weight of CP1 may be used for the upper left subblock 1071, the weight of CP3 may be used for the lower left subblock 1073, and the weight of CP4 may be used for the lower right subblock 1074. The weight of the upper right subblock 1072 may be set to one of the following: the average of all three weights; the average of the weights of CP1 and CP4; the median of the three weights; one of the three weights of CP1, CP3, and CP4 that has the smallest difference from equal weights; one of the weights of CP1 and CP4 that has a smaller difference from equal weights; equal weights; or another default value.

[0097]

[0105] As shown in Figure 10H, with respect to the 3CP combination {CP2, CP3, CP4}, CU or PU 1080 can be divided into four subblocks 1081, 1082, 1083, and 1084. The weight of CP2 can be used for the upper right subblock 1082. The weight of CP3 can be used for the lower left subblock 1083. The weight of CP4 can be used for the lower right subblock 1084. The weight of the upper left subblock 1081 can be set to one of the following: the average of all three weights; the average of the weights of CP2 and CP3; the median of the three weights of the control point; one of the three weights of CP2, CP3, and CP4 that has the smallest difference from equal weights; one of the weights of CP2 and CP3 that has a smaller difference from equal weights; equal weights; or another default value.

[0098]

[0106] Further illustrative partitions relating to four CPs are provided below. As shown in Figure 11, with respect to the combination {CP1, CP2, CP3, CP4}, the CU or PU 1100 can be divided into four subblocks 1101, 1102, 1103, and 1104. The weighting of CP1 may be used for the upper left subblock 1101. The weighting of CP2 may be used for the upper right subblock 1102. The weighting of CP3 may be used for the lower left subblock 1103. The weighting of CP4 may be used for the lower right subblock 1104.

[0099]

[0107] In some embodiments, non-temporary computer-readable storage media are also provided, which may include instructions that can be executed by an apparatus for performing the methods described above (such as the disclosed encoder and decoder). Common forms of non-temporary media include, for example, floppy disks, flexible disks, hard disks, solid drives, magnetic tapes, or any other magnetic data storage media, CD-ROMs, any other optical data storage media, any physical media having a pattern of holes, RAM, PROMs, EPROMs, FLASH®-EPROMs, or any other flash memory, NVRAMs, caches, registers, any other memory chips or cartridges, and network versions thereof. The apparatus may include one or more processors (CPUs), input / output interfaces, network interfaces, and / or memory.

[0100]

[0108] It should be noted that relational terms such as “first,” “second,” etc., in this specification are used solely to distinguish one entity or operation from another entity or operation, and do not imply or require any actual relationship or sequence between these entities or operations. Furthermore, the terms “contains,” “has,” and other similar forms are intended to be semantically equivalent and are intended to be open in that any item or group of items following any one of these terms does not imply that such item or group of items is a general reference or is limited to only the listed items or group of items.

[0101]

[0109] As used herein, unless otherwise specified, the term "or" encompasses all possible combinations unless impossible. For example, if it is stated that a database may contain A or B, then unless otherwise specified or impossible, the database may contain A, or B, or A and B. As a second example, if it is stated that a database may contain A, B, or C, then unless otherwise specified or impossible, the database may contain A, or B, or C, or A and B, or A and C, or B and C, or A and B and C.

[0102]

[0110] It is understood that the above embodiments may be implemented by hardware, software (program code), or a combination of hardware and software. If implemented by software, it may be stored in the above computer-readable medium. The software may be executed by a processor as disclosed. The computer units and other functional units described in this disclosure may be implemented by hardware, software, or a combination of hardware and software. Those skilled in the art will also understand that several of the above modules / units may be combined into a single module / unit, and each of the above modules / units may be further divided into several submodules / subunits.

[0103]

[0111] Some embodiments of this disclosure may be further described using the following items: 1. A method for processing video data, Determining weights for at least one of inherited affine merge candidates, constructed affine merge candidates, or zero motion vectors of coding units; and A method comprising predicting coding units in both directions based on determined weights. 2. The method described in item 1, wherein the weighting is determined based on a weighting index transmitted via a bitstream. 3. The determined weighting is the method described in item 1, which includes the weighting of the constructed affine merge candidates. A method further comprising determining the weights of a constructed affine merge candidate based on the weights associated with the upper left control point or the upper right control point, depending on whether the control points associated with the constructed affine merge candidate have one or more weights. 4. The determined weighting is the method described in item 1 or 3, which includes the weighting of the constructed affine merge candidate. Determining the weights of multiple control points as the weights of the constructed affine merge candidate, depending on whether the multiple control points associated with the constructed affine merge candidate have the same weights; or A method further comprising determining default values ​​as weights for a constructed affine merge candidate, depending on that multiple control points have different weights. 5. The determined weighting is the method described in item 1 or 3, which includes the weighting of the constructed affine merge candidate. Determining the weights used by most of the control points from among the weights of multiple control points corresponding to the constructed affine merge candidate; and A method that further includes setting the determined weights as the weights for the constructed affine merge candidates. 6. The determined weighting is the method described in item 1 or 3, which includes the weighting of the constructed affine merge candidate. Determining the difference between equal weighting and weighting of multiple control points; From among multiple control points, determine a first control point having a weight that has the minimum difference from equal weights; and A method further comprising setting the weights of the first control points as the weights of the constructed affine merge candidates. 7. The determined weighting is the method described in item 1 or 3, which includes the weighting of the constructed affine merge candidate. Determining the average weight of multiple control points corresponding to the constructed affine merge candidate; and A method that further includes setting average weights as the weighting for the constructed affine merge candidates. 8. Bidirectional prediction of coding units based on determined weights is a method of any one of items 1 to 7, which includes processing video data using affine merge mode, A method for constructing an affine merge candidate list, further comprising inserting zero motion vectors with equal weights into the affine merge candidate list before inserting zero motion vectors with unequal weights. 9. Bidirectional prediction of coding units based on determined weights is a method of any one of items 1 to 7, which includes processing video data using affine merge mode, and the method further includes When constructing a list of affine merge candidates, the process includes determining the order of the multiple constructed affine merge candidates based on the weighting of the control points corresponding to each of the multiple constructed affine merge candidates. A method by which a constructed affine merge candidate having control points with less diverse weightings is assigned a higher priority than a constructed affine merge candidate having control points with more diverse weightings. 10. Determining the order of multiple constructed affine merge candidates: the method of item 9, which includes using the default order of two constructed merge candidates, depending on whether the weighting of the control points corresponding to each of the two constructed merge candidates has the same level of diversity. 11. The method described in item 9 or 10, Determining the availability of the first constructed affine merge candidate based on the weighting of control points corresponding to the first constructed affine merge candidate; and A method further comprising adding a first constructed affine merge candidate to an affine merge candidate list in response to a decision that a first constructed affine merge candidate is available. 12. Determining the availability of the first constructed affine merge candidate is: Determining whether the weightings of the two control points corresponding to the first constructed affine merge candidate have different signs; and The method according to item 11, comprising determining that the first constructed affine merge candidate is not available, based on the determination that the weights of two control points corresponding to the first constructed affine merge candidate have different signs. 13. A method for processing video data, wherein this method is Determining the weighting of the first subblock within a coding unit based on the weighting of the control points of the coding unit; and A method comprising predicting a first subblock in both directions based on determined weights. 14. Determining the weighting of the first subblock within the coding unit is: Depending on whether the first subblock contains one of the control points, the weight of one of the control points may be used as the weighting of the first subblock, or The method according to item 13, comprising determining the weighting of the first subblock according to predetermined rules, depending on whether the first subblock does not contain control points. 15. The coding unit is divided into four subblocks and has two control points, as described in item 13; determining the weighting of the first subblock within the coding unit is: Depending on whether the first subblock contains one of the two control points, the weight of one of the two control points may be used as the weighting of the first subblock, or Depending on the fact that the first subblock does not contain two control points, the weighting of the first subblock is as follows: Weighting of one of two control points that have a shorter distance to the first subblock, or The method described in item 13, including setting it to one of the default values. 16. The method of item 13, wherein the coding unit is divided into two subblocks and has three control points; determining the weighting of the first subblock within the coding unit is: Depending on whether the first subblock contains only one of the three control points, the weighting of the first subblock may be based on the weight of one of the control points, or A method comprising using a default value or the weight of one of two control points as the weighting of the first subblock, depending on that the first subblock includes at least two of three control points. 17. The coding unit is divided into four subblocks, each having three control points with first, second, and third weights; determining the weight of the first subblock within the coding unit is: Depending on whether the first subblock includes at least one of the three control points, the weighting of the first subblock may be the weight of one of the control points, or Depending on the fact that the subblock does not contain three control points, the weighting of the first subblock is as follows: The mean values ​​of the first, second, and third weights, The median of the first, second, and third weights, Equal weighting of coding units, One of the first, second, and third weights having the smallest difference from equal weights, One of the first and second weights having a smaller difference from equal weightings, or The method described in item 13, including setting it to one of the default values. 18. Image processing device: This device is Memory for storing instructions; and To this device: Determining weights for at least one of inherited affine merge candidates, constructed affine merge candidates, or zero motion vectors of coding units; and A video processing device including a processor configured to execute instructions that cause the device to predict coding units bidirectionally based on determined weights. 19. The apparatus described in item 18, in which the weighting is determined based on a weighting index transmitted via a bitstream. 20. The determined weights are those of the apparatus described in item 18, which include the weights of the constructed affine merge candidates, and the processor further provides the apparatus with: A device configured to execute an instruction to determine the weights of a constructed affine merge candidate based on the weight associated with the upper left control point or the weight associated with the upper right control point, if the multiple control points associated with the constructed affine merge candidate have one or more weights. 21. The determined weights are those of the apparatus described in item 18 or 20, which include the weights of the constructed affine merge candidates, and the processor further provides the apparatus with: If multiple control points associated with a constructed affine merge candidate have the same weighting, determine the weights of the multiple control points as the weighting of the constructed affine merge candidate; or A device configured to execute an instruction that causes a default value to be determined as the weight of a constructed affine merge candidate when multiple control points have different weights. 22. The determined weights are those of the apparatus described in item 18 or 20, which include the weights of the constructed affine merge candidates, and the processor further provides the apparatus with: From among the weights of multiple control points corresponding to the constructed affine merge candidate, determine the weights used by most of the control points; and A device configured to execute instructions that cause the determined weights to be set as the weights of the constructed affine merge candidates. 23. The determined weights are those of the apparatus described in item 18 or 20, which include the weights of the constructed affine merge candidates, and the processor further provides the apparatus with: Determining the difference between equal weighting and weighting of multiple control points; From among multiple control points, determine a first control point having a weight that has the minimum difference from equal weights; and A device configured to execute an instruction that causes the weighting of a first control point to be set as the weighting of the constructed affine merge candidate. 24. The determined weights are those of the apparatus described in item 18 or 20, which include the weights of the constructed affine merge candidates, and the processor further provides the apparatus with: Determining the average weight of multiple control points corresponding to the constructed affine merge candidate; and A device configured to execute instructions that cause the average weights to be set as the weights for the constructed affine merge candidates. 25. An apparatus according to any one of items 18 to 24, which includes processing video data using affine merge mode, wherein the apparatus further includes a processor: A device configured to execute an instruction to insert zero motion vectors with equal weights into an affine merge candidate list before inserting zero motion vectors with unequal weights when constructing an affine merge candidate list. 26. An apparatus according to any one of items 18 to 24, which includes processing video data using affine merge mode, wherein the apparatus further includes: When constructing a list of affine merge candidates, the system is configured to execute an instruction that determines the order of multiple constructed affine merge candidates based on the weighting of the control points corresponding to each of the multiple constructed affine merge candidates. A constructed affine merge candidate having control points with less diverse weightings is assigned a higher priority than a constructed affine merge candidate having control points with more diverse weightings. 27. In the device described in item 26, when determining the order of multiple constructed affine merge candidates, the processor further provides the device: A device configured to execute an instruction that causes the default order of two constructed merge candidates to be used, depending on whether the weighting of the control points corresponding to each of the two constructed merge candidates has the same level of diversity. 28. The apparatus of item 26 or 27, wherein the processor further comprises: Determining the availability of the first constructed affine merge candidate based on the weighting of control points corresponding to the first constructed affine merge candidate; and A device configured to execute an instruction to add a first constructed affine merge candidate to an affine merge candidate list if a first constructed affine merge candidate is available. 29. In the device described in item 28, when determining the availability of the first constructed affine merge candidate, the processor further provides the device with: Determining whether the weightings of the two control points corresponding to the first constructed affine merge candidate have different signs; and The apparatus described in item 28, configured to execute an instruction causing it to determine that a first constructed affine merge candidate is not available if the weights of two control points corresponding to a first constructed affine merge candidate have different signs. 30. Image processing device: This device is Memory for storing instructions; and To this device: Determining the weighting of the first subblock within a coding unit based on the weighting of the control points of the coding unit; A device including a processor configured to execute instructions causing a first subblock to be predicted bidirectionally based on determined weights. 31. In the apparatus described in item 30, when determining the weighting of the first subblock within the coding unit, the processor further provides the apparatus: If the first subblock includes one of the control points, use the weight of one of the control points as the weighting of the first subblock, or A device configured to execute an instruction to determine the weighting of a first subblock according to a predetermined rule if the first subblock does not contain a control point. 32. The apparatus according to item 30, wherein the coding unit is divided into four subblocks and has two control points; When determining the weighting of the first subblock within the coding unit, the processor further provides the device with: Depending on whether the first subblock contains one of the two control points, the weight of one of the two control points may be used as the weighting of the first subblock, or A device configured to execute an instruction to set the weighting of a first subblock to the weighting of one of the two control points having a shorter distance to the first subblock, or to a default value, depending on whether the first subblock does not contain two control points. 33. The device according to item 30, wherein the coding unit is divided into two subblocks and has three control points; When determining the weighting of the first subblock within the coding unit, the processor further provides the device with: Depending on the first subblock which contains only one of the three control points, the weight of one of the two control points may be used as the weighting of the first subblock, or A device configured to execute an instruction to use a default value or the weight of one of two control points as the weighting of a first subblock, depending on the first subblock which includes at least two of three control points. 34. The apparatus according to item 30, wherein the coding unit is divided into four subblocks and has three control points having first, second, and third weights, respectively; When determining the weighting of the first subblock within the coding unit, the processor further provides the device with: Depending on the first subblock which contains at least one of the three control points, the weight of one of the three control points may be used as the weighting of the first subblock, or Depending on the subblock that does not contain three control points, the weighting of the first subblock is as follows: The mean values ​​of the first, second, and third weights, The median of the first, second, and third weights, Equal weighting of coding units, One of the first, second, and third weights having the smallest difference from equal weights, One of the first and second weights having a smaller difference from equal weightings, or A device configured to execute a command that causes it to set to one of the default values. 35. A non-temporary computer-readable medium that stores a set of instructions executable by one or more processors of a video processing device in order to cause the video processing device to perform a method, wherein the method Determining weights for at least one of inherited affine merge candidates, constructed affine merge candidates, or zero motion vectors of coding units; and A non-temporal, computer-readable medium that includes bidirectional prediction of coding units based on determined weightings. 36. A non-transient computer-readable medium as described in item 35, in which the weighting is determined based on a weighting index transmitted via a bitstream. 37. The determined weights, including the weights of the constructed affine merge candidates, are in a non-temporary computer-readable medium as described in item 35. One set of instructions further to the device: A non-temporary computer-readable medium, executable by one or more processors of the device, to determine the weights of a constructed affine merge candidate based on the weight associated with the upper-left control point or the weight associated with the upper-right control point, when the multiple control points associated with the constructed affine merge candidate have one or more weights. 38. A non-temporary computer-readable medium as described in item 35 or 37, wherein a set of instructions is further transmitted to the device: If multiple control points associated with a constructed affine merge candidate have the same weighting, determine the weights of the multiple control points as the weighting of the constructed affine merge candidate; and A non-temporary computer-readable medium that can be operated by one or more processors of the device to determine default values ​​as weights for constructed affine merge candidates when multiple control points have different weights. 39. A non-temporary computer-readable medium as described in item 35 or 37, wherein a set of instructions is further transmitted to the device: Determining the weights used by most of the control points from among the weights of multiple control points corresponding to the constructed affine merge candidate; and A non-temporary computer-readable medium that can be executed by one or more processors of the device to set the determined weights as the weights of the constructed affine merge candidates. 40. A non-temporary computer-readable medium as described in item 35 or 37, wherein a set of instructions is further transmitted to the device: Determining the difference between equal weighting and weighting of multiple control points; From among multiple control points, determine a first control point having a weight that has the minimum difference from equal weights; and A non-temporary computer-readable medium that can be executed by one or more processors of the device to set the weighting of a first control point as the weighting of the constructed affine merge candidate. 41. A non-temporary computer-readable medium as described in item 35 or 37, wherein a set of instructions is further transmitted to the device: Determining the average weight of multiple control points corresponding to the constructed affine merge candidate; and A non-temporary computer-readable medium that can be executed by one or more processors of a device to set average weights as weights for constructed affine merge candidates. 42. A non-temporary computer-readable medium as described in any one of items 35 to 41, wherein a set of instructions is further provided to the device: A non-temporary computer-readable medium that can be executed by one or more processors of the device to cause zero motion vectors with equal weightings to be inserted into the affine merge candidate list before inserting zero motion vectors with unequal weightings when constructing the affine merge candidate list. 43. A non-temporary computer-readable medium as described in any one of items 35 to 41, wherein a set of instructions is further provided to the device: When constructing an affine merge candidate list, it is possible for one or more processors in the device to determine the order of multiple constructed affine merge candidates based on the weighting of the control points corresponding to each of the multiple constructed affine merge candidates. A constructed affine merge candidate with control points having less diverse weightings is assigned a higher priority than a constructed affine merge candidate with control points having more diverse weightings, in a non-temporal computer-readable medium. 44. A non-temporary computer-readable medium as described in item 43, wherein a set of instructions is further transmitted to the device: A non-temporary computer-readable medium that can be made by one or more processors of the device to use the default order of two constructed merge candidates, provided that the weighting of the control points corresponding to each of the two constructed merge candidates has the same level of diversity. 45. A non-temporary computer-readable medium as described in item 43 or 44, wherein a set of instructions is further transmitted to the device: Determining the availability of the first constructed affine merge candidate based on the weighting of control points corresponding to the first constructed affine merge candidate; and A non-temporary computer-readable medium that, if a first constructed affine merge candidate is available, can be executed by one or more processors of the device to cause the first constructed affine merge candidate to be added to the affine merge candidate list. 46. ​​A non-temporary computer-readable medium as described in item 45, wherein a set of instructions is further transmitted to the device: Determining whether the weightings of the two control points corresponding to the first constructed affine merge candidate have different signs; and A non-temporary computer-readable medium that can be made by one or more processors of the device to determine that the first constructed affine merge candidate is unavailable if the weightings of two control points corresponding to the first constructed affine merge candidate have different signs. 47. A non-temporary computer-readable medium for storing a set of instructions that can be executed by one or more processors of a video processing device in order to cause the video processing device to perform the method, wherein the method is Determining the weighting of the first subblock within a coding unit based on the weighting of the control points of the coding unit; and A non-temporal computer-readable medium, including bidirectional prediction of a first subblock based on determined weightings. 48. A non-temporary computer-readable medium as described in item 47, for which the weighting of the first subblock within the coding unit is determined: If the first subblock includes one of the control points, use the weight of one of the control points as the weighting of the first subblock, or A non-temporary computer-readable medium, which includes determining the weighting of the first subblock according to predetermined rules if the first subblock does not contain control points. 49. A non-temporary computer-readable medium as described in item 47, wherein a set of instructions is executable by one or more processors of the device, causing the device to further: divide an encoded unit having two control points into four subblocks; Determining the weighting of the first subblock within a coding unit is: If a subblock contains one of two control points, use the weight of one of the two control points as the weighting of the first subblock, or A non-temporary computer-readable medium, which includes setting the weighting of the first subblock to the weighting of one of the two control points having a shorter distance to the first subblock, or to a default value, if the first subblock does not contain two control points. 50. A non-temporary computer-readable medium as described in item 47, wherein a set of instructions is executable by one or more processors of the device, causing the device to further divide the encoded unit into two subblocks having three control points each; Determining the weighting of the first subblock within a coding unit is: If the first subblock contains only one of the three control points, use the weight of one of the control points as the weighting of the first subblock, or If the first subblock contains two of the three control points, use the default value or the weight of one of the two control points as the weighting for the first subblock. Non-temporary computer-readable media, including [specific examples of such media]. 51. A non-temporary computer-readable medium as described in item 47, wherein a set of instructions is executable by one or more processors of the device, causing the device to further divide the encoded unit into four subblocks. The coding unit has three control points with first, second, and third weightings; Determining the weighting of the first subblock within a coding unit is: If the first subblock includes at least one of the three control points, use the weight of one of the control points as the weighting of the first subblock, or If the first subblock does not contain three control points, the weighting of the first subblock is: The mean values ​​of the first, second, and third weights, The median of the first, second, and third weights, Equal weighting of coding units, One of the first, second, and third weights having the smallest difference from equal weights, One of the first and second weights having a smaller difference from equal weightings, or Non-temporary computer-readable media, including setting one of the default values.

[0104]

[0112] In the above specification, embodiments are described with reference to a number of specific details that may vary from embodiment to embodiment. Some adaptations and modifications of the described embodiments may be made. Other embodiments may become apparent to those skilled in the art from the specifications and consideration of the execution of the invention disclosed herein. It is intended that this specification and examples are illustrative only, and that the true scope and spirit of the invention are indicated by the following claims. It is also intended that the sequence of steps shown in the accompanying drawings is for illustrative purposes only and is not intended to limit to any specific sequence of steps. Therefore, those skilled in the art may understand that these steps may be performed in different orders while carrying out the same method.

[0105]

[0113] Exemplary embodiments are disclosed in the accompanying drawings and specification. However, many variations and modifications can be made to these embodiments. Accordingly, specific terms are used, but only in a general and descriptive sense, and not for restrictive purposes. The disclosed embodiments are not limited to the examples described above, but are instead defined by the appended claims in light of their entire scope.

Claims

1. A method for processing video data, Determining weights for at least one of inherited affine merge candidates, constructed affine merge candidates, or zero motion vectors of coding units; and A method comprising predicting the coding unit in both directions based on the weights determined above.

2. The method according to claim 1, wherein the weighting is determined based on a weighting index transmitted via a bitstream.

3. The determined weights include the weights of the constructed affine merge candidates, and the method further, The method according to claim 1, comprising determining the weights of the constructed affine merge candidates based on a weight associated with the upper left control point or a weight associated with the upper right control point, depending on whether the plurality of control points associated with the constructed affine merge candidates have one or more weights.

4. The determined weights include the weights of the constructed affine merge candidates, and the method further, Determining the weights of the control points associated with the constructed affine merge candidate as the weights of the constructed affine merge candidate, in accordance with the fact that the control points associated with the constructed affine merge candidate have the same weights; or The method according to claim 3, comprising determining default values ​​as the weights of the constructed affine merge candidate in accordance with the fact that the plurality of control points have different weights.

5. The determined weights include the weights of the constructed affine merge candidates, and the method further, Determining the weights used by most of the control points from among the weights of the multiple control points corresponding to the constructed affine merge candidate; and The method according to claim 3, comprising setting the determined weights as the weights of the constructed affine merge candidates.

6. The determined weights include the weights of the constructed affine merge candidates, and the method further, Determine the difference between equal weighting and the weighting of the multiple control points; From among the plurality of control points, determine a first control point having a weight that has the minimum difference from the equal weightings; and The method according to claim 3, comprising setting the weights of the first control points as the weights of the constructed affine merge candidates.

7. The determined weights include the weights of the constructed affine merge candidates, and the method further, Determining the average weight of multiple control points corresponding to the constructed affine merge candidate; and The method according to claim 3, comprising setting the average weight as the weight of the constructed affine merge candidate.

8. Bidirectional prediction of the coding units based on the determined weights includes processing the video data using an affine merge mode, and the method further includes The method according to claim 1, wherein when constructing an affine merge candidate list, zero motion vectors having equal weights are inserted into the affine merge candidate list before inserting zero motion vectors having unequal weights.

9. Bidirectional prediction of the coding units based on the determined weights includes processing the video data using an affine merge mode, and the method further includes When constructing a list of affine merge candidates, the process includes determining the order of the multiple constructed affine merge candidates based on the weighting of the control points corresponding to each of the multiple constructed affine merge candidates, The method according to claim 1, wherein a constructed affine merge candidate having control points with less diverse weightings is assigned a higher priority than a constructed affine merge candidate having control points with more diverse weightings.

10. Determining the order of the multiple constructed affine merge candidates is: The default order of the two constructed merge candidates is used, depending on whether the weighting of the control points corresponding to each of the two constructed merge candidates has the same level of diversity. The method according to claim 9, including the method described in claim 9.

11. Determining the availability of the first constructed affine merge candidate based on the weighting of control points corresponding to the first constructed affine merge candidate; and The method of claim 9, further comprising adding the first constructed affine merge candidate to the affine merge candidate list in response to a determination that the first constructed affine merge candidate is available.

12. Determining the availability of the first constructed affine merge candidate means Determining whether the weightings of the two control points corresponding to the first constructed affine merge candidate have different signs; and The method according to claim 11, comprising determining that the first constructed affine merge candidate is unavailable based on the determination that the weightings of the two control points corresponding to the first constructed affine merge candidate have different signs.

13. A method for processing video data, Determining the weighting of a first subblock within a coding unit based on the weighting of control points within the coding unit; and A method comprising predicting the first subblock in both directions based on the weights determined above.

14. Determining the weighting of the first subblock within the coding unit is: Depending on whether the first subblock includes one of the control points, the weight of one of the control points may be used as the weighting of the first subblock, or The method according to claim 13, comprising determining the weighting of the first subblock according to a predetermined rule, depending on whether the first subblock does not include the control point.

15. The coding unit is divided into four subblocks and has two control points; Determining the weighting of the first subblock within the coding unit is: Depending on whether the first subblock includes one of the two control points, the weighting of the first subblock may be the weight of one of the two control points, or The method according to claim 13, comprising setting the weighting of the first subblock to be the weight of the control point of the two control points that is closer to the first subblock, or to a default value, depending on whether the first subblock does not include the two control points.

16. The coding unit is divided into two subblocks and has three control points; Determining the weighting of the first subblock within the coding unit is: Depending on whether the first subblock contains only one of the three control points, the weighting of the first subblock may be the weighting of the one control point among the control points, or The method according to claim 13, wherein, depending on that the first subblock includes at least two of the three control points, a default value or the weight of one of the two control points is used as the weighting of the first subblock.

17. The coding unit is divided into four subblocks and has three control points, each having a first, second, and third weight; Determining the weighting of the first subblock within the coding unit is: Depending on whether the first subblock includes at least one of the three control points, the weighting of the first subblock may be the weighting of one of the three control points, or Depending on whether the subblock includes the three control points, the weighting of the first subblock is as follows: The average values ​​of the first, second, and third weights, The median of the first, second, and third weightings, Equal weighting of the aforementioned coding units, One of the first, second, and third weights having the minimum difference from the aforementioned equal weights, One of the first and second weights having a smaller difference from the equal weights, or Default value, The method according to claim 13, which includes setting it to be one of the following.

18. It is an image processing device: Includes memory for storing instructions, and a processor, the processor being: Determining weights for at least one of the inherited affine merge candidates, constructed affine merge candidates, or zero motion vectors of coding units; and Based on the weights determined above, predict the coding unit in both directions. A device configured to execute the command so that the device performs the aforementioned action.

19. It is an image processing device: Includes memory for storing instructions, and a processor, the processor being: Determining the weighting of a first subblock within a coding unit based on the weighting of the control points of the coding unit; Based on the weighting determined above, predict the first subblock in both directions. A device configured to execute the command so that the device performs the aforementioned action.

20. A non-temporary computer-readable medium for storing a set of instructions, wherein the instructions are executable by one or more processors of a video processing device to cause the video processing device to perform a method, and the method is: Determining weights for at least one of the inherited affine merge candidates, constructed affine merge candidates, or zero motion vectors of coding units; and Based on the weights determined above, predict the coding unit in both directions. Non-temporary computer-readable media, including [specific examples of such media].

Images