Block-size-based motion vector coding in affine mode

Abstract

At least a method and apparatus for efficiently encoding or decoding a video are proposed. For example, a plurality of motion vectors used in determining motion information of a block using affine modeling are obtained, wherein at least one of the plurality of motion vectors is determined based on the block size of the block. The video is then encoded or decoded based on the plurality of motion vectors.

Description

Technical Field

[0001] At least one embodiment of the present invention generally relates to a method or apparatus for video encoding or decoding, and more specifically, to a method or apparatus for efficiently providing video compression and / or decompression based on decoding of one or more motion vectors taking into account block size, especially for affine mode motion decoding. Background Art

[0002] To achieve high compression efficiency, image and video coding schemes typically employ prediction, including motion vector prediction, and transforms to exploit spatial and temporal redundancy in video content. Typically, intra-frame or inter-frame prediction is used to exploit intra-frame or inter-frame correlations, and then the difference between the original image and the predicted image, which is usually represented as a prediction error or prediction residual, is transformed, quantized, and entropy decoded. To reconstruct the video, the compressed data is decoded by the inverse processes corresponding to entropy decoding, quantization, transform, and prediction.

[0003] Recent additions to video compression technology include various industry standards, versions of reference software and / or documents, such as the Joint Exploration Model (JEM) and later VTM (Versatile Video Coding (VVC) Test Model) developed by the JVET (Joint Video Exploration Team) group, which aims to further improve the existing HEVC (High Efficiency Video Coding) standard.

[0004] A recent addition to high compression techniques includes the use of motion models based on affine modeling. In particular, affine modeling is used for motion compensation in encoding and decoding of video pictures. Typically, affine modeling is a model that uses at least two parameters, such as two control point motion vectors (CPMVs) representing the motion at the corresponding corners of a block of a picture, which allows the motion field for the entire block of the picture to be derived to simulate, for example, rotation and similarity (scaling). Summary of the invention

[0005] The shortcomings and deficiencies of the prior art are addressed and addressed by the general aspects described herein.

[0006] According to a first aspect, a method is provided. The method comprises: obtaining a plurality of motion vectors used in determining motion information of a block using affine modeling, wherein at least one of the plurality of motion vectors is determined based on a block size of the block; and encoding the video based on the plurality of motion vectors.

[0007] According to another aspect, a second method is provided, comprising: obtaining a plurality of motion vectors used in determining motion information of a block using affine modeling, wherein at least one of the plurality of motion vectors is determined based on a block size of the block; and decoding the video based on the plurality of motion vectors.

[0008] According to another aspect, an apparatus is provided. The apparatus includes one or more processors, wherein the one or more processors are configured to: obtain a plurality of motion vectors in determining motion information of a block using affine modeling, wherein at least one of the plurality of motion vectors is determined based on a block size of the block; and encode the video based on the plurality of motion vectors.

[0009] According to another aspect, another apparatus is provided. The apparatus includes one or more processors, wherein the one or more processors are configured to: obtain a plurality of motion vectors used in determining motion information of a block using affine modeling, wherein at least one of the plurality of motion vectors is determined based on a block size of the block; and decode the video based on the plurality of motion vectors.

[0010] According to another general aspect of at least one embodiment, the block size of the block is the width of the block.

[0011] In accordance with another general aspect of at least one embodiment, wherein the at least one of the plurality of motion vectors is determined with respect to its pixel accuracy.

[0012] In accordance with another general aspect of at least one embodiment, the pixel accuracy decreases with increasing block size above a threshold size.

[0013] In accordance with another general aspect of at least one embodiment, at least another motion vector of the plurality of motion vectors is determined based on a height of the block.

[0014] In accordance with another general aspect of at least one embodiment, at least another additional motion vector of the plurality of motion vectors is determined independently of block size.

[0015] According to another general aspect of at least one embodiment, the plurality of motion vectors are control point motion vectors for affine modeling.

[0016] In accordance with another general aspect of at least one embodiment, the plurality of motion vectors are represented as motion vector differences.

[0017] According to another general aspect of at least one embodiment, the affine modeling is in an affine advanced motion vector prediction (AMVP) mode.

[0018] According to another general aspect of at least one embodiment, a non-transitory computer-readable medium is provided that contains data content generated according to any of the described encoding embodiments or variations.

[0019] According to another general aspect of at least one embodiment, there is provided a signal comprising video data generated according to any of the described encoding embodiments or variations.

[0020] According to another general aspect of at least one embodiment, a bitstream is formatted to include data content generated according to any of the described encoding embodiments or variations.

[0021] According to another general aspect of at least one embodiment, there is provided a computer program product comprising instructions which, when executed by a computer, cause the computer to perform any of the described decoding embodiments or variations.

[0022] These and other aspects, features and advantages of the general aspects will become apparent from the following detailed description of illustrative embodiments which is to be read in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 A block diagram of an embodiment of a video encoder is shown.

[0024] Figure 2 A block diagram of an embodiment of a video decoder is shown.

[0025] Figure 3 A diagrammatic example illustrating coding tree units and coding tree concepts for representing compressed HEVC pictures is shown.

[0026] Figure 4 An illustrated example illustrating division of a coding tree unit into coding units, prediction units, and transform units is shown.

[0027] Figure 5 An example of a simple affine model used in a joint exploration model is shown.

[0028] Figure 6 An example of an affine motion vector field based on a 4×4 sub-CU is shown.

[0029] Figure 7 An example of motion vector prediction processing for an affine inter CU is shown.

[0030] Figure 8 An example of a process for creating new candidates in an affine advanced motion vector prediction (AMVP) process is shown.

[0031] Fig. 9 An example of motion vector prediction candidates in affine merge mode is shown.

[0032] Fig.10 An example of spatial derivation of affine motion field control points in the case of affine merging is shown.

[0033] Fig.11 A block diagram of a system in which aspects of the present embodiment may be implemented is shown. DETAILED DESCRIPTION

[0034] Figure 1 An example video encoder 100 is shown, such as a High Efficiency Video Coding (HEVC) encoder. Figure 1 Encoders in which improvements are made to the HEVC standard or encoders employing techniques similar to HEVC may also be shown, such as the VVC (Versatile Video Coding) encoder under development by JVET (Joint Video Exploration Team).

[0035] In this application, the terms "reconstruction" and "decoding" can be used interchangeably, the terms "encoding" and "decoding" can be used interchangeably, and the terms "image", "picture" and "frame" can be used interchangeably. Usually, but not necessarily, the term "reconstruction" is used on the encoder side, while "decoding" is used on the decoder side.

[0036] Before being encoded, the video sequence may undergo a pre-encoding process (101), for example, applying a color transform to the input color picture (e.g., conversion from RGB 4:4:4 to YCbCr 4:2:0), or performing a remapping of the input picture components in order to obtain a signal distribution that is more resilient to compression (e.g., using histogram equalization of one of the color components). Metadata may be associated with the pre-processing and appended to the bitstream.

[0037] In HEVC, to encode a video sequence with one or more pictures, the pictures are partitioned (102) into one or more slices, where each slice may include one or more slice segments. The slice segments are organized into coding units, prediction units, and transform units. The HEVC specification distinguishes between "blocks" and "units", where a "block" addresses a specific area in the sample array (e.g., luma, Y), and a "unit" contains a concatenated block of all coded color components (Y, Cb, Cr, or monochrome), syntax elements, and prediction data (e.g., motion vectors) associated with the block.

[0038] For decoding, the picture is partitioned into coding tree blocks (CTBs) of a square with a configurable size, and a continuous set of coding tree blocks is grouped into slices. A coding tree unit (CTU) contains the CTB of the encoded color component. The CTB is the root of the quadtree partitioned into coding blocks (CBs), and the coding block can be partitioned into one or more prediction blocks (PBs) and forms the root of the quadtree partitioned into transform blocks (TBs). Corresponding to the coding blocks, prediction blocks, and transform blocks, the coding unit (CU) includes a tree structure set of prediction units (PUs) and transform units (TUs), the PU includes prediction information for all color components, and the TU includes a residual coding syntax structure for each color component. The size of the CB, PB, and TB of the luminance component applies to the corresponding CU, PU, ​​and TU. In the present application, the term "block" may be used to represent any one of CTU, CU, PU, ​​TU, CB, PB, and TB. Additionally, "block" may also be used to refer to macroblocks and partitions as specified in H.264 / AVC or other video coding standards, and more generally to data arrays of various sizes.

[0039] In the exemplary encoder 100, a picture is encoded by an encoder element as described below. The picture to be encoded is processed in units of CUs. Each CU is encoded using intra or inter mode. When a CU is encoded in intra mode, it performs intra prediction (160). In inter mode, motion estimation (175) and compensation (170) are performed. The encoder decides (105) whether to encode the CU using one of intra mode or inter mode, and indicates the intra / inter decision through a prediction mode flag. The prediction residual is calculated by subtracting (110) the predicted block from the original image block.

[0040] The prediction residual is then transformed (125) and quantized (130). The quantized transform coefficients as well as the motion vectors and other syntax elements are entropy coded (145) to output a bitstream. The encoder can also skip the transform and apply quantization directly to the untransformed residual signal on a 4×4TU basis. The encoder can also bypass the transform and quantization, i.e., directly encode the residual without applying the transform or quantization process. In direct PCM decoding, no prediction is applied and the coding unit samples are encoded directly into the bitstream.

[0041] The encoder decodes the coded block to provide a reference for further prediction. The quantized transform coefficients are dequantized (140) and inverse transformed (150) to decode the prediction residual. The decoded prediction residual and the prediction block are combined (155) to reconstruct the image block. An in-loop filter (165) is applied to the reconstructed picture, for example, to perform deblocking / SAO (sample adaptive offset) filtering to reduce coding artifacts. The filtered image is stored in a reference picture buffer (180).

[0042] Figure 2 2 shows a block diagram of an example video decoder 200, such as a HEVC decoder. In the decoder 200, a bitstream is decoded by decoder elements as described below. The video decoder 200 generally performs the same operations as described above. Figure 1 The encoding pass described in is reciprocal to the decoding pass, which performs video decoding as part of encoding the video data. Figure 2 Decoders in which improvements are made to the HEVC standard or decoders that employ techniques similar to HEVC, such as VVC decoders, may also be described.

[0043] Specifically, the input to the decoder comprises a video bitstream, which may be generated by the video encoder 100. The bitstream is first entropy decoded (230) to obtain transform coefficients, motion vectors, picture partitioning information, and other coding information. The picture partitioning information indicates the size of the CTU, and the way in which the CTU is partitioned into CUs and, if applicable, into PUs. The decoder can therefore divide (235) the picture into CTUs and each CTU into CUs based on the decoded picture partitioning information. The transform coefficients are dequantized (240) and inverse transformed (250) to decode the prediction residual.

[0044] The decoded prediction residual is combined with the prediction block (255) and the image block is reconstructed. The prediction block can be obtained (270) from intra prediction (260) or motion compensated prediction (i.e., inter prediction) (275). In the case of bidirectional prediction, the two motion compensated predictions can be combined with a weighted sum. An in-loop filter (265) is applied to the reconstructed image. The filtered image is stored in a reference picture buffer (280).

[0045] The decoded picture may further undergo post-decoding processing (285), such as an inverse color transform (e.g., conversion from YCbCr 4:2:0 to RGB 4:4:4) or performing an inverse remapping of the remapping process performed in the pre-encoding process (101). The post-decoding process may use metadata derived in the pre-encoding process and signaled in the bitstream.

[0046] In the HEVC video compression standard, pictures are divided into so-called coding tree units (CTUs), which are usually 64×64, 128×128 or 256×256 pixels in size. Each CTU is represented by a coding tree in the compressed domain. This is a quadtree partitioning of the CTU, where each leaf is called a coding unit (CU), such as Figure 3 As shown in . Each CU is then given some intra-frame or inter-frame prediction parameters (e.g., prediction information). To this end, it is spatially divided into one or more prediction units (PUs), each of which is assigned some prediction information. Figure 4 As shown in , intra or inter coding mode is assigned at the CU level.

[0047] Some richer motion models are supported to improve temporal prediction. To do this, a PU can be spatially divided into sub-PUs, and a richer model can be used to assign a dedicated motion vector to each sub-PU. A CU is no longer divided into PUs or TUs, and some motion data is directly allocated to each CU. In this new codec design, a CU can be partitioned into sub-CUs, and motion vectors can be calculated for each sub-CU. One of the new motion models is an affine model, which basically consists in using an affine model to represent motion vectors in a CU.

[0048] The motion model used is from Figure 5 It is shown that the affine motion field comprises the following motion vector component values ​​for each position (x, y) within the considered block:

[0049]

[0050] Equation 1: Affine model for generating motion fields within a CU for prediction

[0051] Where (v 0x ,v 0y ) and (v 1x ,v 1y ) are the so-called control point motion vectors used to generate the affine motion field. (v 0x ,v 0y ) is the upper left corner control point of the motion vector. (v 1x ,v 1y ) is the upper right control point of the motion vector.

[0052] In fact, to keep the complexity reasonable, such as Figure 6 As shown, a motion vector is calculated for each 4×4 sub-block (sub-CU) of the considered CU. At the center position of each sub-block, an affine motion vector is calculated from the control point motion vector. The obtained motion vector (MV) is represented at 1 / 16 pixel accuracy.

[0053] As a result, the time of the decoding unit in affine mode consists in motion compensating the prediction of each subblock using its own motion vector. Note that a model with 3 control points is also possible. Affine motion compensation can be used in 3 ways: affine inter (AF_INTER), affine merge and affine template. Affine inter and merge modes are described below.

[0054] Affine Inter (AF_INTER)

[0055] Figure 8An exemplary process for creating a new candidate in affine advanced motion vector prediction (AMVP) mode is shown. A CU in AMVP mode can be predicted in affine inter mode, and its size is greater than 8x8. This is signaled by a flag in the bitstream. The generation of the affine motion field for the inter CU includes determining the control point motion vector (CPMV), which is obtained by the decoder by adding the motion vector difference to the control point motion vector prediction (CPMVP). The CPVMP is a pair of motion vector candidates, taken from Figure 7 The lists (A, B, C) and (D, E) shown. Up to 6 CPMVPs can be obtained ( Multiply 3 by 2 of ).

[0056] For a block of height H and width W, the validity of the first CPVMP is checked using Equation 2:

[0057]

[0058]

[0059]

[0060]

[0061]

[0062] Equation 2: Effectiveness test for each CPMVP

[0063] Then, according to the value of the third motion vector (Taken from Figure 7 Position F or G) to sort the valid CPVMPs. The closest It is the vector given by the affine motion model for the 4×4 sub-block at the same position. The closer it is to CPVMP, the better.

[0064] For a block of height H and width W, the cost of each CPMVP is calculated using Equation 3, where X and Y are the horizontal and vertical components of the motion vector, respectively:

[0065]

[0066]

[0067]

[0068] Equation 3: Cost calculated for each CPMVP

[0069] Affine merging method

[0070] In affine merge mode, a CU-level flag indicates whether the merged CU uses affine motion compensation. If so, then Fig. 9 The first available neighboring CU that has been coded in affine mode is selected from the ordered set of candidate positions (A, B, C, D, E).

[0071] Once the first neighboring CU in affine mode is obtained, the neighboring CU is divided into the upper left corner, the upper right corner, and the lower left corner (see Fig.10 ) to retrieve 3 motion vectors and

[0072] Based on these three vectors, the two CPMVs of the upper left and upper right corners of the current CU are derived as follows:

[0073] ο

[0074] ο

[0075] Equation 4: Derivation of the CPMV of the current CU based on the three angular motion vectors of the neighboring CUs

[0076] When the control point motion vector of the current CU is obtained and When , the motion field within the current CU is calculated based on the 4×4 sub-CU through the model of Equation 1.

[0077] Decoding of angles

[0078] The affine model is represented using 2 (or 3) corners of the block. In AMVP mode, the vector is decoded using the prediction value mvp and the residual vector mvd: mv=mvp+mvd. For the first corner, top left (TL), mvd is decoded as mvd_TL=mv_TL-mvp_TL. For the second corner, top right (TR), mvd is decoded as the top left: mvd_TR=mv_TR-mvp_TR.

[0079] Alternatively, mvd is differentially decoded into mvd_TL: mvd_TR = mv_TR - mvp_TR - mvd_TL. The decoding of the third angle uses the same principle. However, with this decoding, mvd uses the same precision for all angles and all block sizes, but the precision required for the second (respectively third) mvd may depend on the block width (respectively height).

[0080] In various embodiments, Figure 1 Encoder 100, Figure 2 The decoder 200 and Fig.11 The system 1100 (described later) is suitable for implementing one or more aspects of the embodiments described herein.

[0081] In at least one embodiment, the decoding of mvd in AMVP affine mode takes into account the block size without any additional signaling. In such an embodiment, the mvd precision of AMVP affine mode is set as follows:

[0082] - The mvd precision of the first corner top left (TL) uses the nominal precision. Alternatively, the precision of the first corner mvd uses the adaptive motion vector range (AMVR) precision signaled in the bitstream.

[0083] - The mvd of the second corner (top right) uses a precision that depends on the block width.

[0084] - The MVD used in the third corner (lower left) depends on the precision of the block height.

[0085] The case of the second angle will be described below, but the same principles apply to the third angle.

[0086] First embodiment

[0087] We use L to represent the block width, A to represent the nominal accuracy (e.g., A=1 / 8 or A=1 / 4 pixel), mvd=mv-mvp to represent the motion vector difference to be encoded, expressed as an integer with nominal accuracy, and B to represent a threshold regarding the desired accuracy.

[0088] In a first embodiment, the mvd used for encoding is given by:

[0089] If L>B, mvd_e=(B*mvd / L), where, then / is integer division,

[0090] Otherwise, mvd_e=mvd.

[0091] On the decoder side, mvd is decoded using the following formula:

[0092] If L>B, then mvd=L*mvd_e / B+(L / (2*B))

[0093] Otherwise mvd=mvd_e.

[0094] According to a variant embodiment, the above mvd quantization process takes the following form:

[0095] mvd_e=(B*mvd+(L / 2)) / L.

[0096] The advantage of this variant is that when reconstructing the considered motion vector difference mvd at the decoder side, the distance between the original unquantized mvd value and the dequantized mvd obtained is smaller.

[0097] According to a variant embodiment, the quantization of the mvd can be done using only a power-of-2 quantizer. This allows to reduce complexity. Usually, a fixed value is set for B (e.g., B=16), meaning that for blocks with a width less than 16 pixels, the precision will be the same as in the first mvd. In this case, the quantizer can be given by a table. For example, for B=16, we have the following table for the quantization of the mvd:

[0098] Block width Mvd quantizer = q 4 1 8 1 16 1 24 1 32 2 48 2 64 3 128 4

[0099] In this case, the quantization of the vector mvd to be encoded is given by:

[0100] mvd_e=(mvd) / (2^(q-1)).

[0101] Thus, the above embodiments illustrate that, after a certain block size threshold, motion accuracy (eg, in terms of pixel resolution) decreases as block size increases, due to the increase in the value of the quantizer q used to quantize Mvd.

[0102] Second embodiment

[0103] According to a second embodiment of the MVD decoding process, the MVD_E vector to be encoded is the difference between the motion vector difference of the second corner control point motion vector and the motion vector difference of the first corner control point motion vector.

[0104] Therefore, mvd_e is expressed as follows:

[0105] mvd_e=(B*mvdd / L)

[0106] Where mvdd=mvd_TR–mvd_TL=(mv_TL-mvp_TL)-(mv_TR–mvp_TR).

[0107] Scaling the second-order motion vector difference is mathematically equivalent to computing the motion vector of the considered affine motion field at an intermediate spatial position I between the top left corner TL and the top right corner TR of the current block, equal to:

[0108]

[0109] The position of the motion vector of the considered affine model at the spatial position is given by:

[0110] mv_I=mv_TL+(B / L)*(mv_TR-mv_TR).

[0111] The second-order motion vector position at spatial position I is strictly equal to the above motion vector for encoding mvd_e=(B*mvdd / L).

[0112] Third embodiment

[0113] According to a third embodiment, it may be desirable to quantize the first order motion vector differences associated with a spatial position I, rather than the second order motion vector differences at said position.

[0114] In this case, the vector to be encoded is given by:

[0115] mvd_I=mvd_TL+(B / L)*(mvd_TR–mvd_TL).

[0116] Although the previous embodiments describe block-size based residual motion vectors using the AMVP coding mode, the methods are also applicable to other coding modes of residual motion vectors.

[0117] Fig.11 A block diagram of an example of a system 1100 in which various aspects and embodiments are implemented is shown. System 1100 can be implemented as a device including various components described below, and is configured to perform one or more aspects described herein. Examples of such devices include, but are not limited to, various electronic devices, such as personal computers, laptop computers, smart phones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected household appliances and servers. The elements of system 1100 can be implemented individually or in combination in a single integrated circuit, multiple ICs and / or discrete components. For example, in at least one embodiment, the processing and encoder / decoder elements of system 1100 are distributed on multiple ICs and / or discrete components. In various embodiments, system 1100 is coupled to other similar systems or other electronic devices via, for example, a communication bus or by a dedicated input and / or output port communication. In various embodiments, system 1100 is configured to implement one or more of the aspects described in this document.

[0118] System 1100 includes at least one processor 1110, which is configured to execute instructions loaded therein, for implementing various aspects described herein, for example. Processor 1110 may include embedded memory, input and output interfaces, and various other circuits known in the art. System 1100 includes at least one memory 1120 (e.g., volatile memory device and / or non-volatile memory device). System 1100 includes storage device 1140, which may include non-volatile memory and / or volatile memory, including but not limited to EEPROM, ROM, PROM, RAM, DRAM, SRAM, flash memory, disk drive, and / or optical drive. As a non-limiting example, storage device 1140 may include internal storage device, attached storage device, and / or network accessible storage device.

[0119] The system 1100 includes an encoder / decoder module 1130, which is configured to, for example, process data to provide encoded video or decoded video, and the encoder / decoder module 1130 may include its own processor and memory. The encoder / decoder module 1130 represents a module (one or more) that may be included in a device to perform encoding and / or decoding functions. As is known, a device may include one or both of the encoding and decoding modules. In addition, the encoder / decoder module 1130 may be implemented as a separate element of the system 1100, or may be incorporated into the processor 1110 as a combination of hardware and software known to those skilled in the art.

[0120] Program code to be loaded onto the processor 1110 or the encoder / decoder 1130 to perform various aspects described in this document may be stored in the storage device 1140 and subsequently loaded onto the memory 1120 for execution by the processor 1110. According to various embodiments, one or more of the processor 1110, the memory 1120, the storage device 1140, and the encoder / decoder module 1130 may store one or more of the various items during the execution of the processes described herein. These stored items may include, but are not limited to, input video, decoded video or portions of decoded video, bitstreams, matrices, variables, and intermediate or final results from the processing of equations, formulas, operations, and operational logic.

[0121] In several embodiments, memory inside the processor 1110 and / or the encoder / decoder module 1130 is used to store instructions and provide working memory for processing required during encoding or decoding. However, in other embodiments, memory external to the processing device (e.g., the processing device may be the processor 1110 or the encoder / decoder module 1130) is used for one or more of these functions. The external memory may be a memory 1120 and / or a storage device 1140, such as a dynamic volatile memory and / or a non-volatile flash memory. In several embodiments, an external non-volatile flash memory is used to store the operating system of the television. In at least one embodiment, a fast external dynamic volatile memory such as RAM is used as working memory for video encoding and decoding operations, such as MPEG-2, HEVC, or VVC (universal video coding).

[0122] As shown in block 1105, input to the elements of system 1100 may be provided through various input devices. Such input devices include, but are not limited to, (i) an RF section that receives an RF signal transmitted over the air, for example, by a broadcaster, (ii) a composite input terminal, (iii) a USB input terminal, and / or (iv) an HDMI input terminal.

[0123] In various embodiments, the input device of block 1105 has associated corresponding input processing elements known in the art. For example, the RF section may be associated with the following necessary elements: (i) selecting a desired frequency (also referred to as selecting a signal, or band limiting a signal to a frequency band), (ii) down-converting the selected signal, (iii) again band-limiting to a narrower frequency band to select (for example) a signal frequency band, which in some embodiments may be referred to as a channel, (iv) demodulating the down-converted and band-limited signal, (v) performing error correction, and (vi) demultiplexing to select a desired data packet stream. The RF section of various embodiments includes one or more elements to perform these functions, such as a frequency selector, a signal selector, a band limiter, a channel selector, a filter, a down-converter, a demodulator, an error corrector, and a demultiplexer. The RF section may include a tuner that performs various of these functions, including, for example, down-converting a received signal to a lower frequency (e.g., an intermediate frequency or a near-baseband frequency) or baseband. In a set-top box embodiment, the RF part and its relevant input processing element receive the RF signal that sends by wired (for example, cable) medium, and filter to the frequency band of expectation again by filtering, down-conversion and perform frequency selection.Various embodiments rearrange the order of above-mentioned (and other) element, remove some in these elements, and / or add other elements that perform similar or different functions.Adding element can include and insert element between existing element, for example, insert amplifier and analog-to-digital converter.In various embodiments, the RF part comprises antenna.

[0124] In addition, the USB and / or HDMI terminals may include corresponding interface processors for connecting the system 1100 to other electronic devices via USB and / or HDMI connections. It should be understood that various aspects of input processing, such as Reed-Solomon error correction, may be implemented, for example, within a separate input processing IC or within the processor 1110 as desired. Similarly, various aspects of USB or HDMI interface processing may be implemented, for example, within a separate interface IC or within the processor 1110 as desired. The demodulated, error corrected, and demultiplexed streams are provided to various processing elements, including, for example, the processor 1110 and an encoder / decoder 1130 operating in conjunction with memory and storage elements, so as to process the data streams as needed for presentation on an output device.

[0125] The various components of the system 1100 may be disposed within an integrated housing. Within the integrated housing, the various components may be interconnected and transmit data therebetween using suitable connection means 1115, such as internal buses known in the art, including I2C buses, wiring, and printed circuit boards.

[0126] The system 1100 includes a communication interface 1150 that enables communication with other devices via a communication channel 1190. The communication interface 1150 may include, but is not limited to, a transceiver configured to send and receive data through the communication channel 1190. The communication interface 1150 may include, but is not limited to, a modem or a network card, and the communication channel 1190 may be implemented, for example, within a wired and / or wireless medium.

[0127] In various embodiments, a Wi-Fi network such as IEEE 802.11 is used to stream data to the system 1100. The Wi-Fi signals of these embodiments are received through a communication channel 1190 and a communication interface 1150 suitable for Wi-Fi communication. The communication channel 1190 of these embodiments is typically connected to an access point or router that provides access to external networks including the Internet to allow streaming applications and other over-the-top communications. Other embodiments use a set-top box that delivers data through an HDMI connection of an input block 1105 to provide streaming data to the system 1100. Still other embodiments use an RF connection of an input block 1105 to provide streaming data to the system 1100.

[0128] The system 1100 can provide output signals to various output devices, including a display 1165, a speaker 1175, and other peripherals 1185. In various examples of embodiments, the other peripherals 1185 include one or more of the following: a stand-alone DVR, a disc player, a stereo system, a lighting system, and other devices that provide functions based on the output of the system 1100. In various embodiments, control signals are transmitted between the system 1100 and the display 1165, the speaker 1175, or other peripherals 1185 using signaling, such as AV. Link, CEC, or other communication protocols, which enable device-to-device control with or without user intervention. The output device can be communicatively coupled to the system 1100 via a dedicated connection through the corresponding interfaces 1160, 1170, and 1180. Alternatively, the output device can be connected to the system 1100 via the communication interface 1150 using the communication channel 1190. The display 1165 and the speaker 1175 may be integrated into a single unit in an electronic device (eg, a television) along with other components of the system 1100. In various embodiments, the display interface 1160 includes a display driver, such as a timing controller (T Con) chip.

[0129] For example, if the RF portion of input 1105 is part of a separate set-top box, the display 1165 and speaker 1175 may alternatively be separate from one or more of the other components. In various embodiments where the display 1165 and speaker 1175 are external components, the output signals may be provided via dedicated output connections, including, for example, an HDMI port, a USB port, or a COMP output.

[0130] The embodiments may be performed by computer software implemented by a processor 1110, or hardware, or a combination of hardware and software. As a non-limiting example, the embodiments may be implemented by one or more integrated circuits. The memory 1120 may be of any type suitable for the technical environment and may be implemented using any appropriate data storage technology, such as, as a non-limiting example, optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memory, and removable memory. The processor 1110 may be of any type suitable for the technical environment and may include, as a non-limiting example, one or more of the following: a microprocessor, a general-purpose computer, a special-purpose computer, and a processor based on a multi-core architecture.

[0131] Various implementations involve decoding. As used in the present application, "decoding" may include, for example, all or part of the processing performed on a received coded sequence to produce a final output suitable for display. In various embodiments, such processes include one or more of the processes typically performed by a decoder, such as entropy decoding, inverse quantization, inverse transform, and differential decoding. In various embodiments, such processes also or alternatively include processes performed by a decoder of the various implementations described in the present application.

[0132] As a further example, in one embodiment, "decoding" refers only to entropy decoding, in another embodiment, "decoding" refers only to differential decoding, and in another embodiment, "decoding" refers to a combination of entropy decoding and differential decoding. Whether the phrase "decoding process" is intended to refer specifically to a subset of operations or generally to a broader decoding process will be clear based on the context of the specific description and is believed to be well understood by those skilled in the art.

[0133] Various implementations involve encoding. In a manner similar to the above discussion of "decoding", "encoding" as used in this application may include, for example, all or part of a process performed on an input video sequence to produce an encoded bitstream. In various embodiments, such processes include one or more processes typically performed by an encoder, such as segmentation, differential encoding, transforms, quantization, and entropy encoding. In various embodiments, such processes also or alternatively include processes performed by encoders of the various implementations described in this application.

[0134] As a further example, in one embodiment, "encoding" refers only to entropy encoding, in another embodiment, "encoding" refers only to differential encoding, and in another embodiment, "encoding" refers to a combination of differential encoding and entropy encoding. Whether the phrase "encoding process" is intended to refer specifically to a subset of operations or generally to a broader encoding process will become clear based on the context of the specific description and is believed to be well understood by those skilled in the art.

[0135] Note that the syntax elements used here are descriptive terms. Therefore, they do not exclude the use of other syntax element names.

[0136] When the figure is presented as a flow chart, it should be understood that it also provides a block diagram of the corresponding apparatus. Similarly, when the figure is presented as a block diagram, it should be understood that it also provides a flow chart of the corresponding method / process.

[0137] Various embodiments relate to rate-distortion optimization. In particular, during the encoding process, a balance or trade-off between code rate and distortion is usually considered, usually given a constraint on computational complexity. Rate-distortion optimization is usually formulated as minimizing a rate-distortion function, which is a weighted sum of code rate and distortion. There are different methods to solve the rate-distortion optimization problem. For example, these methods can be based on extensive testing of all coding options, including all considered modes or decoding parameter values, and a complete evaluation of their decoding costs and the associated distortion of the reconstructed signal after decoding and decoding. Faster methods can also be used to save coding complexity, in particular, to calculate approximate distortion based on prediction or prediction residual signals rather than reconstructed signals. A mixture of these two methods can also be used, for example, by using approximate distortion only for some possible coding options and full distortion for other coding options. Other methods only evaluate a subset of possible coding options. More generally, many methods use any of a variety of techniques to perform optimization, but optimization is not necessarily a complete evaluation of both coding costs and associated distortions.

[0138] The implementations and aspects described herein may be implemented in, for example, a method or process, a device, a software program, a data stream, or a signal. Even if discussed only in the context of a single form of implementation (e.g., discussed only as a method), the implementation of the features discussed may also be implemented in other forms (e.g., a device or program). For example, a device may be implemented with appropriate hardware, software, and firmware. The method may be implemented in, for example, a processor, which generally refers to a processing device, including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. The processor also includes a communication device, such as a computer, a cellular phone, a portable / personal digital assistant ("PDA"), and other devices that facilitate information communication between end users.

[0139] References to "one embodiment," or "an embodiment," or "one implementation," or "implementation," and other variations mean that a particular feature, structure, characteristic, etc. described in connection with the embodiment is included in at least one embodiment. Thus, the appearances of the phrases "in one embodiment," or "in an embodiment," or "in an implementation," or "in an implementation," and any other variations, in various places throughout this document are not necessarily all referring to the same embodiment.

[0140] Additionally, this document may refer to "determining" various pieces of information. Determining information may include, for example, one or more of: estimating the information, calculating the information, predicting the information, or retrieving the information from a memory.

[0141] Additionally, this document may refer to "accessing" various pieces of information. Accessing information may include, for example, one or more of: receiving information, retrieving information (e.g., from memory), storing information, moving information, copying information, calculating information, determining information, predicting information, or estimating information.

[0142] Additionally, this document may refer to "receiving" various information. Like "accessing," receiving is intended to be a broad term. Receiving information may include, for example, one or more of: accessing the information or retrieving the information (e.g., from a memory). Furthermore, during operations such as storing information, processing information, sending information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information, "receiving" is often involved in one way or another.

[0143] It should be understood that, for example, in the case of "A / B," "A and / or B," and "at least one of A and B," use of any of the following " / ," "and / or," and "at least one of" is intended to encompass selection of only the first listed option (A), or only the second listed option (B), or both options (A and B). As a further example, in the case of "A, B, and / or C" and "at least one of A, B, and C," such wording is intended to include selection of only the first listed option (A), or only the second listed option (B), or only the third listed option (C), or only the first and second listed options (A and B), or only the first and third listed options (A and C), or only the second and third listed options (B and C), or all three options (A and B and C). This can be extended to multiple items listed, as will be apparent to one of ordinary skill in this and related arts.

[0144] In addition, as used herein, the word "signal / signaling" especially refers to a corresponding decoder indicating something. For example, in some embodiments, the encoder signals a specific one of a plurality of weights to be used for an intra-frame prediction reference array. In this way, in an embodiment, the same parameters are used on the encoder side and the decoder side. Therefore, for example, the encoder can send (explicit signaling) specific parameters to the decoder so that the decoder can use the same specific parameters. On the contrary, if the decoder already has specific parameters and other parameters, signaling can be used without sending (implicit signaling) to simply allow the decoder to know and select specific parameters. Bit saving is achieved in various embodiments by avoiding the transmission of any actual function. It should be understood that signaling can be implemented in various ways. For example, in various embodiments, one or more syntax elements, as well as flags, etc. are used to signal information to the corresponding decoder. Although the foregoing relates to the verb form of the word "signal / signaling", the word "signal / signaling" can also be used as a noun in this article.

[0145] As will be apparent to one of ordinary skill in the art, implementations may generate various signals formatted to carry information that may be stored or transmitted, for example. The information may include, for example, instructions for executing a method, or data generated by one of the described implementations. For example, a signal may be formatted to carry a bitstream of the described embodiments. Such a signal may be formatted as, for example, an electromagnetic wave (e.g., using a radio frequency portion of a spectrum) or a baseband signal. Formatting may include, for example, encoding a data stream and modulating a carrier with the encoded data stream. The information carried by the signal may be, for example, analog or digital information. As is known, the signal may be transmitted over a variety of different wired or wireless links. The signal may be stored on a processor readable medium.

[0146] We have described a number of embodiments. The features of these embodiments may be provided individually or in any combination. In addition, embodiments may include one or more of the above-described exemplary features, devices, or aspects, alone or in any combination, across various claim categories and types.

[0147] In the present disclosure, various other generalized and specialized aspects, embodiments and claims are also supported and contemplated. For example, various methods and other aspects described in this application can be used to modify modules, such as Figure 1 and Figure 2The motion compensation and estimation modules (170, 175; and 275) of the video encoder 100 and decoder 200 are shown. In addition, aspects of the present invention are not limited to VVC or HEVC, and can be applied to, for example, other standards and recommendations, whether pre-existing or developed in the future, as well as extensions of any such standards and recommendations (including VVC and HEVC). Unless otherwise specified or technically excluded, the aspects described in this application can be used alone or in combination.

Claims

1. A method for decoding a block of a video using affine mode motion coding, comprising: obtaining a plurality of control point motion vectors associated with a plurality of corners of the block; Obtaining a plurality of predicted values ​​of the plurality of control point motion vectors; obtaining motion vector differences between control point motion vectors and their respective prediction values, wherein at least one of the motion vector differences is quantized based on a size of the block; as well as The block is decoded based on the plurality of control point motion vector prediction values ​​and the motion vector differences. The method of claim 1 , wherein the block size of the block is a width of the block.

3. The method of claim 2 , wherein, conditional on the width of the block being greater than a threshold, the at least one motion vector difference is determined based on scaling by a factor equal to an integer division of the width of the block divided by the threshold.

4. The method according to claim 2, wherein: The quantization uses only power-of-2 quantizers.

5. The method of claim 4, wherein the quantization of at least one motion vector difference among the plurality of the motion vector differences further comprises: Based on the width of the block, a quantizer in a quantizer table is selected.

6. The method of claim 5, wherein the accuracy of the at least one motion vector difference decreases with increasing block size above a threshold size.

7. The method of claim 2, wherein the precision of at least one further additional motion vector difference of the plurality of motion vector differences is determined independently of the block size.

8. The method of claim 2, wherein the affine mode motion coding uses an affine advanced motion vector prediction (AMVP) mode.

9. A method for encoding a block of a video using affine mode motion coding, comprising: obtaining a plurality of control point motion vectors associated with a plurality of corners of the block; Determining a plurality of predicted values ​​of the plurality of control point motion vectors; determining motion vector differences between control point motion vectors and their respective predicted values; determining a quantization of at least one of the motion vector differences based on a size of the block; as well as The block is encoded based on a plurality of the motion vector differences.

10. An apparatus for decoding a block of a video using affine mode motion coding, comprising one or more processors, wherein the one or more processors are configured to: obtaining a plurality of control point motion vectors associated with a plurality of corners of the block; Obtaining a plurality of predicted values ​​of the plurality of control point motion vectors; obtaining motion vector differences between control point motion vectors and their respective prediction values, wherein at least one of the motion vector differences is quantized based on a size of the block; as well as The block is decoded based on the plurality of control point motion vector prediction values ​​and the motion vector differences.

11. An apparatus for encoding a block of a video using affine mode motion coding, comprising one or more processors, wherein the one or more processors are configured to: obtaining a plurality of control point motion vectors associated with a plurality of corners of the block; Determining a plurality of predicted values ​​of the plurality of control point motion vectors; determining motion vector differences between control point motion vectors and their respective predicted values; determining a quantization of at least one of the motion vector differences based on a size of the block; as well as The block is encoded based on a plurality of the motion vector differences.

Images