Motion vector coding and decoding using motion vector precision

Through motion vector accuracy control, the number of bits and resource consumption in the video stream is optimized, and the problem of low motion vector encoding and decoding efficiency in the prior art is solved, and more efficient video encoding is achieved.

CN119032570APending Publication Date: 2024-11-26GOOGLE LLC
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202280095561.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2022-05-07
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

When processing motion vectors, existing video encoding technology has problems such as large amount of data and high computing resource consumption. Especially when encoding and decoding motion vector differences (MVD), it is difficult to effectively reduce the number of bits, affecting efficiency.

Method used

Motion vector accuracy (MV precision) is used to determine whether to decode the least significant bits of the integer and decimal parts of the motion vector, and optimize the encoding efficiency through flexible precision signal transmission, reducing the required encoding and decoding of the motion vector and its difference. digit number.

Benefits of technology

Through flexible MV accuracy control, the number of bits in the video stream is reduced, the consumption of computing and storage resources is reduced, and the encoding efficiency is improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119032570A_ABST
    Figure CN119032570A_ABST
Patent Text Reader

Abstract

The MV category of the MVD is decoded. And the MV precision is obtained. MV accuracy is used to determine whether decoding of the least significant bit of the offset bits of the integer portion of the MVD is omitted. At least some of the offset bits are decoded. The integer portion is obtained using at least some of the offset bits of the integer portion and the least significant bits. MV accuracy is used to determine whether decoding of the least significant bit of the decimal bits of the decimal portion is omitted. At least some of the decimal bits of the decimal portion are decoded. The decimal portion is obtained using at least some of the decimal bits of the decimal portion and the least significant bits. The MVD is obtained using at least the integer part and the decimal part. The MV of the current block is obtained using the MVD.
Need to check novelty before this filing date? Find Prior Art

Description

Background Art

[0001] A digital video stream can represent a video using a sequence of frames or still images. Digital video can be used for a variety of applications, including, for example, video conferencing, high-definition video entertainment, video advertising, or sharing of user-generated videos. A digital video stream may contain a large amount of data and consume a considerable amount of computing or communication resources of a computing device for processing, transmission, or storage of video data. Various methods have been proposed to reduce the amount of data in a video stream, including compression and other coding techniques. These techniques may include both lossy and lossless coding techniques. Summary of the invention

[0002] The present disclosure relates generally to encoding and decoding video data using reference frames, and more particularly, to motion vector encoding and decoding using motion vector precision.

[0003] A system of one or more computers may be configured to perform a specific operation or action by installing software, firmware, hardware, or a combination thereof on the system that causes the system to perform an action in operation. One or more computer programs may be configured to perform a specific operation or action by including instructions that, when executed by a data processing device, cause the device to perform an action.

[0004] A general aspect may include a method for decoding a current block by decoding a motion vector (MV) category of a motion vector difference (MVD) of the current block from a compressed bitstream, wherein the MV category may indicate a set of MVDs, each MVD in the set of MVDs corresponding to a corresponding integer part, and wherein the MVD may include an integer part and a fractional part.

[0005] The method may include: obtaining MV precision of MVD; using MV precision to determine whether to omit decoding of the least significant bit in the offset bits of the integer part and whether to set the least significant bit to a predefined value; decoding at least some of the offset bits; using at least some of the offset bits and the least significant bit of the integer part to obtain the integer part; using MV precision to determine whether to omit decoding of the least significant bit in the decimal bits of the fractional part and whether to set the least significant bit to a predefined value; decoding at least some of the decimal bits of the fractional part; using at least some of the decimal bits and the least significant bit of the fractional part to obtain the fractional part; using at least the integer part and the fractional part to obtain MVD; using MVD to obtain a motion vector of the current block; and using the motion vector to obtain a prediction block for the current block. Other embodiments of this aspect include corresponding computer systems, apparatuses, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of the method.

[0006] Implementations may include one or more of the following features. A method wherein, where the MV precision indicates 4 integer pixel magnitudes, the least significant bit of the offset bits of the integer part may constitute 2 bits. Where the MV precision indicates 2 integer pixel magnitudes, the least significant bit of the offset bits of the integer part may constitute 1 bit. Where the MV precision indicates a precision no finer than 1 / 4 pixel precision, the least significant bit of the fractional part may constitute the least significant bit. Where the MV precision indicates a precision no finer than 1 / 2 pixel precision, the least significant bit of the fractional part may constitute two least significant bits.

[0007] A general aspect may include a method for decoding a current block. The method may include: decoding a motion vector (MV) class of a motion vector difference (MVD) of the current block from a compressed bitstream, wherein the MV class indicates a set of MVDs, and wherein the MVD includes an integer portion; obtaining an MV precision of the MVD; decoding at least a subset of bits representing the integer portion of the MVD using the MV precision and the MV class; obtaining the MVD using the bits representing the integer portion of the MVD; and obtaining a motion vector of the current block using the MVD.

[0008] The method may include: using the motion vector to obtain a prediction block for the current block.Other embodiments of this aspect include corresponding computer systems, apparatus, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of the method.

[0009] Implementations may include one or more of the following features. The method, wherein decoding at least a subset of the bits representing the integer portion of the MVD using the MV precision and the MV class may include inferring that at least one least significant bit of the bits representing the integer portion of the MVD is zero based on the MV precision.

[0010] Decoding at least a subset of the bits representing the integer portion of the MVD using the MV precision and the MV class may include inferring two least significant bits of the bits representing the integer portion to be zero in response to determining that the MV precision indicates a 4-integer pixel magnitude.

[0011] Decoding at least a subset of the bits representing the integer portion of the MVD using the MV precision and the MV class may include inferring a least significant bit of the bits representing the integer portion to be zero in response to determining that the MV precision indicates a 2-integer pixel magnitude.

[0012] The method may include decoding at least a subset of the bits representing the fractional portion of the MVD using the MV precision.

[0013] Using MV precision to decode at least a subset of the bits representing the fractional portion of the MVD may include inferring a least significant bit of the bits representing the fractional portion to be zero in response to determining that the MV precision indicates a precision no finer than 1 / 4 pixel precision.

[0014] Using MV precision to decode at least a subset of the bits representing the fractional portion of the MVD may include inferring two least significant bits of the bits representing the fractional portion to be zero in response to determining that the MV precision indicates a precision no finer than 1 / 2 pixel precision.

[0015] Using the MV precision to decode at least a subset of the bits representing the fractional portion of the MVD may include, in response to determining that the MV precision indicates integer pixel precision, inferring that the bits representing the fractional portion are zero.

[0016] The method may include: obtaining candidate MVs for the MV; and setting corresponding precisions of at least some of the candidate MVs as the MV precision.

[0017] Implementations of the described techniques may include hardware, a method or process, or computer software on a computer-accessible medium.

[0018] One general aspect includes a method. The method may include obtaining a motion vector (MV) precision of a motion vector difference (MVD) of a current block. The method may include decoding a subset of bits of a fractional portion of the MVD from a compressed bitstream based on the MV precision.

[0019] The method may include: setting the remaining bits of the fractional part of the MVD to zero; using at least the fractional part to obtain the MVD; using the MVD to obtain a motion vector for the current block. The method may also include: using the motion vector to obtain a prediction block for the current block.

[0020] Other embodiments of this aspect include corresponding computer systems, apparatus, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of the method.

[0021] Implementations may include one or more of the following features. A method may include: decoding an MV category of an MVD from a compressed bitstream, wherein the MV category indicates a set of MVDs, each MVD in the set of MVDs corresponding to a corresponding integer part; and decoding the integer part of the MVD from the compressed bitstream. The method may include: obtaining a candidate mv of an mv; and converting the MV precision of a candidate MV in the candidate MV to match the MV precision.

[0022] It should be understood that aspects can be implemented in any convenient form. For example, aspects can be implemented by a suitable computer program that can be carried on a suitable carrier medium, which can be a tangible carrier medium (e.g., a disk) or an intangible carrier medium (e.g., a communication signal). Aspects can also be implemented using suitable equipment, which can take the form of a programmable computer running a computer program arranged to implement the methods and / or techniques disclosed herein. Aspects can be combined so that features described in the context of one aspect can be implemented in another aspect.

[0023] These and other aspects of the present disclosure are disclosed in the following detailed description of the embodiments, the appended claims and the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The description herein refers to the drawings described below, wherein like reference numerals refer to like parts throughout the several views.

[0025] Figure 1 is a schematic diagram of a video encoding and decoding system.

[0026] Figure 2 is a block diagram of an example of a computing device that may implement a transmitting station or a receiving station.

[0027] Figure 3 is a diagram of an example of a video stream to be encoded and then decoded.

[0028] Figure 4 is the block diagram of the encoder.

[0029] Figure 5 is the block diagram of the decoder.

[0030] Figure 6 is a diagram of motion vectors representing integer-pixel and sub-pixel motion.

[0031] Figure 7 is a diagram of a sub-pixel prediction block.

[0032] Figure 8 is a diagram of integer pixel and sub-pixel locations.

[0033] Fig. 9 is an example of a flow chart of a technique for decoding a motion vector for a current block.

[0034] Fig.10 This is a block diagram illustrating the encoding and decoding of MV precision of an example block.

[0035] Fig.11 Frame data that may be included in a compressed bitstream is illustrated.

[0036] Fig.12is an example of a flow chart of a technique for decoding a motion vector for a current block.

[0037] Fig.13 is an example of a flow chart of a technique for encoding a motion vector for a current block.

[0038] Fig.14 is another example of a flow chart of a technique for decoding a motion vector for a current block.

[0039] Fig.15 is another example of a flow chart of a technique for decoding a motion vector for a current block. DETAILED DESCRIPTION

[0040] As mentioned, compression schemes associated with encoding and decoding a video stream may include decomposing an image into blocks and generating a digital video output bitstream (i.e., an encoded bitstream) using one or more techniques for limiting the information included in the output bitstream. The received bitstream may be decoded to recreate the blocks and source images from the limited information. Encoding a video stream or a portion thereof (such as a frame or block) may include using temporal and spatial similarities in the video stream to improve encoding and decoding efficiency. For example, a current block of a video stream may be encoded based on identifying a difference (residual) between previously encoded and decoded pixel values ​​and pixel values ​​in the current block or between a combination of previously encoded and decoded pixel values ​​and a combination of pixel values ​​in the current block.

[0041] Encoding using spatial similarity is called intra-frame prediction. Intra-frame prediction can attempt to predict the pixel values ​​of a block of a frame of a video stream using pixels outside the block; that is, using pixels in the same frame as the block but outside the block. Intra-frame prediction can be performed along the direction of prediction, where each direction can correspond to an intra-frame prediction mode. The intra-frame prediction mode can be signaled by the encoder to the decoder.

[0042] Encoding using temporal similarity is called inter-frame prediction or motion compensated prediction (MCP). A prediction block for a current block (i.e., the block being encoded or decoded) is generated by finding a corresponding block in a reference frame according to a motion vector (MV). That is, inter-frame prediction attempts to predict the pixel values ​​of a block using one or more blocks from one or more temporally close frames (i.e., reference frames) that may be displaced. Temporally close frames are frames that appear in a video stream that are temporally earlier or later than a frame (i.e., current frame) of the block being encoded (i.e., current block). The motion vector used to generate the prediction block references a frame (i.e., reference frame) other than the current frame (e.g., points to the frame or is used in conjunction with the frame). A motion vector can be defined as a block or pixel offset representing a corresponding block or pixel between a reference frame and a current frame.

[0043] The motion vector of the current block in the motion compensated prediction can be encoded into and decoded from the compressed bitstream. The motion vector of the current block (i.e., the block being encoded) is described with respect to the co-located block in the reference frame. The motion vector describes the offset (i.e., displacement) in the horizontal direction relative to the co-located block in the reference frame (i.e., MV x ) and displacement in the vertical direction (i.e., MV y ). Therefore, MV can be represented as a 3-tuple (f, MV x , MV y ), where f indicates a reference frame (eg, is an index of a reference frame), MV x is the horizontal offset of the co-located position relative to the reference frame, and MV y is the vertical offset of the co-location position relative to the reference frame. Therefore, the offset is at least MV x and MV y is written (ie, encoded) into a compressed bitstream, and is read (ie, decoded) from an encoded bitstream.

[0044] In order to reduce the rate cost of encoding the motion vector, the motion vector can be differentially encoded. That is, the predicted motion vector (PMV) is selected as the reference motion vector, and only the difference between the motion vector (MV) and the reference motion vector (also called the motion vector difference (MVD)) is encoded into the bitstream. For example, the reference (or predicted) motion vector can be the motion vector of one of the neighboring blocks. Therefore, DMV=MV-PMV. The neighboring blocks may include spatial neighboring blocks (i.e., blocks in the same current frame as the current block). The neighboring blocks may include temporal neighboring blocks (i.e., blocks in frames other than the current frame). The encoder encodes and decodes the DMV in the compressed bitstream; and the decoder decodes the DMV from the compressed bitstream and adds the DMV to the predicted motion vector (PMV) to obtain the motion vector (MV) of the current block.

[0045] In some cases, the prediction block that produces the best (e.g., smallest) residual may not correspond to a pixel in the reference frame. That is, the best motion vector may point to a location between pixels of the block in the reference frame. In this case, sub-pixel-level motion-compensated prediction is useful.

[0046] MCP may involve the use of a sub-pixel interpolation filter that generates filtered sub-pixel values ​​at defined locations between integer pixels (also referred to as whole pixels) along rows, columns, or both. The interpolation filter may be one of several interpolation filters available for use in motion compensated prediction. Sub-pixel interpolation is further described below.

[0047] Different interpolation filters may be available. Each of the interpolation filters may be designed to provide different frequency responses. In one example, available interpolation filters may include smoothing filters, normal filters, sharpening filters, and bilinear filters. The interpolation filters to be used by the decoder to generate prediction blocks may be signaled in the header of the frame containing the block to be predicted. Therefore, the same interpolation filter is used to generate sub-pixel prediction blocks for all blocks of the frame. The interpolation filter may be signaled at the codec unit level. Therefore, the same interpolation filter is used for each sub-block (e.g., each prediction block) of the codec unit to generate sub-pixel prediction blocks for the sub-blocks of the codec unit.

[0048] The encoder can generate a prediction block based on each of the available interpolation filters. The encoder then selects (i.e., signals to the decoder) the filter that produces, for example, the best rate-distortion ratio. The rate-distortion ratio refers to the ratio of the amount of balanced distortion (i.e., loss of video quality) to the rate (i.e., number of bits) required for encoding. The codec unit (sometimes referred to as a superblock or macroblock) can have a size of 128×128 pixels, 64×64 pixels, or some other size, and can be recursively decomposed into blocks with a size as small as 4×4 pixels (in one example).

[0049] As used herein, encoding and decoding an MV refers to encoding and decoding the motion vector itself and encoding and decoding the MVD of the motion vector. In either case, encoding and decoding an MV includes encoding and decoding the horizontal offset of the motion vector (i.e., the MV x ) for encoding and decoding and vertical shift (i.e., MV y ) for encoding and decoding, or for the horizontal offset of the motion vector difference (ie, MVD x ) for encoding and decoding and vertical shift (i.e., MVD y ) to encode and decode. When implemented by an encoder, "encoding" means encoding in a compressed bitstream. When implemented by a decoder, "encoding" means decoding from a compressed bitstream.

[0050] Encoding and decoding the motion vector may include entropy coding and decoding the horizontal offset and the vertical offset. Therefore, the context for the motion vector is determined, and the motion vector is coded and decoded using a probability model corresponding to the context. Entropy coding and decoding is a technique for "lossless" coding and decoding that relies on a probability model for modeling the distribution of values ​​that appear in the encoded video bit stream. By using a probability model based on the measured or estimated distribution of values, entropy coding and decoding can reduce the number of bits required to represent video data to close to the theoretical minimum. In practice, the actual reduction in the number of bits required to represent video data may depend on the accuracy of the probability model, the number of bits through which coding and decoding is performed, and the computational accuracy of the fixed point arithmetic used to perform coding and decoding.

[0051] Several inter prediction modes may be available. For example, one available inter prediction mode indicates that the motion vector of the block is 0. This may be called ZEROMV mode. Another inter prediction mode may indicate that the motion vector of the block is a reference motion vector. This may be called REFMV mode. When the motion vector of the block is not zero and is different from the reference motion vector, the reference motion vector may be used to encode the motion vector. This mode may be called NEWMV mode. When the inter prediction mode is NEWMV, the MVD of the MV may be encoded and decoded. Other inter prediction modes may be available.

[0052] As mentioned above, filters with different frequency responses can be used to generate motion vectors at sub-pixel positions. Therefore, and due to the use of these filters, reference blocks at different sub-pixel positions may have different characteristics in the transform domain. For example, a reference block generated by a low-pass filter at a sub-pixel position may have lower energy in the high frequency band than a reference block at an integer pixel position. Since the residual block is the difference between the source block and the reference block, the energy distribution in the residual block is related to the energy distribution of the reference block. As also mentioned above, the efficiency of entropy coding and decoding can be directly related to the probability model, which in turn is selected based on the context model.

[0053] As implied, motion vectors may be coded to (e.g., using) a particular resolution or motion vector precision (MV precision). For example, some (older) codecs may only support integer precision; the H.264 and H.265 codecs support motion vectors with 1 / 4 precision; and AV1 supports motion vector precision of 1 / 4 or 1 / 8 MV precision.

[0054] Conventional codecs may include a frame-level syntax element that indicates the MV precision (including the precision of the MVD) of the motion vectors of the inter-prediction blocks of the frame. In one example, the frame-level flag may indicate whether to use 1 / 4 or 1 / 8 MV precision to encode and decode the motion vectors of the blocks of the frame. In either case, the MV precision is at a given frame.

[0055] However, frame-level MV accuracy may not capture local (e.g., super-block level or block level) motion characteristics. Video frames may contain mixed motion, where one portion of the frame (e.g., frame background) has no motion and another portion (e.g., frame foreground) has very high motion. Therefore, encoding and decoding all motion vectors for a frame at a given MV accuracy may reduce compression efficiency because more bits will be used than would otherwise be necessary.

[0056] For example, encoding and decoding each motion vector of a frame to 1 / 8 MV precision may use a large number of codec bits, which may outweigh the benefit of using 1 / 8 MV precision for at least some of the motion vectors of the frame. That is, the bit rate required to encode and decode the motion vectors (or MVD) at a higher MV precision (e.g., 1 / 8 MV precision) may outweigh any distortion gain compared to encoding and decoding at a lower MV precision (e.g., 1 / 2 or even integer MV precision). For some types of video content (e.g., blocks within a frame), it may be sufficient to encode and decode the motion using only integer pixel MV precision and no interpolation.

[0057] Flexible signaling (codec) of MV precision is used in accordance with implementations of the present disclosure to improve coding efficiency and reduce the number of bits required to encode and decode MVs (including encoding and decoding of MVDs). MV precision can be encoded and decoded at the block level and / or the block group level. A block group can be a frame group, a frame, a super block, or some other block group. Therefore, MV precision encoding and decoding can be hierarchical, where MV precision encoded and decoded at a higher level (e.g., a frame or super block) can be used to limit the possible MV precision values ​​of a lower level (e.g., a super block or a block). The lowest level of the hierarchy is the current block (i.e., the block being predicted or encoded and decoded).

[0058] The MV precision of a lower level (e.g., block) is considered to be within or "limited by" the higher level (e.g., super block) MV precision. As further described herein, the higher level MV precision may be a maximum MV precision. The lower level MV precision being "limited by" the higher level MV precision means that the lower level MV precision is less than or equal to the higher level MV precision. For purposes of illustration, the motion vector of any sub-block of a super-block may not have an MV precision that exceeds the maximum MV precision set of the super-block. Similarly, the higher level MV precision may be a minimum MV precision. The lower level MV precision being "limited by" the higher level MV precision means that the lower level MV precision is greater than or equal to the higher level MV precision. For purposes of illustration, the motion vector of any sub-block in a frame may not have an MV precision that is less than the minimum MV precision set of the frame.

[0059] For illustration, conventionally, a motion vector may be encoded using a first bit indicating the integer (or magnitude) portion of the motion vector and a second bit indicating the fractional portion (i.e., sub-pixel precision). In one example, 12 bits may be used to encode and decode the motion vector, with 9 most significant bits (MSBs) indicating the integer portion and 3 least significant bits (LSBs) indicating the fractional portion up to 1 / 8 MV precision. The first LSB indicates whether the precision is 1 / 8. If the value of the first LSB is equal to 1, the precision is 1 / 8; otherwise, the precision is not equal to 1 / 8. When the precision is not equal to 1 / 8 (i.e., the first LSB is equal to 0), the second LSB indicates whether the precision is 1 / 4. If the first LSB is equal to 0 and the second LSB is equal to 1, the precision is 1 / 4. If the first LSB is 0 and the second LSB is 0, the precision may be an integer or 1 / 2. If both the first LSB and the second LSB are 0, the third LSB indicates whether the precision is 1 / 2 or an integer. Thus, 000, 001, 010, 011, 100, 101, 110, 111 may indicate integer, 1 / 8, 1 / 4, 1 / 8, 1 / 2, 1 / 8, 1 / 4, and 1 / 8 precision of a particular motion vector, respectively.

[0060] For example, if the motion vectors of 30 blocks of a frame require integer precision, the coded motion vectors of these blocks will include a total of 90 (30*3) 0 bits. However, if the headers of these blocks (or super blocks or frames that include these blocks) indicate (e.g., include syntax elements that indicate) that the blocks are coded using integer precision, then using flexible MV precision, as described herein, can save approximately these 90 bits (actually, less than a full 60 bits, which becomes apparent later). Bits can be saved because the decoder can infer that these bits are 0 based on the coded MV precision.

[0061] Similarly, assume that the frame header includes an MV precision indicating that blocks of the frame can be encoded or decoded using up to 1 / 2 precision. Therefore, only integer or 1 / 2 MV precision can be used to encode or decode blocks of this frame. Therefore, it is not necessary to use 2 LSBs to encode or decode the MV precision. One LSB bit is sufficient, thus saving approximately 30 (30*1=30) bits. Another bit can be inferred to be zero based on the MV precision.

[0062] In order to reduce the number of bits required to encode and decode the MV (or corresponding MVD), some codecs can encode and decode the offset values ​​within the category. The category can indicate the magnitude of the MVD. Flexible motion vector precision can be used to reduce the number of bits required to encode and decode the offset. As further described below, the MV precision of the block can limit the possible values ​​that the offset can take. Therefore, by identifying that certain offsets are impossible under given MV precision, the number of bits required to encode and decode the offset can be reduced, and the unencoded bits can be inferred. Similarly, MV precision can limit the possible values ​​that fractional precision can take. Therefore, by identifying that certain precisions are impossible under given MV precision, the number of bits required to encode and decode the fractional part can be reduced, and the unencoded bits can be inferred.

[0063] Further details of motion vector coding using motion vector precision are described herein initially with reference to a system in which motion vector coding using motion vector precision may be implemented.

[0064] Figure 1 1 is a schematic diagram of a video encoding and decoding system 100. The transmitting station 102 may be, for example, Figure 2 The computer with internal hardware configuration is described. However, other suitable implementations of the sending station 102 are possible. For example, the processing of the sending station 102 can be distributed among multiple devices.

[0065] The network 104 can connect the sending station 102 and the receiving station 106 for encoding and decoding of the video stream. Specifically, the video stream can be encoded in the sending station 102, and the encoded video stream can be decoded in the receiving station 106. The network 104 can be, for example, the Internet. The network 104 can also be a local area network (LAN), a wide area network (WAN), a virtual private network (VPN), a cellular telephone network, or any other means of transferring the video stream from the sending station 102 to the receiving station 106 (in this example).

[0066] In one example, receiving station 106 may be a Figure 2 The computer with internal hardware configuration is described. However, other suitable implementations of the receiving station 106 are possible. For example, the processing of the receiving station 106 can be distributed among multiple devices.

[0067] Other implementations of the video encoding and decoding system 100 are possible. For example, one implementation may omit the network 104. In another implementation, the video stream may be encoded and then stored for transmission to a receiving station 106 or any other device with memory at a later time. In one implementation, the receiving station 106 receives (e.g., via the network 104, a computer bus, and / or some communication pathway) the encoded video stream and stores the video stream for later decoding. In an example implementation, the Real-time Transport Protocol (RTP) is used to send the encoded video over the network 104. In another implementation, a transport protocol other than RTP may be used, such as a video streaming protocol based on the Hypertext Transfer Protocol (HTTP-based).

[0068] When used in a video conferencing system, for example, the sending station 102 and / or the receiving station 106 may include the capability to both encode and decode video streams as described below. For example, the receiving station 106 may be a video conference participant that receives an encoded video bitstream from a video conference server (e.g., the sending station 102) for decoding and viewing, and further encodes its own video bitstream and sends the video bitstream to the video conference server for decoding and viewing by other participants.

[0069] Figure 2 is a block diagram of an example of a computing device 200 (eg, a device) that may implement a sending station or a receiving station. For example, the computing device 200 may implement Figure 1 The computing device 200 may be in the form of a computing system including multiple computing devices, or in the form of a computing device (e.g., a mobile phone, a tablet computer, a laptop computer, a notebook computer, a desktop computer, etc.).

[0070] The CPU 202 in the computing device 200 may be a conventional central processing unit. Alternatively, the CPU 202 may be any other type of device or devices, now or later developed, that are capable of manipulating or processing information. Although the disclosed implementations may be practiced with one processor (e.g., CPU 202) as shown, more than one processor may be used to achieve advantages in speed and efficiency.

[0071] In one implementation, the memory 204 in the computing device 200 may be a read-only memory (ROM) device or a random access memory (RAM) device. Any other suitable type of storage device may be used as the memory 204. The memory 204 may include code and data 206 accessed by the CPU 202 using the bus 212. The memory 204 may further include an operating system 208 and an application 210, which includes at least one program that permits the CPU 202 to perform the methods described herein. For example, the application 210 may include applications 1 to N, which further include a video codec application that performs the methods described herein. The computing device 200 may also include an auxiliary storage device 214, which may be, for example, a memory card used with a mobile computing device. Because a video communication session may contain a considerable amount of information, they may be stored in whole or in part in the auxiliary storage device 214 and loaded into the memory 204 as needed for processing.

[0072] The computing device 200 may also include one or more output devices, such as a display 218. In one example, the display 218 may be a touch-sensitive display that combines a display with a touch-sensitive element operable to sense touch input. The display 218 may be coupled to the CPU 202 via the bus 212. In addition to or in lieu of the display 218, other output devices may be provided that permit a user to program or otherwise use the computing device 200. When the output device is or includes a display, the display may be implemented in various ways, including by a liquid crystal display (LCD), a cathode ray tube (CRT) display, or a light emitting diode (LED) display, such as an organic LED (OLED) display.

[0073] The computing device 200 may also include or be in communication with an image sensing device 220, such as a camera or any other image sensing device 220 now known or later developed that can sense an image, such as an image of a user operating the computing device 200. The image sensing device 220 may be positioned so that it is pointed toward the user operating the computing device 200. In one example, the position and optical axis of the image sensing device 220 may be configured so that the field of view includes an area directly adjacent to the display 218 and from which the display 218 is visible.

[0074] The computing device 200 may also include or be in communication with a sound sensing device 222, such as a microphone or any other sound sensing device now known or later developed that can sense sounds near the computing device 200. The sound sensing device 222 may be positioned so that it is directed toward a user operating the computing device 200, and may be configured to receive sounds, such as speech or other utterances, uttered by the user while the user is operating the computing device 200.

[0075] although Figure 2 The CPU 202 and memory 204 of the computing device 200 are depicted as being integrated into one unit, but other configurations may be utilized. The operation of the CPU 202 may be distributed across multiple machines (where a separate machine may have one or more of the processors), which may be directly coupled or coupled across a local area network or other network. The memory 204 may be distributed across multiple machines, such as network-based memory or memory in multiple machines that perform the operations of the computing device 200. Although depicted here as one bus, the bus 212 of the computing device 200 may be composed of multiple buses. Further, the auxiliary storage device 214 may be directly coupled to other components of the computing device 200 or may be accessed via a network, and may include an integrated unit (such as a memory card) or multiple units (such as multiple memory cards). Therefore, the computing device 200 may be implemented in a wide variety of configurations.

[0076] Figure 3 300 is a diagram of an example of a video stream 300 to be encoded and subsequently decoded. Video stream 300 includes a video sequence 302. At the next level, video sequence 302 includes several adjacent frames 304. Although three frames are depicted as adjacent frames 304, video sequence 302 may include any number of adjacent frames 304. Adjacent frames 304 may then be further subdivided into individual frames, such as frame 306. At the next level, frame 306 may be divided into a series of planes or fragments 308. For example, fragment 308 may be a subset of a frame that permits parallel processing. Fragment 308 may also be a subset of a frame that may separate video data into individual colors. For example, a frame 306 of color video data may include a luminance plane and two chrominance planes. Fragment 308 may be sampled at different resolutions.

[0077] Regardless of whether the frame 306 is divided into segments 308, the frame 306 can be further subdivided into blocks 310, which can contain data corresponding to, for example, 16×16 pixels in the frame 306. The blocks 310 can also be arranged to include data from one or more segments 308 of pixel data. The blocks 310 can also be any other suitable size, such as 4×4 pixels, 8×8 pixels, 16×8 pixels, 8×16 pixels, 16×16 pixels, or larger. Unless otherwise noted, the terms block and macroblock are used interchangeably herein.

[0078] Figure 4 4 is a block diagram of encoder 400. As described above, encoder 400 may be implemented in transmission station 102, such as by providing a computer software program stored in a memory (e.g., memory 204). The computer software program may include machine instructions that, when executed by a processor such as CPU 202, cause transmission station 102 to generate a signal in a manner that is consistent with the present invention. Figure 4 The video data is encoded in the manner described. The encoder 400 may also be implemented as dedicated hardware included in, for example, the transmission station 102. In a particularly desirable implementation, the encoder 400 is a hardware encoder.

[0079] The encoder 400 has the following stages for performing various functions in a forward path (shown by solid connected lines) to produce an encoded or compressed bitstream 420 using the video stream 300 as input: an intra / inter prediction stage 402, a transform stage 404, a quantization stage 406, and an entropy coding stage 408. The encoder 400 may also include a reconstruction path (shown by dashed connected lines) for reconstructing frames for encoding of future blocks. Figure 4 , the encoder 400 has the following stages for performing various functions in the reconstruction path: a dequantization stage 410, an inverse transform stage 412, a reconstruction stage 414, and a loop filter stage 416. Other structural variations of the encoder 400 may be used to encode the video stream 300.

[0080] When the video stream 300 is presented for encoding, the corresponding frame 304, such as frame 306, can be processed in units of blocks. At the intra / inter prediction stage 402, the corresponding block can be encoded using intra-frame prediction (also referred to as intra-prediction) or inter-frame prediction (also referred to as inter-prediction). In any case, a prediction block can be formed. In the case of intra-frame prediction, the prediction block can be formed by samples in the current frame that have been previously encoded and reconstructed. In the case of inter-frame prediction, the prediction block can be formed by samples in one or more previously constructed reference frames.

[0081] Next, refer to Figure 4 , the prediction block can be subtracted from the current block at the intra / inter prediction stage 402 to produce a residual block (also referred to as a residual). The transform stage 404 uses a block-based transform to transform the residual into, for example, a transform coefficient in the frequency domain. The quantization stage 406 uses a quantizer value or quantization level to convert the transform coefficient into a discrete quantum value, which is called a quantized transform coefficient. For example, the transform coefficient can be divided by the quantizer value and truncated. The quantized transform coefficient is then entropy encoded by the entropy encoding stage 408. The entropy encoded coefficient is then output to a compressed bit stream 420 along with other information for decoding the block, which may include, for example, the prediction type used, the transform type, the motion vector, and the quantizer value. Various techniques such as variable length coding (VLC) or arithmetic coding can be used to format the compressed bit stream 420. The compressed bit stream 420 may also be referred to as a coded video stream or a coded video bit stream, and the terms will be used interchangeably herein.

[0082] Figure 4 400 and decoder 500 (described below) use the same reference frame to decode the compressed bitstream 420. The reconstruction path in 414 (shown by the dashed connecting line) can be used to ensure that the encoder 400 and the decoder 500 (described below) use the same reference frame to decode the compressed bitstream 420. The reconstruction path performs functions similar to those performed during the decoding process discussed in more detail below, including dequantizing the quantized transform coefficients at the dequantization stage 410 and inverse transforming the dequantized transform coefficients at the inverse transform stage 412 to produce a derivative residual block (also referred to as a derivative residual). At the reconstruction stage 414, the prediction block predicted at the intra / inter prediction stage 402 can be added to the derivative residual to create a reconstructed block. A loop filtering stage 416 can be applied to the reconstructed block to reduce distortion, such as blocking artifacts.

[0083] Other variations of the encoder 400 may be used to encode the compressed bitstream 420. For example, for certain blocks or frames, a non-transform based encoder may directly quantize the residual signal without the transform stage 404. In another implementation, the encoder may have the quantization stage 406 and the dequantization stage 410 combined in a common stage.

[0084] Figure 5 2 is a block diagram of decoder 500. Decoder 500 may be implemented in receiving station 106, for example, by providing a computer software program stored in memory 204. The computer software program may include machine instructions that, when executed by a processor such as CPU 202, cause receiving station 106 to receive the received data in the form of a decoder 500. Figure 5 The decoder 500 may also be implemented in hardware included in the transmitting station 102 or the receiving station 106, for example.

[0085] Similar to the reconstruction path of the encoder 400 discussed above, in one example, the decoder 500 includes the following stages for performing various functions to produce an output video stream 516 from the compressed bitstream 420: an entropy decoding stage 502, a dequantization stage 504, an inverse transform stage 506, an intra / inter prediction stage 508, a reconstruction stage 510, a loop filtering stage 512, and a deblocking filtering stage 514. Other structural variations of the decoder 500 may be used to decode the compressed bitstream 420.

[0086] When the compressed bitstream 420 is presented for decoding, the data elements within the compressed bitstream 420 may be decoded by an entropy decoding stage 502 to produce a set of quantized transform coefficients. A dequantization stage 504 dequantizes the quantized transform coefficients (e.g., by multiplying the quantized transform coefficients by a quantizer value), and an inverse transform stage 506 inversely transforms the dequantized transform coefficients to produce a derivative residual, which may be the same as the derivative residual created by the inverse transform stage 412 in the encoder 400. Using the header information decoded from the compressed bitstream 420, the decoder 500 may use an intra / inter prediction stage 508 to create a prediction block that is the same as the prediction block created in the encoder 400 (e.g., at the intra / inter prediction stage 402). At a reconstruction stage 510, the prediction block may be added to the derivative residual to create a reconstructed block. A loop filtering stage 512 may be applied to the reconstructed block to reduce blocking artifacts.

[0087] Other filtering can be applied to the reconstructed blocks. In this example, deblocking filter stage 514 is applied to the reconstructed blocks to reduce blocking distortion, and the result is output as output video stream 516. Output video stream 516 can also be referred to as decoded video stream, and the terms will be used interchangeably in this article. Other variations of decoder 500 can be used to decode compressed bit stream 420. For example, decoder 500 can generate output video stream 516 without deblocking filter stage 514.

[0088] Figure 6 is a diagram showing motion vectors for integer pixel and sub-pixel motion. Figure 6 In the example, pixels from a reference frame 630 are used to inter-predict several blocks 602, 604, 606, 608 of a current frame 600. In this example, the reference frame 630 is a reference frame in a video sequence (such as the video stream 300) that includes the current frame 600, also known as a temporally neighboring frame. The reference frame 630 is a reconstructed frame (i.e., a frame that has been reconstructed, such as by Figure 4 The reconstructed frame may be a frame that has been encoded and decoded using the reconstruction path of the current frame 600, which has been stored in a so-called last reference frame buffer and may be used to encode and decode blocks of the current frame 600. Other (e.g., reconstructed) frames or portions of such frames may also be used for inter-frame prediction. Other available reference frames may include: a golden frame, which is another frame of the video sequence that may be selected (e.g., periodically) according to any number of techniques; and constructed reference frames, which are constructed from one or more other frames of the video sequence but are not included as decoded output (such as Figure 5 A portion of a frame is shown in the output video stream 516).

[0089] Prediction block 632 used to encode block 602 corresponds to motion vector 612. Prediction block 634 used to encode block 604 corresponds to motion vector 614. Prediction block 636 used to encode block 606 corresponds to motion vector 616. Finally, prediction block 638 used to encode block 608 corresponds to motion vector 618. In this example, each of blocks 602, 604, 606, 608 is inter-predicted using a single motion vector and therefore a single reference frame, but the teachings herein also apply to inter-prediction using more than one motion vector (such as bi-prediction and / or compound prediction using two different reference frames), where pixels from each prediction are combined in some manner to form a predicted block.

[0090] Figure 7 is a diagram of a sub-pixel prediction block. Figure 7 include Figure 6630 and the neighboring pixels of block 632. Integer pixels within reference frame 630 are shown as hollow circles. In this example, integer pixels represent reconstructed pixel values ​​of reference frame 630. Integer pixels are arranged in an array along the x-axis and the y-axis. Pixels forming prediction block 632 are shown as solid circles. Prediction block 632 is generated by sub-pixel motion along two axes.

[0091] Generating the prediction block 632 may require two interpolation operations. In some cases, generating the prediction block may require only one interpolation operation along one of the x-axis and the y-axis. The first interpolation operation is used to generate the intermediate pixels, and then the second interpolation operation is used to generate the pixels of the prediction block from the intermediate pixels. The first interpolation operation and the second interpolation operation may be respectively along the horizontal direction (i.e., along the x-axis) and along the vertical direction (i.e., along the y-axis). Alternatively, the first interpolation operation and the second interpolation operation may be respectively along the vertical direction (i.e., along the y-axis) and along the horizontal direction (i.e., along the x-axis). The first interpolation operation and the second interpolation operation may use the same interpolation filter type. Alternatively, the first interpolation operation and the second interpolation operation may use different interpolation filter types.

[0092] In order to generate pixel values ​​for sub-pixels of prediction block 632, an interpolation process may be used. In one example, an interpolation filter such as a finite impulse response (FIR) filter is used to perform the interpolation process. The interpolation filter may include a 6-tap filter, an 8-tap filter, or other number of taps. The taps of the interpolation filter weight spatially adjacent pixels (integer or sub-pixels) to generate sub-pixel values ​​using coefficient values. Generally speaking, the interpolation filters used to generate each sub-pixel value at different sub-pixel positions (e.g., 1 / 2, 1 / 4, 1 / 8, 1 / 16, or other sub-pixel positions) between two pixels are different (i.e., have different coefficient values).

[0093] Figure 8 is a diagram of the locations of integer pixels and sub-pixels. Figure 8 In the example of , a 6-tap filter is used. This means that the values ​​of sub-pixels or pixel positions 820, 822, 824 can be interpolated by applying an interpolation filter to pixels 800-810. Figure 8 Only the sub-pixel position between two pixels 804 and 806 is shown in FIG. However, the sub-pixel values ​​between other integer pixels in the row of pixels can be determined in the same manner. For example, the sub-pixel value between two pixels 806 and 808 can be determined or generated by applying an interpolation filter to pixels 802, 804, 806, 808, 810 and integer pixels adjacent to pixel 810 (if available).

[0094] Using different coefficient values ​​(regardless of their magnitude) in the interpolation filters produces different filter characteristics, and therefore different compression performance. In some embodiments, the interpolation filter set can be designed for 1 / 16 pixel precision and include at least two of a bilinear filter, an 8-tap filter (EIGHTTAP), a sharp 8-tap filter (EIGHTTAP_SHARP), or a smooth 8-tap filter (EIGHTTAP_SMOOTH). Each interpolation filter has a different frequency response.

[0095] As further described below, MV precision may be encoded for block groups and / or blocks (e.g., prediction blocks). In one example, MV precision may be encoded in a header of a block group or a header of a block. A block group may be a frame group, a frame, a fragment of a block, a tile of a block, or a superblock. More generally, a block group may be any structure for packetizing data and providing identification information for the contained data. In AV1 terminology, such a structure is referred to as an open bitstream unit (OBU).

[0096] The encoder and decoder may support the same allowed MV precision set. The allowed MV precision set may be an ordered set. Any suitable data structure may be used to implement (or represent) the allowed MV precision set. In one example, the data structure may be an enumeration given below:

[0097] In the enumeration, MV_PRECISION_4_PEL, MV_PRECISION_2_PEL, MV_PRECISION_1_PEL, MV_PRECISION_HALF_PEL, MV_PRECISION_QTR_PEL, and MV_PRECISION_EIGHTH_PEL define constants, each of which has a corresponding value (e.g., the constant MV_PRECISION_1_PEL has an integer value of 2). The constant NUM_MV_PRECISIONS is the number of allowed MV precisions in the set. Although the allowed MV precision set includes six (6) MV precisions described herein, more or fewer MV precisions are also possible. For example, the allowed MV precision set may include an MV precision MV_PRECISION_SIXTEENTH_PEL corresponding to 1 / 16 fractional pixel precision. In another example, the allowed MV precision set may also include an MV precision MV_PRECISION_8_PEL corresponding to 8 pixel precision. For example, the allowed MV precision set may not include the MV precision MV_PRECISION_EIGHTH_PEL.

[0098] The set of allowed MV precisions is ordered meaning that two MV precisions in the set can be compared to determine which is smaller or larger or if the MV precisions are equal. From the context, references to "MV precision" should be understood to mean one of the MV precision values ​​from the set of allowed MV precisions.

[0099] MV precision MV_PRECISION_EIGHTH_PEL indicates 1 / 8 MV precision. MV precision MV_PRECISION_QTR_PEL indicates 1 / 4 MV precision. MV precision MV_PRECISION_HALF_PEL indicates 1 / 2 MV precision. MV precision MV_PRECISION_1_PEL=2, MV_PRECISION_2_PEL=1, and MV_PRECISION_4_PEL indicate the magnitude of the motion vector of integer precision. When the MV precision is MV_PRECISION_2_PEL, the MV magnitude is a multiple of 2, and when the MV precision is MV_PRECISION_4_PEL, the MV magnitude is a multiple of 4. For example, if the MV precision of the block is MV_PRECISION_2_PEL, the supported motion vectors are 0, 2, 4, 6, 8, etc. Similarly, if MV precision is equal to MV_PRECISION_4_PEL, the value of the motion vector is a multiple of 4, such as 0, 4, 8, 12, 16, etc.

[0100] In one example, an MV precision list may be dynamically created before starting the decoding of the current block (i.e., the current prediction block), before starting the decoding of the current super block including the current block, before starting the decoding of the current frame including the super block, or before starting the decoding of any block group. In one example, a dynamic precision list (DPL) may be generated based on the MV precision of the previous decoding. The MV precision of the previous decoding is the MV precision of the neighboring block previously decoded. In one example, the MV precision list that the decoder will use (for the block group) with the block group may be sent in the compressed bit stream. For example, the encoder may send the MV precision that the decoder will use. For example, the encoder may encode MV_PRECISION_4_PEL and MV_PRECISION_EIGHTH_PEL in the compressed bit stream for the block group. Therefore, all motion vectors of the blocks in the block group must have one of the MV precisions MV_PRECISION_4_PEL or MV_PRECISION_EIGHTH_PEL.

[0101] The MV precision for block coding indicates the MV precision of the MV of the block (or equivalently, the MV precision of the MVD of the MV). The MV precision for block group coding may be or may indicate a limit on the MV precision of the MVD of the blocks in the block group. That is, no motion vector in a block in the block group may have an MV precision that violates the limit (e.g., is not within the limit). The limit may include a maximum MV prediction or a minimum MV precision. One of more syntax elements (e.g., a flag) may be encoded in a block such as Figure 5 The bitstream of the compressed bitstream 420 may include a value indicating whether the limit is a maximum value or a minimum value. In one example, both the maximum MV precision and the minimum MV precision may be encoded for a block group.

[0102] For illustration, assume that MV_PRECISION_HALF_PEL is encoded as the maximum MV precision of the super block. Therefore, no sub-block of the super block can have an MV precision that violates (i.e., exceeds) MV_PRECISION_HALF_PEL. Therefore, no sub-block of the super block can have any of the MV precisions MV_PRECISION_QTR_PEL or MV_PRECISION_EIGHTH_PEL. As another illustration, assume that MV_PRECISION_1_PEL is encoded as the minimum MV precision of the frame. Therefore, no block in the block group can be one of the MV precisions MV_PRECISION_2_PEL or MV_PRECISION_4_PEL. As another illustration, the maximum MV precision of the frame can be MV_PRECISION_QTR_PEL, and the minimum MV precision of the super block of the frame can be MV_PRECISION_1_PEL. Therefore, the motion vector of a sub-block of a super-block may have only one of the MV precisions MV_PRECISION_1_PEL, MV_PRECISION_HALF_PEL, or MV_PRECISION_QTR_PEL.

[0103] Fig. 9 900 is an example of a flow chart of a technique 900 for decoding a motion vector for a current block. The technique 900 may be implemented as a software program that may be executed, for example, by a computing device such as the sending station 102 or the receiving station 106. The software program may include machine-readable instructions that may be stored in a memory such as the memory 204 or the secondary storage device 214 and that, when executed by a processor such as the CPU 202, may cause the computing device to perform the technique 900. The technique 900 may be implemented in whole or in part in a computer program. Figure 5 The technique 900 may be implemented in the intra / inter prediction stage 508 of the decoder 500. The technique 900 may be implemented using dedicated hardware or firmware. Multiple processors, memories, or both may be used.

[0104] At 902, at least one of a maximum MV precision or a minimum MV precision may be obtained for a block group. In one example, a compressed bitstream (such as Figure 5 The decoder may decode at least one of the maximum MV precision or the minimum MV precision based on the compressed bitstream 420. In one example, the maximum MV precision may be inferred based on other syntax elements. In one example, the minimum MV precision may be inferred based on other syntax elements. As mentioned above, a block group may be a super block, a frame, a fragment, a tile, a super block, or some other entity, object, data structure, etc. that the decoder processes as a block group or a block in which the decoder processes.

[0105] At 904, a block-level MV precision for decoding the current block is obtained. The block-level MV precision is limited by the maximum MV precision or the minimum MV precision. That is, the block-level MV precision: when the maximum MV precision is obtained at 902, it must be less than or equal to the maximum MV precision; when the minimum MV precision is obtained at 902, it must be greater than or equal to the minimum MV precision; when both the maximum MV precision and the minimum MV precision are obtained at 902, both situations are true.

[0106] For illustration, assume that the group block is a super block. The maximum MV precision of the super block can be denoted as max_mb_precision, the minimum MV precision of the super block can be denoted as min_mb_precision, and the block-level MV precision can be denoted as pb_mv_precision. For clarity, pb_mv_precision indicates the precision of the MVD of the predicted block (i.e., the current block to be predicted) (or equivalently, the MV). In one example, the block-level MV precision can be signaled at the super block level or at the predicted block level of the sub-block of the super block. The header of the frame (or the super block itself) can include a syntax element (e.g., a flag) that indicates whether the maximum MV precision or the minimum MV precision is signaled at the super block level.

[0107] Fig.10 This is a block diagram illustrating the encoding and decoding of MV precision of an example block. Fig.10 Frame data 1000 that may be included in a compressed bitstream is illustrated. Frame data 1000 may include super block data, such as super block data 1001. Frame data 1000 may include other data 1002A-1002G that may not be relevant to the description of the codec of MV precision.

[0108] The header of the frame may include a first flag 1004 and a second flag 1006. The first flag 1004 may indicate whether super block data of the frame data 1000 (e.g., a header of a super block of the frame) includes a corresponding syntax element min_mb_precision. If the first flag 1004 has a specific value (e.g., 1, as illustrated), the super block data includes the corresponding syntax element min_mb_precision; otherwise, the super block data will not include the corresponding syntax element min_mb_precision. Similarly, the second flag 1006 may indicate whether the super block data includes the corresponding syntax element max_mb_precision. If the second flag 1006 has a specific value (e.g., 1, as illustrated), the super block data includes the corresponding syntax element max_mb_precision; otherwise, the super block data will not include the corresponding syntax element max_mb_precision. In some examples, only one of the first flag or the second flag may be included in the frame data 1000.

[0109] When the first flag is illustrated as being equal to a particular value (e.g., 1), the super block data 1001 includes a syntax element min_mb_precision (i.e., syntax element 1008); and when the second flag is illustrated as being equal to a particular value (e.g., 1), the super block data 1001 includes a syntax element max_mb_precision (i.e., syntax element 1010). For a particular block in the super block, the block data may include a syntax element 1012 (i.e., pb_mv_precision) indicating the precision of the motion vector of the block. The MV precision of the block is limited to the maximum MV precision max_mb_precision and / or the minimum MV precision min_mb_precision, whichever is included in the super block data. That is, min_mb_precision ≤ pb_mv_precision ≤ max_mb_precision.

[0110] In the case where only max_mb_precision is included and max_mb_precision is equal to the smallest possible MV precision value (e.g., MV_PRECISION_4_PEL), then the block data 1003 may not include the MV precision of the block because pb_mv_precision can be inferred to be equal to max_mb_precision. Similarly, in the case where only min_mb_precision is included and min_mb_precision is equal to the largest possible MV precision value (e.g., MV_PRECISION_EIGHTH_PEL), then the block data 1003 may not include the MV precision of the block because pb_mv_precision can be inferred to be equal to min_mb_precision.

[0111] Although Fig.10 Although not specifically shown and as described above, the frame data 1000 may include a frame-level maximum MV precision (eg, max_frm_precision) and a frame-level minimum MV precision (eg, min_frm_precision) of the frame. Fig.11 is a block diagram illustrating another example of MV precision encoding and decoding of a block. Fig.11 Frame data 1100 that may be included in a compressed bitstream is illustrated. Frame data 1100 may include super block data, such as super block data 1101. Super block data may include block data, such as block data 1103. Frame data 1100 may include other data illustrated by boxes filled with ellipsis and that may not be relevant to the description of the codec of MV precision.

[0112] The frame data 1100 includes a first flag 1102 indicating whether the frame data 1100 includes min_frm_precision and a second flag 1104 indicating whether the frame data 1100 includes max_frm_precision. The first flag 1102 and the second flag 1104 are illustrated as having a value (i.e., 1) indicating that the frame data 1100 includes the syntax element min_frm_precision (i.e., syntax element 1106) and the max_frm_precision (i.e., syntax element 1108). The flag 1110 having a zero value of the super block data 1101 illustrates that the super block data 1101 does not include the syntax element min_frm_precision (i.e., syntax element 1106) and the max_frm_precision (i.e., syntax element 1108). Fig.101008 of the Super Block. Thus, the min_mb_precision value for this Super Block may be assumed to be (e.g., inferred to be, set to, etc.) the value of min_frm_precision. A flag 1112 of the Super Block Data 1101 and having a value indicates that the Super Block Data 1101 includes a maximum MV precision max_mb_precision. Thus, the Super Block Data 1101 includes a syntax element 1114 for max_mb_precision. For a particular block in the Super Block, the Block Data may include a syntax element 1116 (i.e., pb_mv_precision) indicating the precision of the motion vectors for that block.

[0113] As can be appreciated from the foregoing, other arrangements of syntax elements and flags are possible.

[0114] In one example, a block-level maximum MV precision may be inferred based on the size of the block. For example, a small block (e.g., 4×4 or 8×8) that may be characterized by or associated with a high motion level may be associated with a motion vector having a large number of values ​​but lower precision. The MV precision of the motion vectors of such blocks may be no greater than a threshold maximum MV precision. The threshold maximum MV precision may be MV_PRECISION_HALF_PEL or some other MV precision. More generally, if the size of the codec block is less than or equal to a minimum threshold block size, the MV precision may be inferred to be a particular MV precision.

[0115] Similarly, large blocks (e.g., 64x64 or 128x128) that may be characterized by or associated with slow motion may be associated with motion vectors having small magnitudes but higher precision. The MV precision of the motion vectors for such blocks may be no less than a threshold minimum MV precision. The threshold minimum MV may be MV_PRECISION_QTR_PEL or some other MV precision. More generally, if the size of the codec block is greater than or equal to the maximum threshold block size, the MV precision may be inferred to be a particular MV precision.

[0116] In one example, at least one of the maximum MV precision (max_mb_precision) or the minimum MV precision (min_mb_precision) may be inferred based on the inter prediction mode of the block. For example, a composite inter prediction using 2 new motion vectors may be associated with a motion vector having a higher precision. In this context, composite prediction refers to obtaining two prediction blocks using 2 different MVs and combining them to obtain a prediction block. Each of the MVs may be as described above with respect to NEWMV. The MV precision of the motion vector of such a compositely predicted block may not be less than a threshold minimum MV precision, which may be the same as or different from the threshold minimum MV precision described above. Other associations between the inter prediction mode and the threshold minimum MV precision or the threshold maximum MV precision are possible.

[0117] According to the foregoing, entropy coding can be used to code the precision pb_mv_precision of the motion vector of the prediction block. In one example, pb_mv_precision can be coded using one symbol. A context for coding the syntax element (ie, symbol) pb_mv_precision is determined, and a probability model corresponding to the context is used for pb_mv_precision.

[0118] As is well known, entropy coding is a technique for "lossless" coding that relies on a probability model for modeling the distribution of values ​​that appear in the encoded video bitstream. By using a probability model based on the measured or estimated distribution of values, entropy coding can reduce the number of bits required to represent video data to a value close to the theoretical minimum. In practice, the actual reduction in the number of bits required to represent video data can depend on the accuracy of the probability model, the number of bits through which coding and decoding are performed, and the computational accuracy of the fixed point arithmetic used to perform coding and decoding. The purpose of context modeling is to obtain the probability distribution of subsequent entropy coding and decoding engines (such as arithmetic coding and decoding, Huffman coding and decoding, and other variable length to variable length coding and decoding engines). In order to achieve good compression performance, a large number of contexts may be required. For example, some video coding and decoding systems can include hundreds or even thousands of contexts only for transform coefficient coding and decoding. Each context can correspond to a probability distribution.

[0119] In one example, the context of the entropy symbol may be derived using at least one of the maximum and / or minimum allowed MV precisions of the hierarchical structure including the block. For example, the context may be derived using at least one max_mb_precision or min_mb_precision (if available). For example, the context may be derived using at least one of (additionally or alternatively and if available) max_frm_precision or min_frm_precision. The context may be additionally derived using the MV precision of the neighboring blocks of the block. The neighboring blocks may include spatial neighboring blocks. The spatial neighboring blocks may be or include the upper and left neighboring blocks. However, other neighboring blocks are possible. Therefore, in one example, the context of the entropy symbol may be derived from the maximum allowed MV precision of the block and the MV precision of the neighboring blocks (e.g., the top and left neighboring blocks).

[0120] In one example, the encoder may signal a flag (denoted as pb_mv_precision_same_as_max_precision_flag), and the decoder may decode the flag to indicate whether pb_mv_precision is the same as max_mb_precision. If pb_mv_precision_same_as_max_precision_flag has a value of 1, the value of pb_mv_precision may be set to the same value as max_mv_precision. If the value of pb_mv_precision_same_as_max_precision_flag is equal to 0, an additional symbol may be signaled (i.e., encoded by the encoder and decoded by the decoder) to indicate the MV precision value. The additional symbol may be the difference between the max_mb_precision-1 value and the pb_mb_precision value. Table I illustrates pseudo code for differential encoding and decoding of pb_mv_precision when pb_mv_precision_same_as_max_precision_flag is used.

[0121] In one example, the encoder may signal a flag (denoted as pb_mv_precision_same_as_min_precision_flag) and the decoder may decode the flag to indicate whether pb_mv_precision is the same as min_mb_precision. Table II illustrates pseudo code for differential encoding and decoding of pb_mv_precision when pb_mv_precision_same_as_min_precision_flag is used.

[0122] In another example, the MV precision of a block can be predicted from the MV precision of neighboring blocks. For example, the predicted MV precision pred_mv_precision can be set equal to the maximum (or minimum) MV precision of the top and left (or other) neighboring blocks. For the predicted block, a flag (pb_mv_precision_same_as_pred_mv_precision_flag) can be signaled to indicate whether pb_mv_precision is the same as pred_mv_precision. Table III illustrates pseudo code for differential encoding and decoding of pb_mv_precision when pb_mv_precision_same_as_pred_mv_precision_flag is used.

[0123] In one example, and as mentioned, a DPL of MV precision can be created (e.g., constructed) from the MV precision of neighboring blocks of the current block. The neighboring blocks used to obtain the DPL may include blocks in zero or more rows above the current block, blocks in zero or more columns to the left of the current block, or a combination thereof. The DPL includes a list of MV precisions supported for the current prediction block. That is, the MV precisions included in the DPL are limited to any maximum MV precision and / or minimum MV precision among the MV precisions of the current block. In the DPL, the most likely MV precision may be placed at index 0, and the least likely MV precision may be placed at the maximum index.

[0124] The MV accuracies may be sorted according to the number of times they are used by neighboring blocks (from most likely to least likely); and in case of a tie, the MV accuracies may be sorted based on the distance of the neighboring blocks from the current block. The distance between the current block and the neighboring block may be measured as the distance between the co-located pixels (e.g., the top left pixel) of the blocks.

[0125] The compressed bitstream may include in the DPL an index of the MV precision to be used for pb_mv_precision. For illustration, and without loss of generality, the DPL may include MV_PRECISION_1_PEL, MV_PRECISION_EIGHTH_PEL, MV_PRECISION_4_PEL, and MV_PRECISION_QTR_PEL in this order. The compressed bitstream may include index=1 into the DPL. Thus, the MV precision pb_mv_precision of the current block may be set to MV_PRECISION_EIGHTH_PEL.

[0126] In one example, the DPL includes all allowed precisions for a block. If some allowed MV precisions are not found from neighboring blocks, those precisions can be placed at the end of the DPL list. In another example, if the DPL does not include the MV precision for a block, the difference between the most likely MV precision and the actual MV precision (i.e., pb_mv_precision) can be encoded in the compressed bitstream.

[0127] In one example, the generation of the DPL may also depend on at least one of a block size, a prediction mode, a motion model, or a motion vector candidate list. For example, if the neighboring block has a size that is not within a size threshold of the current block, the MV precision of the block may not be added to the DPL list. For example, if the motion vector of the neighboring block is not a candidate motion vector of the current block, the MV precision of the current block may not be added to the DPL list.

[0128] In one example, and as mentioned above, the MV precision of the current block can be inferred based on the size of the block. For example, if the block is determined to be a small block (e.g., 4×4 or 8×8), it can be assumed that pb_mv_precision is MV_PRECISION_EIGHTH_PEL, and pb_mv_precision does not need to be decoded from the compressed bitstream (or encoded in the compressed bitstream).

[0129] In one example, the MV accuracy of the current block can be inferred based on the prediction mode of the block. For illustration, the codec can support several motion modes. The motion mode (i.e., inter-frame prediction mode) indicates the type of motion compensation to be performed to obtain the prediction block for the current block. The inter-frame prediction mode of the current block can be decoded from the compressed bitstream. For example, the AV1 codec supports three motion modes (signaled using the syntax element motion_mode): overlapped block motion compensation (OBMC), local warping model (LOCALWARP), and simple translation model (SIMPLE). If the motion mode of the current block is not the simple translation model (SIMPLE), the MV accuracy is not decoded and is inferred to the maximum accuracy value (e.g., MV_PRECISION_EIGHTH_PEL). Translational motion is characterized as a pixel shift on the x-axis and y-axis of the prediction block in the reference frame compared to the block being predicted.

[0130] As mentioned, interpolation filters may be used to obtain sub-pixel values. Some codecs may encode a particular interpolation filter that a decoder will use to calculate sub-pixel values. For example, an encoder may encode (and a decoder may decode) a syntax element indicating an interpolation filter to be used for the current block. The syntax element may be an index of an interpolation filter among the available interpolation filters.

[0131] In some examples, if the MV precision pb_mv_precision of the current block is less than a predefined MV precision threshold (e.g., MV_PRECISION_1_PEL), then no interpolation filter is signaled for the predicted block, but the interpolation filter can be inferred to be a filter designated as a default filter. The default filter is an interpolation filter among the available interpolation filters that the encoder and decoder are configured to use as a default. Each of the available interpolation filters can have a different cutoff frequency and can be designed to cope with various types of noise and / or distortion that can occur in a reference frame or block.

[0132] In another example, if the MV precision of the current block does not have sub-pixel precision, the interpolation filter is not signaled. That is, in the case where the MV precision cannot be an MV precision indicating sub-pixel precision, the compressed bitstream will not include syntax elements indicating the interpolation filter. For example, if the MV precision is one of MV_PRECISION_4_PEL, MV_PRECISION_2_PEL, or MV_PRECISION_0_PEL, the interpolation filter is not signaled. As mentioned, the MV precision can be a block-level MV precision or a block group MV precision. That is, if pb_mv_precision is decoded as integer MV precision or is inferred as integer MV precision, the compressed bitstream will not include an indication of the interpolation filter of the current block.

[0133] In one example, a particular interpolation filter may be associated with a corresponding MV precision. For illustration, and without loss of generality, the MV precision MV_PRECISION_EIGHTH_PEL may be associated with an interpolation filter called EIGHTTAP_SMOOTH. Thus, if MV_PRECISION_EIGHTH_PEL is decoded as pb_mv_precision for the current block, the EIGHTTAP_SMOOTH filter will be used.

[0134] In another example, more than one interpolation filter can be associated with an MV precision value. For example, assume that MV_PRECISION_QTR_PEL is associated with interpolation filters called EIGHTTAP_SMOOTH and EIGHTTAP_SHARP. Further assume that four interpolation filters are available and are ordered so that EIGHTTAP_SMOOTH and EIGHTTAP_SHARP are associated with orders 2 and 3, respectively. Therefore, 2 bits will be required to encode and decode a particular interpolation filter. However, if MV_PRECISION_QTR_PEL is determined as the value of pb_mv_precision, only one bit (instead of two) will be required to indicate whether the interpolation filter to be used is an EIGHTTAP_SMOOTH filter or an EIGHTTAP_SHARP filter.

[0135] Return to Fig. 9 At 906, the MV of the current block is decoded using block-level MV precision. As mentioned above, decoding the MV may mean or include decoding the DMV, which may be added to the predicted MV (PMV). Decoding the MV is further described below. At 908, the MV is used to obtain a prediction block for the current block. This may be as described with respect to Figure 5The intra / inter prediction stage 508 is used to obtain the predicted block.

[0136] For simplicity, the description herein may refer to the encoding and decoding of motion vectors. However, encoding and decoding motion vectors includes separately encoding and decoding the horizontal offset of the motion vector (i.e., MV x ) and vertical displacement (i.e., MV y ) for encoding and decoding. As mentioned, instead of signaling the motion vector itself, the MVD value can be signaled in the bitstream. In some codecs, the integer pixel part of the MVD can be signaled first, followed by the fractional (sub-pixel) part of the MVD.

[0137] The integer pixel part of the MVD can be divided into a predefined number of categories (MV categories). For each prediction block, the MV category can be signaled first, followed by one or more bits (starting from the LSB to the most significant bit (MSB)) that specify the integer part of the offset, which is the difference between the MVD and the start magnitude of the MV category of the MVD. The number of bits required to encode the offset depends on the category and the number of MVDs represented by the category.

[0138] In other words, in order to reduce the number of bits required for encoding and decoding the MVD, the magnitude of the MVD can be divided into categories, wherein each category includes a certain number of MVDs. The encoder can classify the MVD into the selected MV category among the available MV categories. In another example, each of the horizontal offset and the vertical offset can be separately classified into the corresponding MV category. The MV category can be selected based on the magnitude of the MVD. Selecting the MV category can include selecting the MV category for the category horizontal offset, and separately selecting the MV category for the vertical offset. In each of the available MV categories, several MVDs may be available. In one example, the MV category can be selected based on both the horizontal offset and the vertical offset of the DMV. Therefore, encoding and decoding the MVD includes encoding and decoding the MV category of the MVD, and encoding and decoding the offset within the category indicating a specific MVD within the category.

[0139] For illustration, and without loss of generality, the available MV classes may include classes labeled class0, class1, class2, class3, etc. corresponding to integer values ​​0, 1, 2, 3, etc. Any number of classes may be available. In one example, 11 classes are available. Class i in these MV classes may include (or represent) a 2 i+1 A certain number of MVDs (num_mvds i ) MVD of MV category i. The base MVD of MV category i (base_mvd i) (e.g., the first MVD represented by an MV category) may be the MVD following the last MVD (last_mvd) represented by MV category i-1; and the last MVD (last_mvd i ) can be represented by the base MVD (base_mvd i ) plus the number of MVDs represented by that class. Thus, for illustration, for class3 (i.e., i=3), the number of MVDs represented by that class is 16; the first MVD (i.e., base_mvd) is 14; and the last MVD represented by that class is 29. That is, class3 represents MVDs 14 to 29. These relationships can be symbolically given by: , where base_mvd 0 =0, and last_mvd 0 =1

[0140] Conventionally, encoding and decoding an MVD includes encoding and decoding an MVD class (i.e., MV class) and encoding and decoding an offset indicating a specific MVD within the class. For illustration, assume that the MVD is given by class MV class3 (described above) and offset 7 within the class. Therefore, to encode and decode the MVD, bit 11 (indicating MV class=3) and bit 0111 (indicating offset 7) will be used. Note that since the maximum number of offsets that can be required for encoding and decoding for this class is 16, 4 bits are used to encode and decode the offset.

[0141] Using the signaled (ie, encoded) or inferred MVD precision for the current block, the corresponding number of bits required to encode and decode the integer and / or fractional parts of the MVD may be reduced.

[0142] Fig.12 1 is an example of a flow chart of a technique 1200 for decoding a motion vector for a current block. The technique 1200 may be implemented as a software program that may be executed, for example, by a computing device such as the sending station 102 or the receiving station 106. The software program may include machine-readable instructions that may be stored in a memory such as the memory 204 or the secondary storage device 214 and that, when executed by a processor such as the CPU 202, may cause the computing device to perform the technique 1200. The technique 1200 may be implemented in whole or in part in a computer system. Figure 5The technique 1200 may be implemented in the entropy decoding stage 502 or the inter / intra prediction stage 508 of the decoder 500. The technique 1200 may be implemented using dedicated hardware and firmware. Multiple processors, memories, or both may be used.

[0143] Technique 1200 may be performed when decoding an inter-predicted current block. Technique 1200 may be performed to obtain the components of the motion vector of the current block. Thus, technique 1200 may be performed a first time to obtain the horizontal component of the motion vector and a second time to obtain the vertical component of the motion vector.

[0144] Technique 1200 may be performed to use a compressed bitstream such as Figure 5 The prediction block is obtained by using a motion vector decoded from a compressed bitstream 420) (ie, using data decoded from the compressed bitstream). The motion vector is differentially coded.

[0145] At 1202, an MV class of an MVD of a current block is decoded from a compressed bitstream. As described above, the MV class may indicate or include a set of MVDs; and the MVD may include at least one of an integer portion or a fractional portion. At 1204, an MV precision of the MVD is obtained. The MV precision may be obtained as described above. The MV precision may be as described above with respect to pb_mv_precision. In one example, the MV precision may be decoded from the compressed bitstream, as described above. In one example, the MV precision may be inferred, also as described above.

[0146] At 1206, at least a subset of bits indicating an integer portion of the MVD is decoded from the compressed bitstream using the MV precision and the MV category. More specifically, the bits indicating the integer portion are bits indicating an offset, which is a difference between the integer portion of the MVD and a starting magnitude of the MV category of the MVD. Although a total of N bits may be used to represent the integer portion of the MVD, M bits (M≤N) may be decoded and the remaining NM bits inferred to be 0. Table IV illustrates pseudo code for decoding at least a subset of bits representing the integer portion of the MVD using the MV precision and the MV category.

[0147] At row 1, the offset is initialized to zero. Rows 3-5 of Table IV determine the number of bits of the offset that can be inferred. The variable start_lsb indicates the number of LSB bits of the integer portion that can be inferred. If the MV precision is MV_PRECISION_4_PEL, then the two least significant bits can be inferred to be zero, and decoding of the bits of the offset can begin after those two bits. Therefore, in response to determining that the MV precision indicates a 4-integer pixel magnitude, the two least significant bits of the bits representing the integer portion can be inferred to be zero.

[0148] If the MV precision is MV_PRECISION_2_PEL, then the least significant bit can be inferred to be zero, and decoding of the offset bits can begin after that bit. Thus, in response to determining that the MV precision indicates a 2-integer pixel magnitude, it can be inferred that one of the least significant bits (i.e., the least significant bit) of the bits representing the integer portion is zero.

[0149] The loop in lines 6-9 decodes a number of bits from the bitstream equal to the number of bits corresponding to the MV class minus the number of inferred bits. After decoding each bit, it is added to the bit representing the offset by right-shifting the mv_bit by the loop index i corresponding to the current bit position of the offset and ORing the right-shifted bit with the current value of the offset. Thus, at least a subset of the bits representing the fractional portion of the MVD can be decoded using MV precision.

[0150] At 1208, the bits representing the integer portion of the MVD are used to obtain the MVD. For example, the sign data (described below), the integer portion, and the fractional (i.e., sub-pixel) portion data (described below), if any, may be combined to obtain the MVD itself. At 1210, the MVD is used to obtain a motion vector for the current block. As described above, the MVD may be added to the PMV to obtain the motion vector. At 1212, the motion vector may be used to obtain a prediction block for the current block. This may be performed as described above. Figure 5 The intra / inter prediction stage 508 is used to obtain the predicted block.

[0151] In one example, the technique 1200 may also include decoding whether the value of the MVD is zero. If the value of the MVD is not zero, the technique 1200 may decode the symbol data of the MVD and proceed to perform steps 1202-1208. If the value of the MVD is zero, the technique 1200 does not perform steps 1202-1208.

[0152] In one example, the MV precision may indicate that the MVD includes a fractional component; and the technique 1200 may further include using the MV precision to decode at least a subset of the bits representing the fractional portion of the MVD. For example, the codec may be designed to use a number of bits (e.g., 3 bits) to represent the fractional component of the MVD. The bits indicate the sub-pixel precision of the MVD. The MV precision may be used to infer at least some of the fractional bits. Therefore, based on the MV precision pb_mv_precision, fewer bits indicating the fractional component need to be included in the compressed bitstream. Table V illustrates pseudocode for decoding the bits indicating the fractional bits (fr_bits) of the MVD.

[0153] At line 1, the decimal places are initialized to 0. At lines 2-3, if the MV precision is greater than MV_PRECISION_1_PEL, fr_bits is updated to the value resulting from ORing the current value of fr_bits with a bit (fr_bit) decoded from the compressed bitstream and shifted left by 2 positions. At lines 4-5, if the MV precision is greater than MV_PRECISION_HALF_PEL, fr_bits is updated to the value resulting from ORing the current value of fr_bits with a bit (fr_bit) decoded from the compressed bitstream and shifted left by 1 position. At lines 6-7, if the MV precision is greater than MV_PRECISION_QTR_PEL, fr_bits is updated to the value resulting from ORing the current value of fr_bits with a bit (fr_bit) decoded from the compressed bitstream. Although not explicitly shown in Table V, if pb_mv_precision is less than or equal to MV_PRECISION_1_PEL, no bits are decoded from the compressed bitstream for the MVD fractional part, and the fractional part is set to 0 (ie, fr_bits = 0, at row 1).

[0154] Therefore, in response to determining that the MV precision indicates a precision no finer than 1 / 4 pixel precision, it can be inferred that the least significant bit of the bits representing the fractional portion is zero. Similarly, in response to determining that the MV precision indicates a precision no finer than 1 / 2 pixel precision, it can be inferred that the two least significant bits of the bits representing the fractional portion are zero. Additionally, in response to determining that the MV precision indicates integer pixel precision, it can be inferred that all bits representing the fractional portion are zero.

[0155] As mentioned above, the motion vector of the current block can be differentially encoded relative to the predicted motion vector. This can be referred to as predictive coding of the motion vector. The predicted motion vector (i.e., PMV) can be selected from a list of candidate motion vectors (i.e., a list of candidate motion vectors) ref_mv_stack[]. The encoder and decoder of the codec can use the same rules or heuristics to construct a list of motion vector candidates ref_mv_stack[] for the current block. The encoder encodes the index ref_mv_idx of the motion vector candidate in the list that is the PMV; and the decoder decodes the index to obtain the PMV from the list (i.e., ref_mv_stack[ref_mv_idx]). As mentioned, the decoder also decodes the MVD and uses MV = MVD + ref_mv_stack[ref_mv_idx] to obtain the motion vector of the block. Several techniques are known for obtaining a list of motion vector candidates from the spatial and temporal neighbors of the current block, and the disclosure herein is not limited to any particular way of obtaining such a list.

[0156] At least some of the candidate motion vectors of ref_mv_stack may have MV precision inconsistent with the MV precision pb_mv_precision of the current block. That is, the MV precision of such blocks may be higher (i.e., greater than) or lower (i.e., less than) the pb_mv_precision of the motion vector of the current block. In some implementations according to the present disclosure, the MV precision of such candidate motion vectors may be converted to the MV precision pb_mv_precision of the prediction block.

[0157] For illustration, and without loss of generality, assume that pb_mv_precision is MV_PRECISION_4_PEL, but one of the candidate motion vectors has an MV precision of MV_PRECISION_QTR_PEL. Therefore, the MV precision of this candidate motion vector is rounded up to MV_PRECISION_4_PEL. Therefore, the corresponding MV precisions can be maintained (e.g., stored in a memory) in association with the candidate motion vectors so that these maintained MV precisions can be converted to match the MV precision pb_mv_precision of the current block. That is, the corresponding precisions of at least some of the candidate MVs are set (e.g., rounded up or down) to the MV precision pb_mv_precision of the current block. At least some of the candidate MVs whose MV precisions are set (e.g., converted) are those candidate MVs that are inconsistent with the MV precision pb_mv_precision of the current block.

[0158] Alternatively or additionally, converting the MV precision of the motion vector may include altering the motion vector itself (or a copy thereof). For illustration, and without loss of generality, as discussed above, the 3 LSBs of the motion vector may be used to indicate the fractional precision of the motion vector. If pb_mv_precision is MV_PRECISION_1_PEL, the 3 LSBs of the candidate motion vector (or a copy thereof) may be set to 0. More generally, bits of the candidate motion vector indicating a precision (i.e., magnitude and / or fractional precision) greater than pb_mv_precision may be set to 0.

[0159] The above description mainly describes the operation of the decoder. However, and as can be understood, parallel operations can be performed by the encoder. For example, the decoder decodes syntax elements from a compressed bitstream, while the encoder encodes such syntax elements in the compressed bitstream (or signals such syntax elements in the compressed bitstream); and the decoder omits decoding certain syntax elements, while the encoder omits encoding such syntax elements in the compressed bitstream (i.e., does not include or signal such syntax elements in the compressed bitstream). Encoding and decoding (e.g., encoding or decoding) syntax elements means encoding and decoding the values ​​of the syntax elements.

[0160] therefore, Fig.13 1300 for encoding a motion vector for a current block. Technique 1300 may be implemented as, for example, a software program that may be executed by a computing device such as transmitting station 102 or receiving station 106. The software program may include machine-readable instructions that may be stored in a memory such as memory 204 or auxiliary storage device 214 and that, when executed by a processor such as CPU 202, may cause the computing device to perform technique 1300. Technique 1300 may be implemented in whole or in part in Figure 4 The technique 1300 is implemented in the inter / intra prediction stage 408 of the encoder 400. The technique 1300 can be implemented using dedicated hardware and firmware. Multiple processors, memories, or both can be used.

[0161] At 1302, at least one of the maximum MV precision or the minimum MV precision of the block group is obtained. At 1304, a block-level MV precision limited to at least one of the maximum MV precision or the minimum MV precision is obtained for encoding the current block. For example, the encoder can select at least one of the maximum MV precision or the minimum MV precision and the block-level MV precision based on the block group and the content of the current block being encoded. For example, the encoder can evaluate rate-distortion information, bit budget or other factors. However, the teachings of this article are not limited in any way to any technology used by the encoder to select the maximum MV precision of the block group, the minimum MV precision of the block group or the block-level MV precision.

[0162] Although Fig.13It is not specifically shown in the figure, but the encoder can encode zero or more of the maximum MV precision, the minimum MV precision, or the block-level MV precision. As mentioned above, in some cases, some of the maximum MV precision, the minimum MV precision, or the block-level MV precision can be inferred by the decoder. Therefore, the encoder does not need to encode them in the compressed bit stream. In one example, the encoder can encode the MV of the block group based on the MV precision of the blocks in the block group. For example, the encoder can perform a first pass of encoding all the blocks in the block group (for example, predicting without writing to the compressed bit stream). The encoder can use the first pass encoding data (for example, results, statistics, etc.) to obtain the block group level MV precision.

[0163] At 1306, the MVD of the current block is encoded using block-level MV precision. Encoding the MV is similar to encoding Fig. 9 The decoding of the MV is performed in parallel. At 1308, a prediction block for the block is obtained. The prediction block can be used to obtain a residual block that can be encoded in a compressed bit stream, such as Figure 4 described.

[0164] With respect to Table IV, the encoder may perform parallel operations to encode the bits corresponding to the offsets of the MVD in the compressed, as illustrated in Table VI.

[0165] With respect to Table V, the encoder may also perform parallel operations to encode bits corresponding to the fractional component of the MVD based on the MV precision, as illustrated in Table VII. Given an MVD calculated (e.g., determined, selected, etc.) by the encoder (where the 3 LSBs of the MVD indicate the fractional precision), the pseudo-code of Table VII illustrates the skipping of encoding of bits that may be inferred based on the MV precision of the current block (i.e., the MV precision of the motion vector or MVD of the current block).

[0166] At rows 1-2, if the MV precision of the current block (i.e., the MVD of the current block) is greater than MV_PRECISION_1_PEL, the third LSB indicating whether the MVD precision is 1 / 2 precision is encoded in the bitstream. At rows 3-4, if the MV precision of the MVD precision of the MVD of the current block is greater than MV_PRECISION_HALF_PEL, the second LSB indicating whether the MVD precision is 1 / 4 precision is encoded in the bitstream. At rows 5-6, if the MV precision of the current block is greater than MV_PRECISION_QTR_PEL, the LSB indicating whether the MVD precision is 1 / 8 precision is encoded in the bitstream. Although not explicitly shown in Table VII, if pb_mv_precision is less than or equal to MV_PRECISION_1_PEL, no bits are encoded in the compressed bitstream for the fractional portion of the MVD.

[0167] Fig.14 14 is another example of a flow chart of a technique 1400 for decoding a motion vector of a current block. The technique 1400 decodes an MVD for obtaining an MV of a current block. The MVD includes an integer part and a fractional part. The technique 1400 obtains the integer part and the fractional part of the MVD.

[0168] Technique 1400 may be implemented, for example, as a software program that may be executed by a computing device such as sending station 102 or receiving station 106. The software program may include machine-readable instructions that may be stored in a memory such as memory 204 or secondary storage device 214 and that, when executed by a processor such as CPU 202, may cause the computing device to perform technique 1400. Technique 1400 may be implemented in whole or in part in a computer system. Figure 5 The technique 1400 is implemented in the inter / intra prediction stage 508 of the decoder 500. The technique 1400 can be implemented using dedicated hardware and firmware. Multiple processors, memories, or both can be used.

[0169] At 1402, an MV category of an MVD is decoded from a compressed bitstream, which may be Figure 5 The compressed bit stream 420 is obtained. As described above, the MV class indicates a set of MVDs, where each MVD in the set of MVDs corresponds to a corresponding integer part. At 1404, the MV precision of the MVD is obtained. The MV precision can be obtained as described above. For example, the MV precision can be obtained as described with respect to Fig.12 MV accuracy is obtained as described in 1204.

[0170] At 1406, the technique 1400 uses the MV precision to determine whether to omit decoding of the least significant bit in the offset bits of the integer portion and whether to set the least significant bit to a predefined value. In one example, using the MV precision to determine whether to omit decoding of the least significant bit in the offset bits of the integer portion and whether to set the least significant bit to a predefined value can be as described in the pseudo code with respect to Table IV, such as according to lines 3-5. Thus, in the case where the MV precision indicates a 4-integer pixel magnitude, the least significant bit in the offset bits of the integer portion can be or constitute 2 bits; in the case where the MV precision indicates a 2-integer pixel magnitude, the least significant bit in the offset bits of the integer portion can be or constitute 1 bit.

[0171] At 1408, at least some of the offset bits are decoded from the compressed bitstream. At least some of the offset bits may be decoded according to rows 6-9 of Table IV. At 1410, at least some of the offset bits and the least significant bit of the integer portion may be used to obtain the integer portion. The integer portion may be obtained according to rows 6-9 of Table IV. When the loop of rows 6-9 is completed, the integer portion will be obtained. Thus, in the case where the MV precision indicates a 4-integer pixel magnitude, the least significant bit of the offset bits of the integer portion may be or constitute 2 bits; and in the case where the MV precision indicates a precision not finer than 1 / 4 pixel precision, the least significant bit of the fractional portion may be or constitute the least significant bit.

[0172] At 1412, technique 1400 uses the MV precision to determine whether to omit decoding of the least significant bit in the fractional digits of the fractional portion and whether to set the least significant bit to a predefined value. In one example, using the MV precision to determine whether to omit decoding of the least significant bit in the fractional digits of the fractional portion and whether to set the least significant bit to a predefined value can be as described with respect to the pseudo code of Table V. Setting the least significant bit in the fractional digits to zero is accomplished by setting fr_bits to zero at row 1 of Table V.

[0173] At 1414, at least some of the fractional digits of the fractional portion are decoded from the compressed bitstream, such as described with respect to one or more of rows 3, 5, or 7 of Table V. Which of these rows is executed depends on the MV precision, as described. At 11416, at least some of the fractional digits of the fractional portion and the least significant bit are used to obtain the fractional portion. That is, when the pseudocode of Table V is completed, the fractional portion will be obtained.

[0174] At 1418, the MVD is obtained using at least the integer part and the fractional part. The integer part and the fractional part can be combined to obtain the MVD using MVD = (2 << ((MV class)+1)) + ((offset << 3) | fr_bits). In one example, the sign of the MVD (mv_sign) can also be decoded from the compressed bitstream. The sign mv_sign can be a flag indicating whether the MVD is positive (e.g., mv_sign = 0) or negative (e.g., mv_sign = 1). Therefore, the final value of the MVD can be obtained as follows: (mv_sign ? -MVD : MVD).

[0175] At 1420, the MVD is used to obtain the MV of the current block. Obtaining the MV may include obtaining the PMV and obtaining the MV, because MV = PMV + MVD. At 1422, the motion vector is used to obtain a prediction block for the current block. The prediction block may be as described with respect to Figure 5 It is obtained as described in the intra / inter prediction stage 508.

[0176] Fig.15 1500 is another example of a flow chart of a technique 1500 for decoding a motion vector for a current block. Technique 1500 may be implemented as, for example, a software program that may be executed by a computing device such as transmitting station 102 or receiving station 106. The software program may include machine-readable instructions that may be stored in a memory such as memory 204 or auxiliary storage device 214 and that, when executed by a processor such as CPU 202, may cause the computing device to perform technique 1500. Technique 1500 may be implemented in whole or in part in Figure 5 The technique 1500 is implemented in the inter / intra prediction stage 508 of the decoder 500. The technique 1500 can be implemented using dedicated hardware and firmware. Multiple processors, memories, or both can be used.

[0177] At 1502, an MV precision of an MVD for a current block is obtained. The MV precision may be obtained as described above. At 1504, a subset of bits of a fractional portion of the MVD is decoded from a compressed bitstream based on the MV precision, as described above. That is, the MV precision may be used to determine (e.g., select, identify, etc.) a subset of bits to be decoded. Equivalently, the MV precision may be used to determine the remaining bits of the fractional bits that are to be set to zero. At 1506, the remaining bits of the fractional portion of the MVD are set to zero.

[0178] At 1508, at least the fractional portion is used to obtain an MVD, as described above. At 1510, the MVD is used to obtain a motion vector for the current block, such as described above. At 1512, the motion vector is used to obtain a prediction block for the current block, as described above.

[0179] In one example, technique 1500 may include obtaining a candidate MV for the MV. The MV precision of the candidate MV in the candidate MV may be converted to match the MV precision, as described above. In one example, technique 1500 may include decoding an MV category of an MVD from a compressed bitstream, as described above. The MV category may indicate a set of MVDs. Each MVD in the set of MVDs corresponds to a corresponding integer part. The integer part of the MVD may be decoded from the compressed bitstream.

[0180] For ease of explanation, the techniques described herein (such as Fig. 9 , Fig.12 , Fig.13 , Fig.14 and Fig.15 The corresponding techniques 900, 1200, 1300, 1400 and 1500 of the present invention are depicted and described as a series of steps or operations. However, the steps or operations according to the present invention can be performed in various orders and / or concurrently. Additionally, other steps or operations not presented and described herein can be used. In addition, all the illustrated steps or operations may not be required to implement the method according to the disclosed subject matter.

[0181] The aspects of encoding and decoding described above illustrate some examples of encoding and decoding techniques. However, it should be understood that when those terms are used in the claims, encoding and decoding may mean compression, decompression, transformation or any other processing or change of data.

[0182] The word "example" is used herein to mean serving as an example, instance or illustration. Any aspect or design described herein as an "example" is not necessarily interpreted as being preferred or advantageous over other aspects or designs. Rather, the use of the word "example" is intended to present concepts in a specific way. As used in this application, the term "or" is intended to mean an inclusive "or" rather than an exclusive "or". That is, unless otherwise specified or clear from the context, "X includes A or B" is intended to mean any one of the natural inclusive arrangements. That is, if X includes A; X includes B; or X includes both A and B, "X includes A or B" is satisfied under any of the aforementioned examples. In addition, the articles "one" and "an" as used in this application and the appended claims should generally be interpreted as meaning "one or more", unless otherwise specified or clear from the context to refer to the singular form. In addition, the use of the term "implementation" or "an implementation" throughout the text is not intended to mean the same embodiment or implementation, unless described as such.

[0183] The implementation of the sending station 102 and / or the receiving station 106 (and the algorithms, methods, instructions, etc. stored thereon and / or executed by it (including by the encoder 400 and the decoder 500)) can be implemented in hardware, software, or any combination thereof. The hardware may include, for example, a computer, an intellectual property (IP) core, an application specific integrated circuit (ASIC), a programmable logic array, an optical processor, a programmable logic controller, a microcode, a microcontroller, a server, a microprocessor, a digital signal processor, or any other suitable circuit. In the claims, the term "processor" should be understood to cover any one of the aforementioned hardware individually or in combination. The terms "signal" and "data" are used interchangeably. Further, the parts of the sending station 102 and the receiving station 106 do not necessarily have to be implemented in the same manner.

[0184] Further, in one aspect, for example, the sending station 102 or the receiving station 106 can be implemented using a general purpose computer or general purpose processor with a computer program that, when executed, performs any of the corresponding methods, algorithms and / or instructions described herein. Additionally or alternatively, for example, a special purpose computer / processor can be utilized that can include other hardware for executing any of the methods, algorithms or instructions described herein.

[0185] The sending station 102 and the receiving station 106 can be implemented, for example, on a computer in a video conference system. Alternatively, the sending station 102 can be implemented on a server, and the receiving station 106 can be implemented on a device (such as a handheld communication device) separated from the server. In this example, the sending station 102 can use an encoder 400 to encode the content into an encoded video signal and send the encoded video signal to the communication device. Then, the communication device can use a decoder 500 to decode the encoded video signal. Alternatively, the communication device can decode the content (for example, the content not sent by the sending station 102) stored locally on the communication device. Other suitable transmission and reception implementation schemes are available. For example, the receiving station 106 can be a generally fixed personal computer rather than a portable communication device, and / or the device including the encoder 400 can also include a decoder 500.

[0186] Further, all or part of the implementation of the present disclosure may take the form of a computer program product that can be accessed from, for example, a computer-usable or computer-readable medium. A computer-usable or computer-readable medium can be any device that can, for example, tangibly contain, store, communicate, or transmit a program for use by or in conjunction with any processor. The medium can be, for example, an electronic, magnetic, optical, electromagnetic, or semiconductor device. Other suitable media are also available.

[0187] The embodiments, implementations and aspects described above have been described to allow easy understanding of the invention without limiting the invention. On the contrary, the invention is intended to cover various modifications and equivalent arrangements included within the scope of the appended claims, which scope should be given the broadest interpretation to cover all such modifications and equivalent structures permitted under the law.

Claims

1. A method for decoding a current block, comprising: decoding a motion vector (MV) class of a motion vector difference (MVD) of the current block from a compressed bitstream, wherein the MV category indicates a set of MVDs, each MVD in the set of MVDs corresponds to a respective integer part, and wherein the MVD comprises an integer part and a fractional part; Obtaining the MV accuracy of the MVD; using the MV precision to determine whether to omit decoding of the least significant bit in the offset bits of the integer part and whether to set the least significant bit in the offset bits of the integer part to a predefined value; decoding at least some of the offset bits; obtaining the integer portion using the at least some of the offset bits and the least significant bit of the integer portion; using the MV precision to determine whether to omit decoding of the least significant bit among the decimal places of the fractional part and whether to set the least significant bit to the predefined value; decoding at least some of the fractional digits of the fractional portion; obtaining the fractional portion using the at least some of the fractional digits of the fractional portion and the least significant digit; using at least the integer part and the fractional part to obtain the MVD; Using the MVD to obtain a motion vector of the current block; as well as The motion vector is used to obtain a prediction block for the current block.

2. The method of claim 1, wherein, where the MV precision indicates 4 integer pixel magnitudes, the least significant bits in the offset bits of the integer portion constitute 2 bits.

3. The method of any one of claims 1 to 2, wherein in case the MV precision indicates a 2-integer pixel magnitude, the least significant bit in the offset bits of the integer part constitutes 1 bit.

4. The method of any one of claims 1 to 3, wherein in the case where the MV precision indicates a precision no finer than 1 / 4 pixel precision, the least significant bit of the fractional part constitutes the least significant bit.

5. The method of any one of claims 1 to 4, wherein in the case where the MV precision indicates a precision no finer than 1 / 2 pixel precision, the least significant bits of the fractional part constitute the two least significant bits.

6. A method for decoding a current block, comprising: decoding a motion vector (MV) category of a motion vector difference (MVD) of the current block from a compressed bitstream, wherein the MV category indicates a set of MVDs, and wherein the MVD includes an integer portion; Obtaining the MV accuracy of the MVD; decoding at least a subset of bits representing the integer portion of the MVD using the MV precision and the MV class; obtaining the MVD using the bits representing the integer portion of the MVD; Using the MVD to obtain a motion vector of the current block; and The motion vector is used to obtain a prediction block for the current block.

7. The method of claim 6, wherein decoding at least the subset of bits representing the integer portion of the MVD using the MV precision and the MV class comprises: At least one least significant bit of the bits representing the integer portion of the MVD is inferred to be zero based on the MV precision.

8. The method of any one of claims 6 to 7, wherein decoding at least the subset of bits representing the integer portion of the MVD using the MV precision and the MV class comprises: Responsive to determining that the MV precision indicates a 4-integer pixel magnitude, two least significant bits of the bits representing the integer portion are inferred to be zero.

9. The method of any one of claims 6 to 8, wherein decoding at least the subset of bits representing the integer portion of the MVD using the MV precision and the MV class comprises: In response to determining that the MV precision indicates a 2 integer pixel magnitude, a least significant bit of the bits representing the integer portion is inferred to be zero.

10. The method according to any one of claims 6 to 9, further comprising: At least a subset of the bits representing the fractional portion of the MVD are decoded using the MV precision.

11. The method of claim 10, wherein decoding at least the subset of bits representing the fractional portion of the MVD using the MV precision comprises: In response to determining that the MV precision indicates a precision no finer than 1 / 4 pixel precision, a least significant bit of the bits representing the fractional portion is inferred to be zero.

12. The method of claim 10, wherein decoding at least the subset of bits representing the fractional portion of the MVD using the MV precision comprises: In response to determining that the MV precision indicates a precision no finer than 1 / 2 pixel precision, two least significant bits of the bits representing the fractional portion are inferred to be zero.

13. The method of claim 10, wherein decoding at least the subset of bits representing the fractional portion of the MVD using the MV precision comprises: In response to determining that the MV precision indicates integer pixel precision, the bits representing the fractional portion are inferred to be zero.

14. The method of claim 6, further comprising: Obtaining a candidate MV of the MV; as well as The respective precisions of at least some of the candidate MVs are set to the MV precision.

15. A method comprising: Obtaining the motion vector (MV) accuracy of the motion vector difference (MVD) of the current block; decoding a subset of bits of a fractional portion of the MVD from a compressed bitstream based on the MV precision; setting remaining bits of the fractional portion of the MVD to zero; using at least the fractional portion to obtain the MVD; Using the MVD to obtain a motion vector of the current block; as well as The motion vector is used to obtain a prediction block for the current block.

16. The method of claim 15, further comprising: decoding an MV category of the MVD from the compressed bitstream, wherein the MV category indicates a set of MVDs, each MVD in the set of MVDs corresponding to a respective integer part; as well as An integer portion of the MVD is decoded from the compressed bitstream.

17. The method of any one of claims 15 to 16, further comprising: Obtaining a candidate MV of the MV; as well as The MV precision of the candidate MV in the candidate MV is converted to match the MV precision.

18. An apparatus comprising: A processor configured to execute instructions to perform the method according to any one of claims 1 to 5.

19. An apparatus comprising: A processor configured to execute instructions to perform the method according to any one of claims 6 to 14.

20. An apparatus comprising: A processor configured to execute instructions to perform the method as claimed in any one of claims 15 to 17.

21. A computer-readable storage medium, comprising: Instructions which, when executed by a processor, cause the processor to perform the method as claimed in any one of claims 1 to 5.

22. A computer-readable storage medium, comprising: Instructions which, when executed by a processor, cause the processor to perform the method as claimed in any one of claims 6 to 14.

23. A computer-readable storage medium, comprising: Instructions which, when executed by a processor, cause the processor to perform the method as claimed in any one of claims 15 to 17.