Method and apparatus for history-based motion vector prediction and computer program therefor

Abstract

To provide a method and a device for predicting a motion vector of a history base, and a computer program.SOLUTION: The method for video decryption for a decoder includes the step of acquiring a current picture from a coded video bit stream, the current picture being divided in a plurality of units, each unit being divided in a plurality of blocks, and a plurality of blocks of each unit being arranged as a grid. The method further includes the step of decrypting the current block from a plurality of blocks by using an entry from a history motion vector (HMVP) buffer for one of the units. The method also includes the step of updating the HMVP buffer by the motion vector of the decrypted current block. The method further includes the step of determining whether the current block is located in the top of the row in one grid of the units.SELECTED DRAWING: Figure 13

Description

[Technical Field]

[0001] [Incorporated by reference] This application claims the benefit of priority to U.S. Patent Application No. 16 / 203,364, entitled "METHOD AND APPARATUS FOR HISTORY-BASED MOTION VECTOR PREDICTION," filed November 28, 2018, which in turn claims the benefit of priority to U.S. Provisional Application No. 62 / 698,559, entitled "METHOD AND APPARATUS FOR HISTORY-BASED MOTION VECTOR PREDICTITION," filed July 16, 2018, the entire contents of which are incorporated herein by reference.

[0002] [Technical field] This disclosure describes embodiments generally related to video encoding. [Background technology]

[0003] The background discussion provided herein is intended to generally present the context for the present disclosure. The work of the current inventors is not admitted expressly or impliedly as prior art to the present disclosure to the extent that that work is described in this background section, as well as aspects of the description that may not otherwise qualify as prior art at the time of filing.

[0004] Video encoding and decoding using inter-picture prediction with motion compensation has been known for decades. Uncompressed digital video can include a series of pictures, each having spatial dimensions of, for example, 1920 x 1080 luminance samples and associated chrominance samples. The series of pictures can have a fixed or variable picture rate (also known informally as frame rate), for example, 60 pictures per second or 60 Hz. Uncompressed video has significant bitrate requirements. For example, 1080p60 4:2:0 video (1920 x 1080 luminance sample resolution at a 60 Hz frame rate) with 8 bits per sample requires a bandwidth approaching 1.5 Gbit / s. One hour of such video requires more than 600 Gbytes of storage.

[0005] One goal of video encoding and decoding can be reducing redundancy in an input video signal through compression. Compression can help reduce the aforementioned bandwidth or storage requirements by more than two orders of magnitude, in some cases. Both lossless and lossy compression, as well as combinations thereof, can be used. Lossless compression refers to a technique that allows an exact copy of the original signal to be reconstructed from a compressed version of the original signal. When using lossy compression, the reconstructed signal may not be identical to the original signal, but the distortion between the original and reconstructed signal is small enough to make the reconstructed signal useful for its intended use. For video, lossy compression is widely used. The amount of acceptable distortion depends on the application; for example, users of certain consumer streaming applications may tolerate higher distortion than users of television contribution applications. The achievable compression ratio can reflect that higher allowable / tolerable distortion can result in a higher compression ratio.

[0006] Motion compensation may be a lossy compression technique and may relate to a technique in which blocks of sample data from a previously reconstructed picture or part thereof (reference picture) are spatially shifted in a direction indicated by a motion vector (hereinafter, MV) and then used to predict a newly reconstructed picture or part thereof. In some cases, the reference picture may be the same as the picture currently being reconstructed. The MV may have two or three dimensions, X and Y, where the third dimension is an indication of the reference picture in use (the latter may indirectly be the temporal dimension).

[0007] In some video compression techniques, the MV applicable to a region of sample data can be predicted from other MVs, for example, from an MV associated with another region of sample data that is spatially adjacent to the region being reconstructed and precedes that MV in decoding order. Doing so can significantly reduce the amount of data required to encode the MV, thereby removing redundancy and increasing compression. MV prediction can work effectively, for example, when encoding an input video signal derived from a camera (known as natural video), because there is a statistical possibility that regions larger than the region to which a single MV is applicable will move in a similar direction and, therefore, in some cases, can be predicted using similar motion vectors derived from MVs of neighboring regions. As a result, the MV found for a given region will be similar or identical to the MV predicted from surrounding MVs, which, after entropy coding, can be represented using fewer bits than would be used to directly encode the MV. In some cases, MV prediction can be an example of lossless compression of a signal (i.e., an MV) derived from the original signal (i.e., a sample stream). In other cases, the MV prediction itself can be lossy, for example, due to rounding errors in computing the predictor from several surrounding MVs.

[0008] Various MV prediction mechanisms are described in H.265 / HEVC (ITU-T Rec. H.265, "High Efficiency Video Coding", December 2016). Among the many MV prediction mechanisms provided by H.265, the one described here is a technique that will be referred to as "spatial merging".

[0009] A history buffer of motion vector predictors may be used to perform encoding or decoding. Generally, the maintenance of the history buffer is performed after each block and completed in encoding or decoding order. If this block is coded in inter mode with a set of MV information, the MV of this block is placed in the HMVP buffer to update the buffer. When coding or decoding a current block, the MV predictor for the current block may come from previously coded spatial / neighboring blocks. Some of these blocks may still be in the HMVP buffer. When placing a newly decoded / coded MV in the HMVP buffer, several comparisons may be performed to ensure that the new MV is different from all previous MVs in the HMVP buffer. If an MV with the same value already exists in the buffer, the old MV will be removed from the buffer, and the new MV will be placed in the buffer as the last entry. These general history buffer maintenance procedures do not properly reset the history buffer when information that may not be relevant to the current block being coded or decoded needs to be removed from the history buffer. Summary of the Invention

[0010] An exemplary embodiment of the present disclosure includes a video decoding method for a decoder. The method includes obtaining a current picture from an encoded video bitstream, where the current picture is divided into a plurality of units, each unit is divided into a plurality of blocks, and the plurality of blocks within each unit are arranged as a grid. The method further includes, for one of the units, decoding a current block from the plurality of blocks using an entry from a history motion vector (HMVP) buffer. The method further includes updating the HMVP buffer with the motion vector of the decoded current block. The method further includes determining whether the current block is at the beginning of a row included in the grid of one of the units. The method further includes resetting the HMVP buffer in response to determining that the current block is at the beginning of a row.

[0011] An exemplary embodiment of the present disclosure includes a video decoder for video decoding. The video decoder includes a processing circuit configured to obtain a current picture from an encoded video bitstream, where the current picture is divided into a plurality of units, each unit is divided into a plurality of blocks, and the plurality of blocks within each unit are arranged as a grid. The processing circuit is further configured to decode a current block from the plurality of blocks using an entry from a historical motion vector (HMVP) buffer for one of the units. The processing circuit is further configured to update the HMVP buffer with the motion vector of the decoded current block. The processing circuit is further configured to determine whether the current block is at the beginning of a row included in the grid of one of the units. The processing circuit is further configured to reset the HMVP buffer in response to determining that the current block is at the beginning of a row.

[0012] An exemplary embodiment of the present disclosure includes a non-transitory computer-readable medium having instructions that, when executed by a processor in a video decoder, cause the processor to perform a method. The method includes obtaining a current picture from an encoded video bitstream, where the current picture is divided into a plurality of units, each unit is divided into a plurality of blocks, and the plurality of blocks in each unit are arranged as a grid. The method further includes, for one of the units, decoding a current block from the plurality of blocks using an entry from a historical motion vector (HMVP) buffer. The method further includes updating the HMVP buffer with the motion vector of the decoded current block. The method further includes determining whether the current block is at the beginning of a row included in the grid of one of the units. The method further includes resetting the HMVP buffer in response to determining that the current block is at the beginning of a row. [Brief explanation of the drawings]

[0013] Further features, nature and various advantages of the disclosed subject matter will become more apparent from the following detailed description and accompanying drawings:

[0014] [Figure 1] FIG. 1 is a simplified block diagram schematic of a communication system (100) according to one embodiment.

[0015] [Figure 2] FIG. 2 is a simplified block diagram schematic of a communication system (200) according to one embodiment.

[0016] [Figure 3] FIG. 2 is a schematic diagram of a simplified block diagram of a decoder according to one embodiment.

[0017] [Figure 4] FIG. 2 is a schematic diagram of a simplified block diagram of an encoder according to one embodiment.

[0018] [Figure 5] 4 shows a block diagram of an encoder according to another embodiment;

[0019] [Figure 6] 4 shows a block diagram of a decoder according to another embodiment;

[0020] [Figure 7] FIG. 1 is a schematic diagram of a current block and surrounding spatial merge candidates.

[0021] [Figure 8] FIG. 1 is a schematic diagram of a merge candidate list structure.

[0022] [Figure 9] FIG. 1 is a schematic diagram of an extended merge mode.

[0023] [Figure 10A] FIG. 1 illustrates an embodiment of a history-based motion vector prediction buffer. [Figure 10B] FIG. 1 illustrates an embodiment of a history-based motion vector prediction buffer.

[0024] [Figure 11] FIG. 2 illustrates an example picture divided into coding tree units.

[0025] [Figure 12] FIG. 1 illustrates an example picture divided into tiles.

[0026] [Figure 13] FIG. 2 illustrates one embodiment of a process performed by an encoder or decoder.

[0027] [Figure 14] FIG. 1 is a schematic diagram of a computer system according to one embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0028] FIG. 1 illustrates a simplified block diagram of a communication system (100) according to one embodiment of the present disclosure. The communication system (100) includes multiple terminal devices that can communicate with each other, for example, via a network (150). For example, the communication system (100) includes a first pair of terminal devices (110) and (120) interconnected via the network (150). In the example of FIG. 1, the first pair of terminal devices (110) and (120) perform unidirectional data transmission. For example, the terminal device (110) may encode video data (e.g., a stream of video pictures captured by the terminal device (110)) for transmission to another terminal device (120) via the network (150). The encoded video data may be transmitted in the form of one or more encoded video bitstreams. The terminal device (120) may receive the encoded video data from the network (150), decode the encoded video data to reconstruct the video pictures, and display the video pictures according to the reconstructed video data. Unidirectional data transmission may be common in media provisioning applications and the like.

[0029] In another example, the communication system 100 includes a second pair of terminal devices 130 and 140 for bidirectional transmission of encoded video data, such as may occur during a video conference. For the bidirectional transmission of data, each of the terminal devices 130 and 140 may encode video data (e.g., a stream of video pictures captured by the terminal device) for transmission to the other of the terminal devices 130 and 140 over the network 150. Each of the terminal devices 130 and 140 may also receive encoded video data transmitted by the other of the terminal devices 130 and 140, decode the encoded video data to reconstruct the video pictures, and display the video pictures on an accessible display device in accordance with the reconstructed video data.

[0030] In the example of FIG. 1 , terminal devices 110, 120, 130, and 140 are illustrated as a server, a personal computer, and a smartphone, but the principles of the present disclosure may not be so limited. Embodiments of the present disclosure find application in laptop computers, tablet computers, media players, and / or dedicated videoconferencing equipment. Network 150 represents any number of networks that convey encoded video data among terminal devices 110, 120, 130, and 140, including, for example, wireline and / or wireless communication networks. Communications network 150 may exchange data over circuit-switched and / or packet-switched channels. Exemplary networks include telecommunications networks, local area networks, wide area networks, and / or the Internet. For purposes of this discussion, the architecture and topology of network 150 may not be important to the operation of the present disclosure, unless otherwise described below.

[0031] 2 illustrates the arrangement of a video encoder and a video decoder in a streaming environment as an example of application of the subject matter of this disclosure. The disclosed subject matter may be equally applicable to other video-enabled applications, including, for example, video conferencing, digital television, storage of compressed video on digital media including CDs, DVDs, memory sticks, etc.

[0032] The streaming system may include a capture subsystem (213). The capture subsystem (213) may include a video source (201), such as a digital camera, that creates a stream of uncompressed video pictures (202). In one example, the stream of video pictures (202) includes samples taken by the digital camera. The stream of video pictures (202), shown in bold to emphasize its high data volume compared to the encoded video data (204) (or encoded video bitstream), may be processed by an electronic device (220) that includes a video encoder (203) coupled to the video source (201). The video encoder (203) may include hardware, software, or a combination thereof to enable or implement aspects of the disclosed subject matter, as described in more detail below. The encoded video data 204 (or encoded bitstream 204), shown with a thin line to emphasize its lower data volume compared to the stream of video pictures 202, may be stored on the streaming server 205 for future use. One or more streaming client subsystems, such as the client subsystems 206 and 208 of FIG. 2, can access the streaming server 205 to retrieve copies 207 and 209 of the encoded video data 204. The client subsystem 206 may include a video decoder 210, for example, within the electronic device 230. The video decoder 210 decodes the incoming copy of the encoded video data 207 and creates an output stream of video pictures 211 that can be rendered on a display 212 (e.g., a display screen) or other rendering device (not shown). In some streaming systems, the encoded video data 204, 207, and 209 (e.g., video streams) may be encoded according to a particular video encoding / compression standard, examples of which include ITU-T Recommendation H.265.In one example, the developing video coding standard is known informally as Versatile Video Coding, or VVC, and the disclosed subject matter may be used in the context of VVC.

[0033] It should be noted that electronic devices 220 and 230 may include other components (not shown). For example, electronic device 220 may include a video decoder (not shown), and similarly, electronic device 230 may include a video encoder (not shown).

[0034] 3 shows a block diagram of a video decoder (310) according to an embodiment of the present disclosure. The video decoder (310) can be included in an electronic device (330). The electronic device (330) can include a receiver (331) (e.g., a receiving circuit). The video decoder (310) can be used in place of the video decoder (210) in the example of FIG. 2.

[0035] The receiver (331) may receive one or more coded video sequences to be decoded by the video decoder (310); in the same or another embodiment, one coded video sequence at a time, where decoding of each coded video sequence is independent of other coded video sequences. The coded video sequences may be received from a channel (301), which may be a hardware / software link to a storage device that stores the coded video data. The receiver (331) may receive the coded video data along with other data, such as coded audio data and / or auxiliary data streams, which may be transferred to each of them using entities (not shown). The receiver (331) may separate the coded video stream from other data. To combat network jitter, a buffer memory (315) may be coupled between the receiver (331) and the entropy decoder / parser (320) (hereinafter, "parser (320)"). In certain applications, the buffer memory (315) is part of the video decoder (310). In other examples, the buffer memory (315) can be external to the video decoder (310) (not shown). In still other examples, in addition to another buffer memory (315) internal to the video decoder (310) that handles, for example, playback timing, there can also be a buffer memory (not shown) external to the video decoder (310) that combats, for example, network jitter. When the receiver (331) receives data from a store / forward device with sufficient bandwidth and controllability or from an isosynchronous network, the buffer memory (315) may not be needed or may be small. For use in such best-effort packet networks, the buffer memory (315) may be needed and may be relatively large, advantageously adaptively sized, and may be implemented, at least in part, outside the video decoder (310) in an operating system or similar element (not shown).

[0036] The video decoder (310) may include a parser (320) to reconstruct symbols (321) from the encoded video sequence. These symbol categories include information used to manage the operation of the video decoder (310) and information to control a rendering device, such as a rendering device (312) (e.g., a display screen), which may not be an integral part of the electronic device (33) as shown in FIG. 3 but may be coupled to the electronic device (330). Control information for the rendering device may be in the form of a Supplementary Enhancement Information (SEI) message or a Video Usability Information (VUI) parameter set fragment (not shown). The parser (320) may parse / entropy decode the received encoded video sequence. The encoding of the encoded video sequence may be according to a video coding technique or standard and may follow various principles, including variable-length coding, Huffman coding, arithmetic coding with or without context-sensitive coding, etc. The parser (320) may extract from the coded video sequence a set of subgroup parameters for at least one of the subgroups of pixels in the video decoder based on at least one parameter of the group. The subgroups may include groups of pictures (GOPs), pictures, tiles, slices, microblocks, coding units (CUs), blocks, transform units (TUs), prediction units (PUs), etc. The parser (320) may also extract information from the coded video sequence, such as transform coefficients, quantized parameter values, motion vectors, etc.

[0037] The parser (320) may perform entropy decoding / parsing operations on the video sequence received from the buffer memory (315) to create symbols (321).

[0038] The reconstruction of the symbols (321) may involve several different units, depending on the type of coded video picture or portion thereof (e.g., inter- and intra-picture, inter- and intra-block) and other factors. Which units are involved and how can be controlled by subgroup control information parsed by the parser (320) from the coded video sequence. Hereinafter, the flow of such subgroup control information between the parser (320) and several units is not shown for clarity.

[0039] In addition to the functional blocks already mentioned, the video decoder (310) may be conceptually subdivided into multiple functional units, as described below. In a practical implementation operating under commercial constraints, many of these units may interact closely with each other and may be at least partially integrated with each other. However, for purposes of describing the disclosed subject matter, a conceptual subdivision into functional units hereinafter is appropriate.

[0040] The first unit is a scalar / inverse transform unit (351), which receives quantized transform coefficients as well as control information, including the transform to use, block size, quantization factor, quantization scaling metrics, etc., as symbols (321) from the parser (32). The scalar / inverse transform unit (351) can output blocks containing sample values ​​that can be input to an aggregator (355).

[0041] In some cases, the output samples of the scalar / inverse transform unit (351) may relate to intra-coded blocks, i.e., blocks that do not use prediction information from a previously reconstructed picture but can use prediction information from a previously reconstructed portion of the current picture. Such prediction information may be provided by the intra-picture prediction unit (352). In some cases, the intra-picture prediction unit (352) generates a block of the same size and the shape of that block during reconstruction using surrounding, already reconstructed information fetched from the current picture buffer (358). The current picture buffer (358) buffers, for example, a partially reconstructed and / or fully reconstructed current picture. The aggregator (355) optionally adds, on a sample-by-sample basis, the prediction information generated by the prediction unit (352) to the output sample information as provided by the scalar / inverse transform unit (351).

[0042] In other cases, the output samples of the scalar / inverse transform unit (351) may relate to an inter-coded, potentially motion-compensated, block. In such cases, the motion-compensated prediction unit (353) may access a reference picture memory (357) to fetch samples used for prediction. After motion-compensating the fetched samples according to the symbols (321) related to the block, these samples may be added by the aggregator (355) to the output of the scalar / inverse transform unit (351) (in this case, referred to as residual samples or residual signals) to generate output sample information. The addresses in the reference picture memory (357) from which the motion-compensated prediction unit (353) fetches prediction samples may be controlled by motion vectors available to the motion-compensated prediction unit (343) in the form of symbols (321), which may have, for example, X, Y, and reference picture components. Motion compensation may also include interpolation of sample values ​​fetched from the reference picture memory (357), motion vector prediction mechanisms, etc., when sub-sample accurate motion vectors are used.

[0043] The output samples of the aggregator (355) can be subjected to various loop filtering techniques in the loop filter unit (356). Video compression techniques can include in-loop filtering techniques controlled by parameters contained in the coded video sequence (also called the coded video bitstream) and made available to the loop filter unit (356) as symbols (321) from the parser (320), but can also depend on previously reconstructed and loop-filtered sample values ​​as well as on meta-information obtained during decoding of coded pictures or previous portions of the coded video sequence (in decoding order).

[0044] The output of the loop filter unit (356) can be output to a rendering device (312), can be stored in a reference picture memory (357) for use in future inter-picture prediction, and can be a sample stream.

[0045] Once a particular coded picture has been sufficiently reconstructed, it can be used as a reference picture for future prediction. For example, once the coded picture corresponding to the current picture has been sufficiently reconstructed to identify the coded picture as a reference picture (e.g., by the parser (320)), the current picture buffer (358) can become part of the reference picture memory (357), and a fresh current picture buffer can be reallocated before beginning reconstruction of a subsequent coded picture.

[0046] The video decoder (310) may perform decoding operations according to a predetermined video compression technique, such as a standard, such as ITU-T Rec. H265. An encoded video sequence may conform to the syntax specified by the video compression technique or standard being used, in the sense that the encoded video sequence adheres to both the syntax of the video compression technique or standard and the profile of the video compression technique or standard as its document. Specifically, a profile may select certain tools from all of the tools available in the video compression technique or standard as the only available tools for use under that profile. Compliance also requires that the complexity of the encoded video sequence be within a range defined by the level of the video compression technique or standard. In some cases, the level constrains the maximum picture size, maximum frame rate, maximum reconstruction sample rate (e.g., measured in megasamples per second), maximum reference picture size, etc. The limits set by the level may, in some cases, be further constrained through a Hypothetical Reference Decoder (HRD) specification and metadata for HRD buffer management signaled in the encoded video sequence.

[0047] In embodiments, the receiver (331) may receive additional (redundant) data along with the encoded video. The additional data may be included as part of the encoded video sequence. The additional data may be used by the video decoder (310) to properly decode the data and / or more accurately reconstruct the original video data. The additional data may be in the form of, for example, temporal, spatial, or signal-to-noise ratio (SNR) enhancement layers, redundant slices, redundant pictures, forward error correction codes, etc.

[0048] 4 shows a block diagram of a video encoder (403) according to an embodiment of the present disclosure. The video encoder (403) is included in an electronic device (420). The electronic device (420) includes a transmitter (440) (e.g., a transmission circuit). The video encoder (403) can be used in place of the video encoder (203) in the example of FIG. 2.

[0049] The video encoder (403) may receive video samples from a video source (401) (which, in the example of FIG. 4, is not part of the electronic device (420)), which may capture video images to be encoded by the video encoder (403). In another example, the video source (401) is part of the electronic device (420).

[0050] The video source (401) may provide a source video sequence to be encoded by the video encoder (403) in the form of a digital video sample stream, which may be of any suitable bit depth (e.g., 8-bit, 10-bit, 12-bit, ...), any color space (e.g., BT.601 Y CrCB, RGB, ...), and any suitable sampling structure (e.g., Y CrCb 4:2:0, Y CrCb 4:4:4). In a media serving system, the video source (401) may be a storage device containing previously prepared video. In a video conferencing system, the video source (401) may be a camera capturing local image information as a video sequence. The video data may be provided as multiple individual pictures that convey motion when viewed as a sequence. The pictures themselves may be organized as a spatial array of pictures, where each picture may contain one or more samples depending on the sampling structure, color space, etc., in use. Those skilled in the art can readily understand the relationship between pixels and samples. The following discussion focuses on samples.

[0051] According to an embodiment, the video encoder (403) can encode and compress pictures of a source video sequence into an encoded video sequence (443) in real time or under any other time constraints required by the application. Enforcing the appropriate encoding speed is one function of the controller (450). In some embodiments, the controller (450) controls and is functionally coupled to other functional units described below. Coupling is not shown for clarity. Parameters set by the controller (450) can include rate control-related parameters (picture skip, quantizer, lambda value for rate-distortion optimization techniques, ...), picture size, group-of-picture (GOP) layout, maximum motion vector search range, etc. The controller (450) can be configured with other appropriate functions associated with the video encoder (403) optimized for a particular system design.

[0052] In some embodiments, the video encoder (403) is configured to operate in an encoding loop. As an oversimplified explanation, in one example, the encoding loop can include a source coder (430) (responsible for creating symbols, such as a symbol stream, based on an input picture to be encoded and reference pictures) and a (local) decoder (433) embedded in the video encoder (403). The decoder (433) reconstructs the symbols to create sample data in a manner similar to that of the (remote) decoder (so that any compression between the symbols and the encoded video bitstream is lossless with the video compression techniques contemplated by the disclosed subject matter). The reconstructed sample stream (sample data) is input to a reference picture memory (434). Because decoding of the symbol stream yields bit-exact results independent of the location (local or remote) of the decoder, the contents of the reference picture memory (434) are also bit-exact between the local and remote encoders. In other words, the prediction part of the encoder "sees" the exact same sample values ​​as the reference picture samples that the decoder "sees" when using the prediction during decoding. This basic principle of reference picture synchrony (and the resulting drift when synchronization cannot be maintained due to, for example, channel errors) is used in a number of related technologies as well.

[0053] The operation of the "local" decoder (433) can be the same as that of a "remote" decoder, such as the video decoder (310), already detailed above in connection with Figure 3. However, briefly referring also to Figure 3, symbols are available, and the encoding / decoding of the symbols into an encoded video sequence by the entropy coder (445) and parser (320) can be lossless, and the entropy decoding portions of the video decoder (310), including the buffer memory (315), and the parser (320), may not be fully implemented in the local decoder (433).

[0054] An observation that can be made at this point is that any decoder technology, other than parsing / entropy decoding, that is present in a decoder must necessarily be present in substantially identical functional form in the corresponding encoder. For this reason, the disclosed subject matter focuses on decoder operation. A description of the encoder technology can be omitted, as it is the opposite of the decoder technology, which is described comprehensively. Only in certain areas is more detailed description necessary and is provided below.

[0055] During operation, in some examples, the source coder (430) may perform motion-compensated predictive coding, which predictively codes an input picture with reference to one or more previously coded pictures from a video sequence designated as “reference pictures.” In this way, the coding engine (432) codes differences between pixel blocks of the input picture and pixel blocks of reference pictures that may be selected as predictive references for the input picture.

[0056] The local video decoder (433) may decode the coded video data of pictures that may be designated as reference pictures based on symbols created by the source coder (430). The operation of the coding engine (432) may advantageously be a lossy process. When the coded video data is decoded by a video decoder (not shown in FIG. 4), the reconstructed video sequence may typically be a replica of the source video sequence with some errors. The local video decoder (433) may replicate the decoding process that may be performed by the video decoder on the reference pictures and store the reconstructed reference pictures in a reference picture cache (434). In this way, the video encoder (403) may locally store copies of reconstructed reference pictures with common content as reconstructed reference pictures (without transmission errors) to be obtained by a far-end video decoder.

[0057] The predictor (435) may perform a predictive search for the coding engine (432). That is, for a new picture to be coded, the predictor (435) may search the reference picture memory (434) for sample data (as candidate reference pixel blocks) or specific metadata, such as reference picture motion vectors, block shapes, etc., which may serve as suitable prediction references for the new picture. The predictor (435) may operate on a sample block-by-pixel block basis to find suitable prediction references. In some cases, as determined by the search results obtained by the predictor (435), the input picture may have prediction references drawn from multiple reference pictures stored in the reference picture memory (434).

[0058] The controller (450) may manage the encoding operations of the source coder (430), including, for example, setting the parameters and subgroup parameters used to encode the video data.

[0059] The outputs of all of the aforementioned functional units may undergo entropy coding in an entropy coder (445), which converts the symbols produced by the various functional units into an encoded video sequence by losslessly compressing the symbols according to techniques known to those skilled in the art, such as Huffman coding, variable length coding, arithmetic coding, etc.

[0060] The transmitter (440) may buffer the encoded video sequence created by the entropy coder (445) and prepare it for transmission over a communication channel (460), which may be a hardware / software link to a storage device that stores the encoded video data. The transmitter (440) may merge the encoded video data from the video coder (403) with other data to be transmitted, such as encoded audio data and / or auxiliary data streams (sources not shown).

[0061] The controller (450) may manage the operation of the video encoder (403). During encoding, the controller (450) may assign each coded picture a particular coding picture type, which may affect the coding technique that may be applied to the respective picture. For example, pictures may often be assigned as one of the following picture types:

[0062] An intra picture (I-picture) may be one that can be coded and decoded without using any other picture in a sequence as a source of prediction. Some video codecs allow different types of intra pictures, including, for example, Independent Decoder Refresh ("IDR") pictures. Those skilled in the art are aware of these variations of I-pictures and their respective uses and characteristics.

[0063] A predictive picture (P picture) may be coded and decoded using intra- or inter-prediction, using at most one motion vector and reference index to predict the sample values ​​of each block.

[0064] Bidirectionally predicted pictures (B-pictures) may be coded and decoded using intra- or inter-prediction, using up to two motion vectors and reference indices to predict the sample values ​​of each block. Similarly, multiple predicted pictures may use two or more reference pictures and associated metadata for the reconstruction of a single block.

[0065] A source picture is generally spatially divided into multiple sample blocks (e.g., blocks of 4x4, 8x8, 4x8, or 16x16 samples each) and may be coded block by block. Blocks may be predictively coded with reference to other (already coded) blocks, as determined by the coding assignment applied to the block's respective picture. For example, blocks of an I-picture may be non-predictively coded, or they may be predictively coded with reference to already coded blocks of the same picture (spatial prediction or intra prediction). Pixel blocks of a P-picture may be predictively coded via spatial or temporal prediction with reference to one previously coded reference picture. Blocks of a B-picture may be predictively coded via spatial or temporal prediction with reference to one or two previously coded reference pictures.

[0066] The video encoder (403) may perform encoding operations according to a predetermined video encoding technique or standard, such as ITU-T Rec. H.265. In doing so, the video encoder (403) may perform various compression operations, including predictive encoding operations that exploit temporal and spatial redundancies in the input video sequence. Thus, the encoded video data may conform to a syntax specified by the video encoding technique or standard being used.

[0067] In one embodiment, the transmitter (440) may transmit additional data along with the coded video. The source coder (430) may include such data as part of the coded video sequence. The additional data may include temporal / spatial / SNR enhancement layers, other forms of redundant data such as redundant pictures and slices, Supplementary Enhancement Information (SEI) messages, Visual Usability Information (VUI) parameter set fragments, etc.

[0068] Video may be captured as multiple source pictures (video pictures) in a time sequence. Intra-picture prediction (often abbreviated as intra-prediction) uses spatial correlation within a given picture, while inter-picture prediction uses correlation (temporal or otherwise) between pictures. In one example, a particular picture being encoded / decoded, called the current picture, is divided into blocks. When a block in the current picture is similar to a reference block in a previously encoded and still buffered reference picture in the video, the block in the current picture can be coded by a vector called a motion vector. A motion vector points to a reference block within a reference picture and may have a third dimension that identifies the reference picture if multiple reference pictures are used.

[0069] In some embodiments, a bi-prediction technique can be used in inter-picture prediction. According to the bi-prediction technique, two reference pictures, such as a first and a second reference picture, are used, both of which are prior to the current picture in decoding order (but may be past and future, respectively, in display order). A block in the current picture can be coded by a first motion vector pointing to a first reference block in the first reference picture and a second motion vector pointing to a second reference block in the second reference picture. The block can be predicted by a combination of the first and second reference blocks.

[0070] Furthermore, merge mode techniques can be used in inter-picture prediction to improve coding efficiency.

[0071] According to some embodiments of the present disclosure, prediction, such as inter-picture prediction and intra-picture prediction, is performed in units of blocks. For example, according to the HEVC standard, pictures in a sequence of video pictures are divided into coding tree units (CTUs) for compression, and the CTUs within a picture have the same size, such as 64×64 pixels, 32×32 pixels, or 16×16 pixels. Generally, a CTU includes three coding tree blocks (CTBs), one luma CTB and two chrominance CTBs. Each CTU can be recursively quadtree-decomposed into one or more coding units (CUs). For example, a 64×64 pixel CTU can be divided into one coding unit (CU) of 64×64 pixels, four CUs of 32×32 pixels, or 16 CUs of 16×16 pixels. In one example, each CU is analyzed to determine the prediction type of the CU, such as an inter-prediction type or an intra-prediction type. A CU is divided into one or more prediction units (PUs) depending on temporal and / or spatial predictability. Generally, each PU includes one luma prediction block (PB) and two chrominance PBs. In one embodiment, prediction operations in coding (encoding / decoding) are performed in units of prediction blocks. Taking a luma prediction block as an example of a prediction block, the prediction block includes a matrix of values ​​(e.g., luma values) for pixels of 8x8 pixels, 16x16 pixels, 8x16 pixels, 16x8 pixels, etc.

[0072] 5 shows a diagram of a video encoder (503) according to another embodiment of the present disclosure. The video encoder (503) is configured to receive a processed block (e.g., a predictive block) of sample values ​​in a current video picture in a sequence of video pictures and encode the processed block into an encoded picture that is part of an encoded video sequence. In one embodiment, the video encoder (503) is used in place of the example video encoder (203) of FIG. 2.

[0073] In an HEVC example, the video encoder (503) receives a matrix of sample values ​​for a processing block, such as a predictive block of 8x8 samples. The video encoder (503) determines whether the processing block is best coded using intra-mode, inter-mode, or bi-predictive mode, e.g., with rate-distortion optimization. When the processing block is coded in intra-mode, the video encoder (503) may code the processing block into a coded picture using intra-prediction techniques; when the processing block is coded in inter-mode or bi-predictive mode, the video encoder (503) may code the processing block into a coded picture using inter-prediction or bi-prediction techniques, respectively. In certain video coding techniques, the merge mode can be an inter-picture prediction sub-mode, in which motion vectors are derived from one or more motion vector predictors without the benefit of coded motion vector components external to the predictors. In certain other video coding techniques, there may be motion vector components applicable to the current block. In one embodiment, the video encoder (503) includes other components, such as a mode decision module (not shown) for determining the mode of the processing block.

[0074] In the example of Figure 5, the video encoder (503) includes an inter-encoder (530), an intra-encoder (522), a residual calculator (523), a switch (526), ​​a residual encoder (524), a general controller (521), and an entropy encoder (525), which are coupled together as shown in Figure 5.

[0075] The inter-encoder (530) is configured to receive samples of a current block (e.g., a processing block), compare the block to one or more reference blocks in a reference picture (e.g., blocks in previous and subsequent pictures), generate inter-prediction information (e.g., a description of redundant information due to inter-coding techniques, motion vectors, merge mode information), and calculate an inter-prediction result (e.g., a predicted block) based on the inter-prediction information using any suitable technique.

[0076] The intra encoder (522) is configured to receive samples of a current block (e.g., a processing block), optionally compare the block with previously coded blocks in the same picture, generate transformed and quantized coefficients, and optionally also generate intra prediction information (e.g., intra prediction direction information according to one or more intra coding techniques).

[0077] The general controller (521) is configured to determine general control data and control other components of the video encoder (503) based on the general control data. In one example, the general controller (521) determines the mode of the block and provides a control signal to the switch (526) based on the mode. For example, when the mode is intra, the general controller (521) controls the switch (526) to select intra-mode results for use by the residual calculator (523) and controls the entropy encoder (525) to select intra-prediction information and include the intra-prediction information in the bitstream. When the mode is inter-mode, the general controller (521) controls the switch (526) to select inter-prediction results for use by the residual calculator (523) and controls the entropy encoder (525) to select inter-prediction information and include the inter-prediction information in the bitstream.

[0078] The residual calculator (523) is configured to calculate the difference (residual data) between the received block and a prediction result selected from the intra-encoder (522) or the inter-encoder (530). The residual encoder (524) operates on the residual data and is configured to encode the residual data to generate transform coefficients. In one example, the residual encoder (524) is configured to transform the frequency-domain residual data to generate transform coefficients. The transform coefficients then undergo a quantization process to obtain quantized transform coefficients.

[0079] The entropy encoder (525) is configured to format the bitstream to include the coded blocks. The entropy encoder (525) is configured to include various information in accordance with an appropriate standard, such as the HEVC standard. In one example, the entropy encoder (525) is configured to include general control data, selected prediction information (e.g., intra-prediction information or inter-prediction information), residual information, and other appropriate information in the bitstream. Note that when coding a block in a merged sub-mode of either an inter mode or a bi-prediction mode in accordance with the disclosed subject matter, no residual information is present.

[0080] 6 shows a diagram of a video decoder (610) according to another embodiment of the present disclosure. The video decoder (610) is configured to receive coded pictures that are part of a coded video sequence and decode the coded pictures to generate reconstructed pictures. In one example, the video decoder (610) is used in place of the video decoder (210) of the example of FIG. 2.

[0081] In the example of FIG. 6, the video decoder (610) includes an entropy decoder (671), an inter-decoder (680), a residual decoder (673), a reconstruction module (674), and an intra-decoder (672), which are coupled together as shown in FIG. 6.

[0082] The entropy decoder (671) can be configured to reconstruct, from a coded picture, specific symbols representing the syntax elements of which the coded picture is composed. Such symbols can include, for example, prediction information (e.g., intra-prediction information or inter-prediction information) that can identify the mode in which the block is coded (e.g., intra-, inter-, bi-prediction, merged submode, or the latter two in another submode), specific samples or metadata used for prediction by the intra decoder (672) or inter decoder (680), respectively, residual information in the form of, for example, quantized transform coefficients, etc. In one example, when the prediction mode is an inter- or bi-prediction mode, the inter-prediction information is provided to the inter decoder (680); and when the prediction type is an intra-prediction type, the intra-prediction information is provided to the intra decoder (672). The residual information can undergo inverse quantization and is provided to the residual decoder (673).

[0083] The inter decoder (680) is configured to receive the inter prediction information and generate an inter prediction result based on the inter prediction information.

[0084] The intra decoder (672) is configured to receive the intra prediction information and to generate a prediction result based on the intra prediction information.

[0085] The residual decoder (673) is configured to perform inverse quantization to extract de-quantized transform coefficients and process the de-quantized transform coefficients to transform the residual from the frequency domain to the spatial domain. The residual decoder (673) may also require certain control information (including quantizer parameters QP), which may be provided by the entropy decoder (671) (data path not shown, as it may only be low-volume control information).

[0086] The reconstruction module (674) is configured to combine, in the spatial domain, the residual as output by the residual decoder (673) and the prediction result (possibly as output by an inter- or intra-prediction module) to form a reconstructed block, which may be part of a reconstructed picture, which may be part of a reconstructed video. It should be noted that other suitable operations, such as a deblocking operation, may be performed to improve visual quality.

[0087] It should be noted that the video encoders (203), (403), and (503) and the video decoders (210), (310), and (610) may be implemented using any suitable technology. In one embodiment, the video encoders (203), (403), and (503) and the video decoders (210), (310), and (610) may be implemented using one or more integrated circuits. In another embodiment, the video encoders (203), (403), and (403) and the video decoders (210), (310), and (610) may be implemented using one or more processors executing software instructions.

[0088] Merge candidates may be formed by checking motion information from spatial or temporal neighboring blocks of the current block. Referring to FIG. 7, the current block (701) contains samples found by the encoder / decoder during the motion search process that are predictable from a spatially shifted previous block of the same size. In some embodiments, instead of directly encoding its motion vector, the motion vector can be derived from metadata associated with one or more reference pictures, e.g., using the motion vector associated with any of five surrounding samples denoted D, A, C, B, and E (702-706, respectively) from the most recent reference picture (in decoding order). Blocks A, B, C, D, and E may be referred to as spatial merge candidates. These candidates may be consecutively checked into a merge candidate list. A pruning operation may be performed to ensure that duplicate candidates are removed from the list.

[0089] In some embodiments, after the spatial candidates are placed in the merge list, temporal candidates are also checked and listed. For example, the collocated block of the current block in the specified reference picture is found. The motion information at position C0 (707) in the reference picture is used as the temporal merge candidate. Position C0 may be a block in the reference picture whose upper left corner is the lower right corner of the collocated block in the reference picture of the current block 701. The collocated block in the reference picture may have the same location coordinates (e.g., x and y coordinates) as the current block 701. If the block at position C0 (707) is not coded in inter mode or is unavailable, the block at position C1 may be used. The block at position C1 may have its upper left corner at a center position (e.g., w / 2, h / 2) of the block within the collocated block in the reference picture. In particular, the block at position C1 may be a sub-block of the collocated block in the reference picture. In the above example, w and h are the width and height of the block, respectively. According to some embodiments, the additional merge candidates include combined bi-predictive candidates and zero motion vector candidates.

[0090] Skip mode may be used to indicate for a block that motion data is estimated instead of being explicitly signaled and that the prediction residual is zero (i.e., no transform coefficients are transmitted). At the beginning of each CU in an inter-picture predicted slice, a skip flag (e.g., skip_flag) may be signaled that implies one or more of: (i) the CU contains only one PU (e.g., 2Nx2N); (ii) merge mode is used to derive motion data; or (iii) no residual data is present in the bitstream.

[0091] According to some embodiments, sub-CU modes are enabled as additional merge candidates. In some embodiments, no additional syntax elements are used to signal sub-CU modes. In some embodiments, two additional merge candidates are added to the merge candidate list of each CU to represent alternative temporal motion vector prediction (ATMVP) mode and spatial-temporal motion vector prediction (STMVP) mode.

[0092] The sequence parameter set may indicate the number of candidates in the merge list. For example, if the sequence parameter set indicates that ATMVP and STMVP are enabled, up to seven merge candidates may be used in the merge list. The encoding logic for the additional merge candidates may be the same as that for other merge candidates in the merge candidate list, such that two or more rate-distortion (RD) checks are performed on two additional merge candidates for each CU in a P slice or B slice. The order of merge candidates may be A, B, C, D, ATMVP, STMVP, E (when there are fewer than six merge candidates in the list), temporal candidate, combined bi-predictive candidate, and zero motion vector candidate. The merge candidate list may be referenced by a merge index. In some embodiments, all bins of the merge index are context-adaptive binary arithmetic coding (CABAC). In other embodiments, only the first bin is context-coded, and the remaining bins are context-bypass coded.

[0093] According to some embodiments, candidate motion vectors are searched from previously coded blocks with a step size of 8x8 blocks. Figure 8 shows a current block 800 surrounded by 8x8 blocks. The closest spatial neighbors are category 1 candidates, including the row immediately above (i.e., the row containing blocks mv0 and mv1), the left column (i.e., the column containing mv2), and the upper right corner (i.e., mv2). Category 2 candidates may include blocks in the outer region that are far from the current block boundary and collocated within a previously coded frame. Category 2 candidates may include up to three candidates. In Figure 8, category 2 candidates may be selected from the outer upper row (i.e., the row containing blocks mv4 and mv5) and the outer left column (i.e., the column containing blocks mv5 and mv6). Neighboring blocks predicted from different reference frames or intra-coded may be pruned from the list. The remaining reference blocks may each be assigned a weight, which may be related to their distance to the current block. 8, the candidate list may include the following category 1 candidates: mv1, mv0, mv2, and mv3. The candidate list may further include the following category 2 candidates: mv5, mv6, and mv4.

[0094] According to some embodiments, the extended merge mode includes additional merge candidates, including blocks that are not immediately adjacent to the current block. These candidates may be in the left, top, bottom-left, top-right, and top-left directions. The maximum number of merge candidates may be 10. FIG. 9 shows a current block 900 surrounded by reference blocks (i.e., blocks with a diagonal pattern) on the left, top-left, top, and top-right. The reference blocks may include neighboring blocks A, B, C, D, and E, corresponding to blocks A, B, C, D, and E in FIG. 7, respectively. In FIG. 9, the top-left corner of the reference block may have an offset of (-96, -96) relative to the current block 900. Each candidate block B(i,j) or C(i,j) may have a vertical offset of 16 compared to its previous B or C candidate block, respectively. Each candidate block A(i,j) or D(i,j) may have a horizontal offset of 16 compared to its previous A or D candidate block, respectively. Each E(i,j) block may have an offset of 16 in both the horizontal and vertical directions compared to its previous E candidate. Candidates may be checked in the direction from the reference block closest to the current block 900 to the reference block farthest from the current block 900. The order of candidates checked may be A(i,j), B(i,j), C(i,j), D(i,j), and E(i,j).

[0095] In Figure 9, the extended neighboring positions may be determined for the current block 900 or for the current picture that includes the current block 900. According to some embodiments, instead of fetching values ​​from these extended neighboring positions, motion information of N previously coded blocks is stored in a history motion vector prediction (HMVP) buffer to provide more motion vector prediction candidates. The HMVP buffer may contain multiple HMVP candidates and may be maintained during the encoding / decoding process. In some embodiments, the HMVP buffer may operate on a first-in-first-out (FIFO) principle so that the most recently coded motion information can be considered first when this HMVP buffer is used during a motion vector prediction process, such as in merge mode or AMVP mode.

[0096] Embodiments of the present disclosure disclose several methods for obtaining motion vector predictors for inter-picture predictive coding. These methods include using MV predictors from a history-based MV buffer and performing buffer management. These methods may be applied to both merge mode and motion vector prediction with differential coding (AMVP mode). Embodiments of the present disclosure may be extended to any video coding method that uses merge and general MV prediction concepts. Also, embodiments of the present disclosure may be applied to skip mode, since skip mode uses merge mode to derive motion information.

[0097] 10A and 10B show the HMVP buffer before and after candidate insertion, respectively. As shown in FIGS. 10A and 10B, the HMVP buffer includes five entries with indexes [0] to [4]. In FIG. 10B, entry CL_0 is inserted at index [4], which causes the other entries to move one position to the left, resulting in entry HMPV_0 being removed from the buffer. Entry CL_0 may contain motion vector predictor information for a previously coded or decoded block.

[0098] According to some embodiments, each entry in the HMVP buffer is motion information from a previously coded block if the block is coded in inter coding mode. The block may be coded in bidirectional prediction mode with two motion vectors or unidirectional mode with one motion vector. For each entry in the HMVP buffer, if coded in bidirectional mode, the entry includes a pair of motion vectors as MV_L0 (with its reference index) and MV_L1 (with its reference index). According to some embodiments, the two unidirectional motion vectors corresponding to bidirectional mode include (i) MV_L0 for L0 prediction, using the reference index of L0 as the original predictor, and (ii) MV_L1 for L1 prediction, using the reference index of L1 as the original predictor.

[0099] In some embodiments, for each original bidirectional MV predictor in the HMVP buffer, two unidirectional MV predictors derived from the original MV predictor are also considered as new candidates in the merge list when the original bidirectional MV predictor is placed in the merge candidate list. In one embodiment, MV_L0 and MV_L1 are placed in the list after their corresponding original bidirectional MV predictors each time the corresponding original bidirectional MVs are placed in the list. In another embodiment, MV_L0 and MV_L1 are placed in the list after N original MV predictors from the HMVP buffer are placed in the list, where N is an integer value. N may be the number of MV candidates allowed to be placed in the merge list from the HMVP buffer, or N may be a fixed number smaller than the maximum allowed number copied from the HMVP buffer to the merge list. According to some embodiments, when entries in the HMVP buffer are used to generate MV predictors in AMVP mode, unidirectional predictors MV_L0 and MV_L1 may be generated for each original bidirectional predictor in the HMVP buffer using a method similar to that described above. These two predictors may be used as additional predictors in the AMVP MV prediction candidate list if the list is not full.

[0100] According to some embodiments, the HMVP buffer is emptied or reset to a zero state when a condition is met, which may be (i) the current CU is the start of a CTU, (ii) the current CU is the start of a tile, (iii) the current CU is the start of a CTU row, or (iv) the current CU is the start of a slice.

[0101] According to some embodiments, after the first CTU of every CTU row is completed, an HMVP_row buffer with the same size HMVP buffer is used to store the entries of the HMVP buffer. Thus, at the start of a new CTU row, the HMVP buffer can be filled with the information in the HMVP_row buffer. By resetting the HMVP buffer at the end of the CTU row and copying the contents of the HMVP_row buffer into the HMVP buffer, the blocks of the first CTU being decoded can be decoded with information from the CTU immediately above the first CTU.

[0102] In some embodiments, for each tile in a picture, an HMVP_row buffer is used to store HMVP information after the first CTU of each tile row is finished. Thus, for the first CTU of a new tile row, the HMVP buffer can be filled using information from the HMVP_row buffer. In some embodiments, the HMVP_row buffer starts in a zero state at the start of the first CTU row of a tile or slice.

[0103] 11 shows an example picture 1100 divided into CTU_00 to CTU_23. CTU_00 to CTU_03 are in the first CTU row CTU_Row_[0]. CTU_10 to CTU_13 are in the second CTU row CTU_Row_[1]. CTU_20 to CTU_23 are in the third CTU row. Each CTU in picture 1100 may be further divided into multiple blocks. The blocks may be CUs or coding blocks (CBs).

[0104] In some embodiments, the HMVP buffer is loaded with an initial value before the first block in the first CTU (e.g., CTU_00) of picture 1100 is decoded. The initial value may be stored in the encoder or decoder's memory. In another example, the HMVP buffer may be initialized to a zero state (e.g., no valid entries in the buffer). In addition, the HMVP_row buffer may be initialized to a zero state before the first block in the first CTU (e.g., CTU_00) of picture 1100 is coded or decoded. When the last block of CTU_00 (e.g., the first CTU of CTU_Row_[0]) is coded or decoded, the contents of the HMVP buffer are copied to the HMVP_row buffer. When the last block of CTU_03 (e.g., the last CTU of CTU_Row_[0]) is coded or decoded, and before the first block of CTU_10 (e.g., the first CTU of CTU_Row_[1]) is coded or decoded, the HMVP buffer is emptied and the contents of the HMVP_row buffer are copied to the HMVP buffer. Therefore, by resetting the HMVP buffer and copying the contents of the HMVP_row buffer into the HMVP buffer, encoding or decoding of blocks of CTU_10 can be performed using information from CTU_00 (e.g., blocks above CTU_10), which may be more relevant to encoding or decoding of CTU_10 than information from CTU_03.

[0105] A similar process of copying the contents of the HMVP buffer to HMVP_row may be performed after the last blocks of CTU_10 and CTU_20 are encoded or decoded. Additionally, a similar process of copying the contents of the HMVP buffer to clear it and copying the contents of the HMVP_row buffer to the HVMP buffer may be performed after the last block of CTU_13 is encoded or decoded.

[0106] Figure 12 shows an example of a picture 1100 divided into two tiles, Tile_1 and Tile_2. In some embodiments, before the first block of the first CTU (e.g., CTU_00) of Tile_1 is encoded or decoded, the HMVP buffer is loaded with initial values. The initial values ​​may be stored in the memory of the encoder or decoder. The HMVP buffer may also be initialized to a zero state. In addition, the HMVP_row buffer may be initialized to a zero state before the first block in the first CTU (e.g., CTU_00) of picture 1100 is encoded or decoded. When the last block of CTU_00 (e.g., the first CTU of CTU_Row_[0] of Tile_1) is encoded or decoded, the contents of the HMVP buffer are copied to the HMVP_row buffer. When the last block of CTU_01 is encoded or decoded (e.g., the last CTU of CTU_Row_[0] of Tile_1), the HMVP buffer is emptied and the contents of the HMVP_row buffer are copied to the HMVP buffer before the first block of CTU_10 is encoded or decoded (e.g., the first CTU of CTU_Row_[1] of Tile_1). Thus, by resetting the HMVP buffer and copying the contents of the HMVP_row buffer to the HMVP buffer, the encoding or decoding of the blocks of CTU_10 can be performed with information from CTU_00, which may be more relevant to the encoding or decoding of CTU_10 than information from CTU_01.

[0107] Tile_2 may be encoded or decoded in parallel with Tile_1 and may have separate HMVP and HMVP_row buffers, each of which may be initialized similarly to Tile_1. When the last block of CTU_02 (e.g., the first CTU of CTU_Row_[0] of Tile_2) is encoded or decoded, the contents of the HMVP buffer are copied to the HMVP_row buffer. When the last block of CTU_03 (e.g., the last CTU of CTU_Row_[0] of Tile_2) is encoded or decoded, the HMVP buffer is emptied and the contents of the HMVP_row buffer are copied to the HMVP buffer before the first block of CTU_12 (e.g., the first CTU of CTU_Row_[1] of Tile_2) is encoded or decoded. Therefore, by resetting the HMVP buffer and copying the contents of the HMVP_row buffer into the HMVP buffer, encoding or decoding of blocks of CTU_12 can be performed using information from CTU_02, which may be more relevant to decoding CTU_12 than information from CTU_03.

[0108] FIG. 13 shows an embodiment of a process performed by an encoder, such as encoder 503, or a decoder, such as decoder 610. The process may start at step S1300, where a current picture is obtained from a video bitstream. For example, picture 1100 (FIG. 11) may be this obtained picture. The process proceeds to step S1302, where a current block is encoded / decoded using one or more entries from the HMVP buffer. For example, with reference to picture 1100, if the first block of CTU_00 is being encoded / decoded, the HMVP buffer may be initialized to an initial state, and the first block may be encoded / decoded with one or more entries from the HMVP buffer after the HMVP buffer is initialized. The process proceeds to step S1304, where the HMVP buffer is updated with motion vector information of the coded / decoded current block.

[0109] The process proceeds to step S1306, where it is determined whether the currently coded / decoded block is at the end of a CTU row. For example, with reference to picture 1100, if the current block to be coded / decoded is the last block of CTU_03, then the next block to be coded / decoded is the first block of the next row, CTU_10. If the currently coded / decoded block is at the end of a CTU row, the process proceeds to step S1308, where the HMVP buffer is reset (e.g., emptied) and the contents of the HMVP_row buffer are copied to the HMVP buffer before the next block is coded / decoded. From step S1308, the process proceeds to step S1314, which is described in further detail below.

[0110] If the currently coded / decoded block is not the end of the CTU row, the process proceeds to step S1310, where it is determined whether the currently coded / decoded block is the last block of the first CTU in the CTU row. If the currently coded / decoded block is the last block of the first CTU in the CTU row, the process proceeds to step S1312, where the contents of the HMVP buffer are copied to the HVMP_row buffer. For example, with reference to picture 1100, if the currently coded / decoded block is the last block of CTU_00, the contents of the HVMP buffer are copied to the contents of the HMVP_row buffer before the first block of CTU_01 is coded / decoded. The process proceeds from step S1312 to step S1314, which will be described in further detail below.

[0111] If the currently coded / decoded block is not the last block of the first CTU in the CTU row, the process proceeds to step S1314, where it is determined whether the currently coded / decoded block is the last block in the obtained picture. If the currently coded / decoded block is the last block in the obtained picture, the process of FIG. 13 ends. For example, if the currently coded / decoded block is the last block of CTU_23, the process of FIG. 13 is complete. If the currently coded / decoded block is not the last block in the obtained picture, the process returns from step S1314 to step S1302.

[0112] The techniques described above can be implemented as computer software using computer-readable instructions and physically stored on one or more computer-readable media. For example, Figure 14 illustrates a computer system (1400) suitable for implementing certain embodiments of the disclosed subject matter.

[0113] Computer software may be coded using any suitable machine code or computer language that may be subject to assembly, compilation, linking, or similar mechanisms to create code containing instructions that may be executed by one or more computer central processing units (CPUs), graphics processing units (GPUs), etc., either directly or through translation, microcode execution, etc.

[0114] The instructions may be executed on various types of computers or components thereof, including, for example, personal computers, tablet computers, servers, smartphones, gaming devices, Internet of Things devices, etc.

[0115] 14 of the computer system (1400) are exemplary in nature and are not intended to suggest any limitation as to the scope of use or functionality of the computer software implementing embodiments of the present disclosure, nor should the arrangement of components be interpreted as having any dependency or requirement related to any one or combination of components illustrated in the exemplary embodiment of the computer system (1400).

[0116] The computer system (1400) may include certain human interface input devices. Such human interface input devices may respond to input by one or more human users, for example, through tactile input (e.g., keystrokes, swipes, data glove movements), audio input (e.g., voice, clapping), visual input (e.g., gestures), or olfactory input (not shown). Human interface devices may also be used to capture certain media that do not necessarily involve direct conscious human input, such as audio (e.g., speech, music, ambient sounds), images (e.g., scanned images, photographic images obtained from still image cameras), and video (e.g., two-dimensional video, three-dimensional video, including stereoscopic video).

[0117] The input human interface devices may include one or more of the following (only one of which is shown): a keyboard (1401), a mouse (1402), a trackpad (1403), a touchscreen (1410), a data glove (not shown), a joystick (1405), a microphone (1406), a scanner (1407), and a camera (1408).

[0118] The computer system (1400) may also include certain human interface output devices. Such human interface output devices may stimulate one or more of the human user's senses, for example, through tactile output, sound, light, and smell / taste. Such human interface output devices may include haptic output devices (e.g., haptic feedback via a touchscreen (1410), data gloves (not shown), or joystick (1405), although haptic feedback devices that do not function as input devices may also exist), audio output devices (e.g., speakers (1409), headphones (not shown)), visual output devices (e.g., screens (1410), including CRT screens, LCD screens, plasma screens, and OLED screens, each with or without touchscreen input capability and each with or without haptic feedback capability, some of which may be capable of outputting two-dimensional visual output or greater than three-dimensional output through means such as stereoscopic output, virtual reality glasses (not shown), holographic displays, and smoke tanks (not shown)), and printers (not shown).

[0119] The computer system (1400) may also include human-accessible storage devices and their associated media, such as CD / DVD ROM / RW (1420) with CD / DVD or similar media (1421), thumb drives (1422), removable hard drives or solid state drives (1423), legacy magnetic media such as tape and floppy disks (not shown), and special ROM / ASIC / PLD-based devices such as security dongles (not shown).

[0120] Those skilled in the art should also understand that the term "computer-readable medium," as used in connection with the presently disclosed subject matter, does not encompass transmission media, carrier waves, or other transitory signals.

[0121] The computer system 1400 may also include interfaces to one or more communications networks. Networks may be, for example, wireless, wired, or optical. Networks may further be local, wide-area, metropolitan, vehicular, and industrial, real-time, delay-tolerant, and the like. Examples of networks include Ethernet, WLAN, cellular networks including GSM, 3G, 4G, 5G, LTE, and the like; TV wired or wireless wide-area digital networks including cable TV, satellite TV, and terrestrial TV; and vehicular and industrial networks including CAN Bus. Certain networks generally require an external network interface adapter attached to a particular general-purpose data port or peripheral bus 1449 (e.g., a USB port on the computer system 1400), while others are generally integrated into the core of the computer system 1400 by attachment to a system bus described below (e.g., an Ethernet interface to a PC computer system or a cellular network interface to a smartphone computer system). Using any of these networks, the computer system 1400 can communicate with other entities. Such communications may be unidirectional, receive-only (e.g., broadcast TV), unidirectional transmit-only (e.g., CANBus to a specific CANBus device), or bidirectional to other computer systems using, for example, local or wide area digital networks. Specific protocols and protocol stacks may be used in each of these networks and network interfaces, as described above.

[0122] The aforementioned human interface devices, human-accessible storage devices, and network interfaces may be attached to the core (1440) of the computer system (1400).

[0123] The core (1440) may include one or more central processing units (CPUs) (1441), graphics processing units (GPUs) (1442), specialized programmable processing units in the form of field programmable gate arrays (FPGAs) (1443), hardware accelerators for specific tasks (1444), etc. These devices, along with read-only memory (ROM) (1445), random access memory (1446), and internal mass storage such as an internal non-user-accessible hard drive or SSD (1447), may be connected via a system bus (1448). In some computer systems, the system bus (1448) may be accessible in the form of one or more physical plugs to allow expansion with additional CPUs, GPUs, etc. Peripheral devices may be attached directly to the core's system bus (1448) or via a peripheral bus (1449). Peripheral bus architectures include PCI, USB, etc.

[0124] The CPU (1441), GPU (1442), FPGA (1443), and accelerator (1444) may combine to execute specific instructions that may constitute the aforementioned computer code. The computer code may be stored in ROM (1445) or RAM (1446). Transient data may also be stored in RAM (1446), while permanent data may be stored, for example, in internal mass storage (1447). Cache memory, which may be closely associated with one or more of the CPU (1441), GPU (1442), mass storage (1447), ROM (1445), RAM (1446), etc., may be used to enable fast storage and retrieval from any of the memory devices.

[0125] The computer-readable medium may have computer code thereon for performing various computer-implemented operations. The medium and computer code may be those specially designed and constructed for the purposes of the present disclosure, or they may be of the kind well known and available to those skilled in the computer software arts.

[0126] By way of example and not limitation, the architecture (1400), and in particular a computer system having a core (1440), can provide functionality as a result of a processor (including a CPU, GPU, FPGA, accelerator, etc.) executing software embodied in one or more tangible computer-readable media. Such computer-readable media can be user-accessible mass storage, as described above, as well as media associated with specific storage of the core (1440) that is non-transitory in nature, such as the core's internal mass storage (1447) or ROM (1445). Software implementing various embodiments of the present disclosure can be stored on such devices and executed by the core (1440). The computer-readable media can include one or more memory devices or chips, depending on particular needs. The software can cause the core (1440) and in particular the processor (including a CPU, GPU, FPGA, etc.) therein to perform particular processes or particular portions of particular processes described herein, including defining data structures stored in RAM (1446) and modifying such data structures according to processes defined by the software. Additionally or alternatively, a computer system may provide functionality as a result of logic hardwired or otherwise embodied in circuitry (e.g., accelerator (1444)), which may operate in place of or in conjunction with software to perform particular processes or portions of particular processes described herein. References to software include logic, and vice versa, where appropriate. References to computer-readable media may include circuitry (e.g., integrated circuits (ICs)) that store software for execution, circuitry that embodies logic for execution, or both, as appropriate. The present disclosure encompasses any suitable combination of hardware and software. Appendix A: Acronyms MV: Motion Vector HEVC: High Efficiency Video Coding SEI: Supplemental Emphasis Information VUI: Video Usability Information GOP: Group of Pictures TU: Conversion unit PU: Prediction Unit CTU: Coding Tree Unit CTB: coding tree block PB: Predicted Block HRD: Hypothetical Reference Decoder SNR: Signal to Noise Ratio CPU: Central Processing Unit GPU: Graphics Processing Unit CRT: cathode ray tube LCD: Liquid crystal display OLED: Organic Light Emitting Diode CD: Compact Disc DVD: Digital Video Disc ROM: Read-Only Memory RAM: Random Access Memory ASIC: Application Specific Integrated Circuit PLD: Programmable Logic Device LAN: Local Area Network GSM: Global System for Mobile Communications LTE: Long Term Evolution CANBus: Controller Area Network Bus USB: Universal Serial Bus PCI: Peripheral Component Interconnect FPGA: Field Programmable Gate Area SSD: Solid State Drive IC: Integrated Circuit CU: Coding Unit

[0127] While this disclosure has described several exemplary embodiments, there are alterations, permutations, and various substitute equivalents that fall within the scope of this disclosure. It will thus be appreciated that those skilled in the art will be able to devise many systems and methods that, although not explicitly shown or described herein, embody the principles of the present disclosure and therefore are within the spirit and scope of the present disclosure.

[0128] (1) A method of video decoding performed by a decoder, the method including the steps of: obtaining a current picture from an encoded video bitstream, the current picture being divided into a plurality of units, each unit being divided into a plurality of blocks, the plurality of blocks in each unit forming a grid; decoding a current block from the plurality of blocks for one of the plurality of units using an entry from a historical motion vector (HMVP) buffer; updating the HMVP buffer with the motion vector of the decoded current block; determining whether the current block is at the beginning of a row included in the grid of one of the plurality of units; and resetting the HMVP buffer in response to determining that the current block is at the beginning of the row.

[0129] (2) The method according to feature (1), further comprising the steps of: (i) determining whether the current block is the last block of the first unit of the row; and (ii) copying the contents of the HMVP buffer to the row buffer in response to determining that the current block is the last block of the first unit of the row.

[0130] (3) The method according to feature (2), further comprising the step of copying the contents of the row buffer to the HMVP buffer after each entry of the HMVP buffer is reset in response to determining that the current block is the beginning of a row.

[0131] (4) A method according to any one of features (1) to (3), wherein the HMVP buffer is a first-in, first-out (FIFO) buffer, and the step of updating the HMVP buffer with the motion vector includes the steps of storing the motion vector in the last entry of the HMVP buffer and deleting the first entry of the HMVP buffer.

[0132] (5) The method according to any one of features (1) to (4), wherein the unit is a coding tree unit (CTU).

[0133] (6) A method according to any one of features (2) to (5), wherein the units are tiles, one of the decoded units is a first tile, and the first tile and second tile from the multiple units are decoded in parallel.

[0134] (7) A video decoder for video decoding, comprising: a processing circuit configured to: obtain a current picture from an encoded video bitstream; divide the current picture into a plurality of units, each unit into a plurality of blocks, the plurality of blocks in each unit forming a grid; for one of the plurality of units, decode a current block from the plurality of blocks using an entry from a history motion vector (HMVP) buffer; update the HMVP buffer with a motion vector of the decoded current block; determine whether the current block is at the beginning of a row included in the grid of one of the plurality of units; and reset the HMVP buffer in response to determining that the current block is at the beginning of the row.

[0135] (8) The video decoder of feature (7), wherein the processing circuit is further configured to determine whether the current block is the last block of the first unit of the row, and to copy the contents of the HMVP buffer to the row buffer in response to determining that the current block is the last block of the first unit of the row.

[0136] (9) The video decoder of feature (8), wherein the processing circuit is further configured to copy the contents of the row buffer to the HMVP buffer after each entry of the HMVP buffer is reset in response to determining that the current block is the beginning of a row.

[0137] (10) A video decoder according to any one of features (7) to (9), wherein the HMVP buffer is a first-in, first-out (FIFO) buffer, and updating the HMVP buffer with the motion vector includes storing the motion vector in the last entry of the HMVP buffer and deleting the first entry of the HMVP buffer.

[0138] (11) The video decoder according to any one of features (7) to (10), wherein the unit is a coding tree unit (CTU).

[0139] (12) A video decoder according to any one of features (8) to (11), wherein the units are tiles, one of the decoded units is a first tile, and the first tile and second tile from multiple units are decoded in parallel.

[0140] (13) A non-transitory computer-readable medium having instructions, when executed by a processor in a video decoder, that cause the processor to perform a method comprising: obtaining a current picture from an encoded video bitstream, the current picture being divided into a plurality of units, each unit being divided into a plurality of blocks, the plurality of blocks in each unit forming a grid; decoding a current block from the plurality of blocks using an entry from a historical motion vector (HMVP) buffer for one of the plurality of units; updating the HMVP buffer with the motion vector of the decoded current block; determining whether the current block is at the beginning of a row included in the grid of one of the plurality of units; and resetting the HMVP buffer in response to determining that the current block is at the beginning of a row.

[0141] (14) The non-transitory computer-readable medium of feature (13), wherein the method further includes the steps of: determining whether the current block is the last block of the first unit of the row; and copying the contents of the HMVP buffer to the row buffer in response to determining that the current block is the last block of the first unit of the row.

[0142] (15) The non-transitory computer-readable medium of feature (14), wherein the method further includes a step of copying the contents of the row buffer to the HMVP buffer after each entry in the HMVP buffer is reset in response to determining that the current block is the beginning of a row.

[0143] (16) A non-transitory computer-readable medium according to any one of features (13) to (15), wherein the HMVP buffer is a first-in, first-out (FIFO) buffer, and the step of updating the HMVP buffer with the motion vector includes the steps of storing the motion vector in the last entry of the HMVP buffer and deleting the first entry of the HMVP buffer.

[0144] (17) The non-transitory computer-readable medium according to any one of features (13) to (16), wherein the unit is a coding tree unit (CTU).

[0145] (18) The non-transitory computer-readable medium of any one of features (14) to (17), wherein the units are tiles, one of the units decoded is a first tile, and the first tile and second tile from multiple units are decoded in parallel.

Claims

1. 1. A method for generating an encoded video bitstream performed by an encoder, comprising: generating an encoded video bitstream by encoding the input video bitstream; and transmitting the encoded video bitstream; Encoding the input video bitstream includes: obtaining a current picture from the input video bitstream, dividing the current picture into a plurality of units, and dividing the current picture into a plurality of tiles, each tile including at least one unit; encoding a first current unit in a first tile of the plurality of tiles; updating a first HMVP buffer with the motion vector of the first current unit that has been coded; determining a position of the first current unit in the first tile of the plurality of tiles; resetting the first HMVP buffer in response to determining that the first current unit is located in a first column of the first tile; A method for generating an encoded video bitstream, comprising:

2. 1. A method for generating an encoded video bitstream performed by an encoder, comprising: generating an encoded video bitstream by encoding the input video bitstream; and storing the encoded video bitstream; Encoding the input video bitstream includes: obtaining a current picture from the input video bitstream, dividing the current picture into a plurality of units, and dividing the current picture into a plurality of tiles, each tile including at least one unit; encoding a first current unit in a first tile of the plurality of tiles; updating a first HMVP buffer with the motion vector of the first current unit that has been coded; determining a position of the first current unit in the first tile of the plurality of tiles; resetting the first HMVP buffer in response to determining that the first current unit is located in a first column of the first tile; A method for generating an encoded video bitstream, comprising:

3. In response to determining that the first current unit is located in a first column of the first tile, copying contents of a first row buffer to the first HMVP buffer. A method for generating an encoded video bitstream according to claim 1 or 2.

4. In response to determining that the first current unit is located in the last column of the first tile, copying the contents of the first HMVP buffer to the first row buffer.

4. The method of claim 3 for generating an encoded video bitstream.

5. encoding a second current unit in a second tile of the plurality of tiles; updating a second HMVP buffer with the motion vector of the second current unit that has been coded; determining a position of the second current unit in the second tile of the plurality of tiles; resetting the second HMVP buffer in response to determining that the second current unit is located in a first column of the second tile; Further comprising: A method for generating an encoded video bitstream according to any one of claims 1 to 4.

6. In response to determining that the second current unit is located in a first column of the second tile, copying contents of a second row buffer to the second HMVP buffer.

6. A method for generating an encoded video bitstream according to claim 5.

7. In response to determining that the second current unit is located in the last column of the second tile, copying the contents of the second HMVP buffer to the second row buffer.

7. The method of claim 6 for generating an encoded video bitstream.

8. the encoding of the first current unit is performed in parallel with the encoding of the second current unit. A method for generating an encoded video bitstream according to any one of claims 5 to 7.

9. the first HMVP buffer and the second HMVP buffer are first-in, first-out (FIFO) buffers; updating the first HMVP buffer with the motion vector of the first current encoded unit includes storing the motion vector in a last entry of the first HMVP buffer and deleting a first entry of the first HMVP buffer; updating the second HMVP buffer with the motion vector of the second current encoded unit includes storing the motion vector in the last entry of the second HMVP buffer and deleting the first entry of the second HMVP buffer; A method for generating an encoded video bitstream according to any one of claims 5 to 8.

10. A video encoder including processing circuitry, A video encoder, wherein the processing circuitry is configured to perform a method for generating an encoded video bitstream according to any one of claims 1 to 9.

Images