Method and device for predicting motion vector of history base and computer program

Abstract

To provide a method and a device for predicting a motion vector of a history base, and a computer program.SOLUTION: The method for video decryption for a decoder includes the step of acquiring a current picture from a coded video bit stream, the current picture being divided in a plurality of units, each unit being divided in a plurality of blocks, and a plurality of blocks of each unit being arranged as a grid. The method further includes the step of decrypting the current block from a plurality of blocks by using an entry from a history motion vector (HMVP) buffer for one of the units. The method also includes the step of updating the HMVP buffer by the motion vector of the decrypted current block. The method further includes the step of determining whether the current block is located in the top of the row in one grid of the units.SELECTED DRAWING: Figure 13

Description

[Technical field]

[0001] [Incorporated by reference] This application claims the benefit of priority to U.S. Patent Application No. 16 / 203,364, entitled "METHOD AND APPARATUS FOR HISTORY-BASED MOTION VECTOR PREDICTION," filed November 28, 2018, which claims the benefit of priority to U.S. Provisional Application No. 62 / 698,559, entitled "METHOD AND APPARATUS FOR HISTORY-BASED MOTION VECTOR PREDICTITION," filed July 16, 2018, the entire contents of which are incorporated herein by reference.

[0002] [Technical field] SUMMARY This disclosure describes embodiments generally related to video encoding. [Background technology]

[0003] The background discussion provided herein is intended to generally present the context of the present disclosure. The work of the current inventors is not admitted, expressly or impliedly, as prior art to the present disclosure to the extent that such work is described in this background section, as well as aspects of the description that may not otherwise qualify as prior art at the time of filing.

[0004] Video encoding and decoding using inter-picture prediction with motion compensation has been known for decades. Uncompressed digital video may include a sequence of pictures, each having spatial dimensions of, for example, 1920x1080 luminance samples and associated chrominance samples. The sequence of pictures may have a fixed or variable picture rate (also known informally as frame rate), for example, 60 pictures / second or 60 Hz. Uncompressed video has significant bitrate requirements. For example, 1080p60 4:2:0 video (1920x1080 luminance sample resolution at 60 Hz frame rate) with 8 bits per sample requires a bandwidth approaching 1.5 Gbit / s. One hour of such video requires more than 600 Gbytes of storage.

[0005] One goal of video encoding and decoding may be the reduction of redundancy in the input video signal through compression. Compression may help reduce the aforementioned bandwidth or storage needs, in some cases by more than one order of magnitude. Both lossless and lossy compression, as well as combinations thereof, may be used. Lossless compression refers to techniques where an exact copy of the original signal can be reconstructed from the compressed original signal. When using lossy compression, the reconstructed signal may not be identical to the original signal, but the distortion between the original and reconstructed signals is small enough to make the reconstructed signal useful for its intended application. For video, lossy compression is widely used. The amount of distortion tolerated depends on the application, e.g., a user of a particular consumer streaming application may tolerate higher distortion than a user of a television contribution application. The achievable compression ratio may reflect that a higher permitted / tolerable distortion can result in a higher compression ratio.

[0006] Motion compensation may be a lossy compression technique and may be related to a technique in which blocks of sample data from a previously reconstructed picture or part thereof (reference picture) are used for prediction of a newly reconstructed picture or part thereof after being spatially shifted in a direction indicated by a motion vector (hereinafter MV). In some cases, the reference picture may be the same as the picture currently being reconstructed. The MV may have two or three dimensions, X and Y, where the third dimension is an indication of the reference picture being used (the latter may indirectly be the temporal dimension).

[0007] In some video compression techniques, the MV applicable to a region of sample data can be predicted from other MVs, for example from an MV associated with another region of sample data that is spatially adjacent to the region being reconstructed and precedes it in decoding order. By doing so, the amount of data required to encode the MV can be significantly reduced, thereby removing redundancy and increasing compression. MV prediction can work effectively, for example when encoding an input video signal derived from a camera (known as natural video), since there is a statistical probability that regions larger than the region to which a single MV is applicable will move in similar directions and therefore, in some cases, can be predicted using similar motion vectors derived from the MVs of neighboring regions. As a result, the MV found for a given region will be similar or identical to the MV predicted from the surrounding MVs, and this, after entropy coding, can be represented with fewer bits than would be used if encoding the MVs directly. In some cases, MV prediction can be an example of lossless compression of a signal (i.e., MV) derived from the original signal (i.e., sample stream). In other cases, the MV prediction itself can be lossy, for example due to rounding errors in computing the predictor from some surrounding MVs.

[0008] Various MV prediction mechanisms are described in H.265 / HEVC (ITU-T Rec. H.265, "High Efficiency Video Coding", December 2016). Among the many MV prediction mechanisms that H.265 offers, the one described here is a technique that will be referred to as "spatial merging".

[0009] A history buffer of motion vector predictors may be used to perform the encoding or decoding. In general, the maintenance of the history buffer is done after each block and is completed in the encoding or decoding order. If this block is coded in inter mode with a set of MV information, the MV of this block is put into the HMVP buffer to update the buffer. When coding or decoding the current block, the MV predictor for the current block may come from previously coded spatial / neighboring blocks. Some of these blocks may still be in the HMVP buffer. When putting the newly decoded / coded MV into the HMVP buffer, some comparisons may be made to ensure that the new MV is different from all previous MVs in the HMVP buffer. If there is already an MV with the same value in the buffer, the old MV will be removed from the buffer and the new MV is put into the buffer as the last entry. These general maintenance procedures of the history buffer do not reset the history buffer properly when information that may not be related to the current block being coded or decoded needs to be removed from the history buffer. Summary of the Invention

[0010] An exemplary embodiment of the present disclosure includes a method of video decoding for a decoder. The method includes obtaining a current picture from an encoded video bitstream, where the current picture is divided into a plurality of units, each unit is divided into a plurality of blocks, and the plurality of blocks in each unit are arranged as a grid. The method further includes, for one of the units, decoding a current block from the plurality of blocks using an entry from a history motion vector (HMVP) buffer. The method further includes updating the HMVP buffer with a motion vector of the decoded current block. The method further includes determining whether the current block is at the beginning of a row contained within the grid of one of the units. The method further includes resetting the HMVP buffer in response to determining that the current block is at the beginning of a row.

[0011] An exemplary embodiment of the present disclosure includes a video decoder for video decoding. The video decoder includes a processing circuit configured to obtain a current picture from an encoded video bitstream, where the current picture is divided into a plurality of units, each unit is divided into a plurality of blocks, and the plurality of blocks in each unit are arranged as a grid. The processing circuit is further configured to decode a current block from the plurality of blocks using an entry from a history motion vector (HMVP) buffer for one of the units. The processing circuit is further configured to update the HMVP buffer with a motion vector of the decoded current block. The processing circuit is further configured to determine whether the current block is at the beginning of a row included in the grid of one of the units. The processing circuit is further configured to reset the HMVP buffer in response to determining that the current block is at the beginning of a row.

[0012] An exemplary embodiment of the present disclosure includes a non-transitory computer-readable medium having instructions that, when executed by a processor in a video decoder, cause the processor to perform a method. The method includes obtaining a current picture from an encoded video bitstream, where the current picture is divided into a plurality of units, each unit is divided into a plurality of blocks, and the plurality of blocks in each unit are arranged as a grid. The method further includes, for one of the units, decoding a current block from the plurality of blocks using an entry from a historical motion vector (HMVP) buffer. The method further includes updating the HMVP buffer with a motion vector of the decoded current block. The method further includes determining whether the current block is at the beginning of a row contained within the grid of one of the units. The method further includes resetting the HMVP buffer in response to determining that the current block is at the beginning of a row. [Brief description of the drawings]

[0013] Further features, nature and various advantages of the disclosed subject matter will become more apparent from the following detailed description and the accompanying drawings:

[0014] [Figure 1] FIG. 1 is a schematic diagram of a simplified block diagram of a communication system (100) according to one embodiment.

[0015] [Diagram 2] FIG. 2 is a schematic diagram of a simplified block diagram of a communication system (200) according to one embodiment.

[0016] [Diagram 3] FIG. 2 is a schematic diagram of a simplified block diagram of a decoder according to one embodiment.

[0017] [Figure 4] FIG. 2 is a schematic diagram of a simplified block diagram of an encoder according to one embodiment.

[0018] [Diagram 5] 4 shows a block diagram of an encoder according to another embodiment;

[0019] [Figure 6] 4 shows a block diagram of a decoder according to another embodiment;

[0020] [Figure 7] FIG. 2 is a schematic diagram of a current block and surrounding spatial merging candidates;

[0021] [Figure 8] FIG. 13 is a schematic diagram of a merge candidate list configuration.

[0022] [Figure 9] FIG. 13 is a schematic diagram of the extended merge mode.

[0023] [Figure 10A] FIG. 2 illustrates an embodiment of a history-based motion vector prediction buffer. [Figure 10B] FIG. 2 illustrates an embodiment of a history-based motion vector prediction buffer.

[0024] [Figure 11] FIG. 2 illustrates an example picture divided into coding tree units.

[0025] [Figure 12] FIG. 2 illustrates an example picture divided into tiles.

[0026] [Figure 13] FIG. 2 illustrates one embodiment of a process performed by an encoder or decoder.

[0027] [Figure 14] FIG. 1 is a schematic diagram of a computer system according to one embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0028] FIG. 1 illustrates a simplified block diagram of a communication system (100) according to an embodiment of the present disclosure. The communication system (100) includes a plurality of terminal devices that can communicate with each other, for example, via a network (150). For example, the communication system (100) includes a first pair of terminal devices (110) and (120) that are interconnected via the network (150). In the example of FIG. 1, the first pair of terminal devices (110) and (120) perform a unidirectional transmission of data. For example, the terminal device (110) may encode video data (e.g., a stream of video pictures captured by the terminal device (110)) for transmission to another terminal device (120) via the network (150). The encoded video data may be transmitted in the form of one or more encoded video bitstreams. The terminal device (120) may receive the encoded video data from the network (150), decode the encoded video data to reconstruct the video pictures, and display the video pictures according to the reconstructed video data. Unidirectional data transmission may be common, such as in media provisioning applications.

[0029] In another example, the communication system (100) includes a second pair of terminal devices (130) and (140) for bidirectional transmission of encoded video data, such as may occur during a video conference. For the bidirectional transmission of data, each of the terminal devices (130) and (140) may encode video data (e.g., a stream of video pictures captured by the terminal device) for transmission to the other of the terminal devices (130) and (140) over the network (150). Each of the terminal devices (130) and (140) may also receive encoded video data transmitted by the other of the terminal devices (130) and (140), decode the encoded video data to reconstruct the video pictures, and display the video pictures on an accessible display device according to the reconstructed video data.

[0030] In the example of FIG. 1, the terminal devices (110), (120), (130), and (140) are illustrated as a server, a personal computer, and a smartphone, but the principles of the present disclosure may not be so limited. Embodiments of the present disclosure find application in laptop computers, tablet computers, media players, and / or dedicated video conferencing equipment. The network (150) represents any number of networks that convey encoded video data among the terminal devices (110), (120), (130), and (140), including, for example, wireline (wired) and / or wireless communication networks. The communication network (150) may exchange data over circuit-switched and / or packet-switched channels. Exemplary networks include telecommunications networks, local area networks, wide area networks, and / or the Internet. For purposes of this discussion, the architecture and topology of the network (150) may not be important to the operation of the present disclosure, unless described below.

[0031] 2 illustrates the arrangement of a video encoder and a video decoder in a streaming environment as an example of an application of the subject matter of this disclosure. The disclosed subject matter may be equally applicable to other video-enabled applications including, for example, video conferencing, digital television, storage of compressed video on digital media including CDs, DVDs, memory sticks, etc.

[0032] The streaming system may include a capture subsystem (213). The capture subsystem (213) may include a video source (201), such as a digital camera, that creates a stream of uncompressed video pictures (202). In one example, the stream of video pictures (202) includes samples taken by the digital camera. The stream of video pictures (202) is shown in bold to emphasize its high data volume compared to the encoded video data (204) (or encoded video bitstream), and may be processed by an electronic device (220) that includes a video encoder (203) coupled to the video source (201). The video encoder (203) may include hardware, software, or a combination thereof to enable or implement aspects of the disclosed subject matter, which are described in more detail below. The encoded video data (204) (or encoded bitstream (204)), shown with thin lines to emphasize its lower data volume compared to the stream of video pictures (202), may be stored on the streaming server (205) for future use. One or more streaming client subsystems, such as the client subsystems (206) and (208) of FIG. 2, may access the streaming server (205) to retrieve copies (207) and (209) of the encoded video data (204). The client subsystem (206) may include a video decoder (210), for example within the electronic device (230). The video decoder (210) decodes the incoming copy of the encoded video data (207) and creates an output stream of video pictures (211) that can be rendered on a display (212) (e.g., a display screen) or other rendering device (not shown). In some streaming systems, the encoded video data 204, 207, and 209 (e.g., the video stream) may be encoded according to a particular video encoding / compression standard, examples of which include ITU-T Recommendation H.265.In one example, the developing video coding standard is known informally as Versatile Video Coding, or VVC, and the disclosed subject matter may be used in the context of VVC.

[0033] It should be noted that electronic devices 220 and 230 can include other components (not shown), such as electronic device 220 can include a video decoder (not shown), and similarly electronic device 230 can include a video encoder (not shown).

[0034] 3 shows a block diagram of a video decoder (310) according to an embodiment of the present disclosure. The video decoder (310) may be included in an electronic device (330). The electronic device (330) may include a receiver (331) (e.g., a receiving circuit). The video decoder (310) may be used in place of the video decoder (210) of the example of FIG. 2.

[0035] The receiver (331) may receive one or more encoded video sequences to be decoded by the video decoder (310); in the same or another embodiment, one encoded video sequence at a time, where the decoding of each encoded video sequence is independent of the other encoded video sequences. The encoded video sequences may be received from a channel (301), which may be a hardware / software link to a storage device that stores the encoded video data. The receiver (331) may receive the encoded video data along with other data, such as encoded audio data and / or auxiliary data streams, that may be transferred to each of them using entities (not shown). The receiver (331) may separate the encoded video stream from other data. To combat network jitter, a buffer memory (315) may be coupled between the receiver (331) and the entropy decoder / parser (320) (hereinafter "parser (320)"). In certain applications, the buffer memory (315) is part of the video decoder (310). In other examples, the buffer memory (315) can be external to the video decoder (310) (not shown). In other examples, there can be a buffer memory (not shown) external to the video decoder (310), e.g., combating network jitter, in addition to another buffer memory (315) internal to the video decoder (310), e.g., handling playback timing. When the receiver (331) is receiving data from a store / forward device of sufficient bandwidth and controllability or from an isosynchronous network, the buffer memory (315) may not be needed or may be small. For use in such best-effort packet networks, the buffer memory (315) may be needed and may be relatively large, advantageously adaptively sized, and may be implemented, at least in part, outside the video decoder (310) in an operating system or similar element (not shown).

[0036] The video decoder (310) may include a parser (320) to reconstruct symbols (321) from the encoded video sequence. These categories of symbols include information used to manage the operation of the video decoder (310) and information to control a rendering device such as a rendering device (312) (e.g., a display screen) that is potentially not an integral part of the electronic device (33) as shown in FIG. 3, but may be coupled to the electronic device (330). The control information for the rendering device may be in the form of a Supplementary Enhancement Information (SEI) message or a Video Usability Information (VUI) parameter set fragment (not shown). The parser (320) may parse / entropy decode the received encoded video sequence. The encoding of the encoded video sequence may be according to a video encoding technique or standard and may follow various principles including variable length coding, Huffman coding, arithmetic coding with or without context sensitivity, etc. The parser (320) may extract from the encoded video sequence a set of subgroup parameters for at least one of the subgroups of pixels in the video decoder based on at least one parameter of the group. The subgroups may include a group of pictures (GOP), a picture, a tile, a slice, a microblock, a coding unit (CU), a block, a transform unit (TU), a prediction unit (PU), etc. The parser (320) may also extract information from the encoded video sequence, such as transform coefficients, quantized parameter values, motion vectors, etc.

[0037] The parser (320) may perform entropy decoding / parsing operations on the video sequence received from the buffer memory (315) to produce symbols (321).

[0038] The reconstruction of the symbols (321) may involve several different units, depending on the type of coded video picture or part thereof (e.g., inter and intra pictures, inter and intra blocks) and other factors. Which units are involved and how can be controlled by subgroup control information parsed by the parser (320) from the coded video sequence. In what follows, the flow of such subgroup control information between the parser (320) and the several units is not shown for clarity.

[0039] In addition to the functional blocks already mentioned, the video decoder (310) may be conceptually subdivided into a number of functional units as described below. In a practical implementation operating under commercial constraints, many of these units may closely interact with each other and may be at least partially integrated with each other. However, for purposes of describing the disclosed subject matter, a conceptual subdivision into functional units hereinafter is appropriate.

[0040] The first unit is a scalar / inverse transform unit (351), which receives quantized transform coefficients as well as control information including the transform to use, block size, quantization factor, quantization scaling metric, etc. as symbols (321) from the parser (32). The scalar / inverse transform unit (351) can output blocks containing sample values ​​that can be input to an aggregator (355).

[0041] In some cases, the output samples of the scalar / inverse transform unit (351) may relate to intra-coded blocks, i.e., blocks that do not use prediction information from a previously reconstructed picture, but can use prediction information from a previously reconstructed portion of the current picture. Such prediction information may be provided by an intra-picture prediction unit (352). In some cases, the intra-picture prediction unit (352) uses surrounding already reconstructed information fetched from a current picture buffer (358) to generate a block of the same size and the shape of that block during reconstruction. The current picture buffer (358) buffers, for example, a partially reconstructed current picture and / or a fully reconstructed current picture. The aggregator (355) optionally adds, on a sample-by-sample basis, the prediction information generated by the prediction unit (352) to the output sample information as provided by the scalar / inverse transform unit (351).

[0042] In other cases, the output samples of the scalar / inverse transform unit (351) may relate to an inter-coded, potentially motion-compensated block. In such cases, the motion compensated prediction unit (353) may access the reference picture memory (357) to fetch samples used for prediction. After motion compensating the fetched samples according to the symbols (321) related to the block, these samples may be added by the aggregator (355) to the output of the scalar / inverse transform unit (351) (in this case called residual samples or residual signals) to generate output sample information. The addresses in the reference picture memory (357) from which the motion compensated prediction unit (353) fetches prediction samples may be controlled by motion vectors available to the motion compensated prediction unit (343) in the form of symbols (321), which may have, for example, X, Y, and reference picture components. Motion compensation may also include interpolation of sample values ​​fetched from the reference picture memory (357) when sub-sample accurate motion vectors are used, motion vector prediction mechanisms, etc.

[0043] The output samples of the aggregator (355) can be subjected to various loop filtering techniques in the loop filter unit (356). The video compression techniques can include in-loop filter techniques controlled by parameters contained in the coded video sequence (also called coded video bitstream) and made available to the loop filter unit (356) as symbols (321) from the parser (320), but can also be responsive to previously reconstructed and loop filtered sample values ​​as well as meta-information obtained during decoding of a coded picture or previous part of the coded video sequence (in decoding order).

[0044] The output of the loop filter unit (356) can be output to a rendering device (312), can be stored in a reference picture memory (357) for use in future inter-picture prediction, and can be a sample stream.

[0045] Once a particular coded picture has been sufficiently reconstructed, it can be used as a reference picture for future prediction. For example, once a coded picture corresponding to a current picture has been sufficiently reconstructed to identify the coded picture as a reference picture (e.g., by the parser (320)), the current picture buffer (358) can become part of the reference picture memory (357), and a fresh current picture buffer can be reallocated before beginning reconstruction of a subsequent coded picture.

[0046] The video decoder (310) may perform decoding operations according to a given video compression technique of a standard such as ITU-T Rec. H265. The encoded video sequence may conform to the syntax specified by the video compression technique or standard used in the sense that the encoded video sequence adheres to both the syntax of the video compression technique or standard and to the profile as a document of the video compression technique or standard. In particular, a profile may select a particular tool as the only available tool for use under the profile from all of the tools available in the video compression technique or standard. Also, what is necessary for conformance is that the complexity of the encoded video sequence is within a range defined by the level of the video compression technique or standard. In some cases, the level constrains the maximum picture size, maximum frame rate, maximum reconstructed sample rate (e.g., measured in megasamples per second), maximum reference picture size, etc. The limits set by the level may be further constrained in some cases through the specification of a Hypothetical Reference Decoder (HRD) and metadata for HRD buffer management signaled in the encoded video sequence.

[0047] In an embodiment, the receiver (331) may receive additional (redundant) data along with the encoded video. The additional data may be included as part of the encoded video sequence. The additional data may be used by the video decoder (310) to properly decode the data and / or to more accurately reconstruct the original video data. The additional data may be in the form of, for example, temporal, spatial or signal-to-noise ratio (SNR) enhancement layers, redundant slices, redundant pictures, forward error correction codes, etc.

[0048] 4 shows a block diagram of a video encoder (403) according to an embodiment of the present disclosure. The video encoder (403) is included in an electronic device (420). The electronic device (420) includes a transmitter (440) (e.g., a transmission circuit). The video encoder (403) can be used in place of the example video encoder (203) of FIG. 2.

[0049] The video encoder (403) may receive video samples from a video source (401) (which, in the example of FIG. 4, is not part of the electronic device (420)), which may capture video images to be encoded by the video encoder (403). In another example, the video source (401) is part of the electronic device (420).

[0050] The video source (401) may provide a source video sequence to be encoded by the video encoder (403) in the form of a digital video sample stream that may be of any suitable bit depth (e.g., 8-bit, 10-bit, 12-bit, ...), any color space (e.g., BT.601 Y CrCB, RGB, ...), and any suitable sampling structure (e.g., Y CrCb 4:2:0, Y CrCb 4:4:4). In a media serving system, the video source (401) may be a storage device that stores previously prepared video. In a video conferencing system, the video source (401) may be a camera that captures local image information as a video sequence. The video data may be provided as a number of individual pictures that convey motion when viewed as a sequence. The pictures themselves may be organized as a spatial array of pictures, where each picture may contain one or more samples depending on the sampling structure, color space, etc. in use. Those skilled in the art can easily understand the relationship between pixels and samples. The following description focuses on samples.

[0051] According to an embodiment, the video encoder (403) can encode and compress pictures of a source video sequence into an encoded video sequence (443) in real-time or under any other time constraint required by the application. Enforcing an appropriate encoding speed is one function of the controller (450). In some embodiments, the controller (450) controls and is functionally coupled to other functional units described below. Coupling is not shown for clarity. Parameters set by the controller (450) can include rate control related parameters (picture skip, quantizer, lambda value for rate distortion optimization techniques, ...), picture size, group of pictures (GOP) layout, maximum motion vector search range, etc. The controller (45) can be configured with other appropriate functions related to the video encoder (403) optimized for a particular system design.

[0052] In some embodiments, the video encoder (403) is configured to operate in an encoding loop. As an oversimplified explanation, in one example, the encoding loop can include a source coder (430) (responsible for creating symbols, such as a symbol stream, based on an input picture to be encoded and reference pictures) and a (local) decoder (433) embedded in the video encoder (403). The decoder (433) reconstructs the symbols to create sample data in a similar manner to how the (remote) decoder does so (so that any compression between the symbols and the encoded video bitstream is lossless with the video compression techniques considered in the disclosed subject matter). The reconstructed sample stream (sample data) is input to a reference picture memory (434). The contents of the reference picture memory (434) are also bit-exact between the local and remote encoders, since the decoding of the symbol stream results in bit-exact results that are independent of the location of the decoder (local or remote). In other words, the prediction part of the encoder "sees" exactly the same sample values ​​as the reference picture samples that the decoder "sees" when using the prediction during decoding. This fundamental principle of reference picture synchrony (and the resulting drift when synchrony cannot be maintained, e.g., due to channel errors) is used in a number of related technologies as well.

[0053] The operation of the "local" decoder (433) can be the same as that of a "remote" decoder, such as the video decoder (310), already detailed above in connection with Figure 3. However, briefly referring also to Figure 3, symbols are available, and the encoding / decoding of the symbols into an encoded video sequence by the entropy coder (445) and parser (320) can be lossless, and the entropy decoding portion of the video decoder (310), including the buffer memory (315) and the parser (320), may not be fully implemented in the local decoder (433).

[0054] An observation that can be made at this point is that any decoder technology, other than parsing / entropy decoding, present in the decoder must necessarily be present in substantially identical functional form in the corresponding encoder. For this reason, the disclosed subject matter focuses on decoder operation. A description of the encoder technology can be omitted, as it is the opposite of the decoder technology, which is described in general terms. Only in certain areas is a more detailed description necessary, and is provided below.

[0055] During operation, in some examples, the source coder (430) may perform motion-compensated predictive coding, which predictively codes an input picture with reference to one or more previously coded pictures from a video sequence designated as “reference pictures.” In this manner, the coding engine (432) codes differences between pixel blocks of the input picture and pixel blocks of reference pictures that may be selected as predictive references for the input picture.

[0056] The local video decoder (433) may decode the encoded video data of pictures that may be designated as reference pictures based on the symbols created by the source coder (430). The operation of the encoding engine (432) may advantageously be a lossy process. When the encoded video data may be decoded in a video decoder (not shown in FIG. 4), the reconstructed video sequence may be a replica of the source video sequence, typically with some errors. The local video decoder (433) may replicate the decoding process that may be performed by the video decoder on the reference pictures and store the reconstructed reference pictures in a reference picture cache (434). In this way, the video encoder (403) may locally store copies of reconstructed reference pictures with common content as reconstructed reference pictures (without transmission errors) to be obtained by the far-end video decoder.

[0057] The predictor (435) may perform a predictive search for the coding engine (432). That is, for a new picture to be coded, the predictor (435) may search the reference picture memory (434) for sample data (as candidate reference pixel blocks) or specific metadata, such as reference picture motion vectors, block shapes, etc., which may serve as suitable prediction references for the new picture. The predictor (435) may operate on a sample block-by-pixel block basis to find suitable prediction references. In some cases, as determined by the search results obtained by the predictor (435), the input picture may have prediction references drawn from multiple reference pictures stored in the reference picture memory (434).

[0058] The controller (450) may manage the encoding operations of the source coder (430), including, for example, setting the parameters and subgroup parameters used to encode the video data.

[0059] All outputs of the aforementioned functional units may undergo entropy coding in an entropy coder (445), which converts the symbols produced by the various functional units into an encoded video sequence by losslessly compressing the symbols according to techniques known to those skilled in the art, such as Huffman coding, variable length coding, arithmetic coding, etc.

[0060] The transmitter (440) may buffer the encoded video sequence created by the entropy coder (445) and prepare it for transmission over a communication channel (460), which may be a hardware / software link to a storage device that stores the encoded video data. The transmitter (440) may merge the encoded video data from the video coder (403) with other data to be transmitted, such as encoded audio data and / or auxiliary data streams (sources not shown).

[0061] The controller (450) may manage the operation of the video encoder (403). During encoding, the controller (450) may assign each coded picture a particular coding picture type, which may affect the coding technique that may be applied to the respective picture. For example, pictures may often be assigned as one of the following picture types:

[0062] An intra picture (I-picture) may be one that can be coded and decoded without using any other picture in a sequence as a source of prediction. Some video codecs allow different types of intra pictures, including, for example, Independent Decoder Refresh ("IDR") pictures. Those skilled in the art are aware of these variations of I-pictures and their respective uses and characteristics.

[0063] A predictive picture (P-picture) may be one that can be coded and decoded using intra- or inter-prediction, using at most one motion vector and reference index to predict the sample values ​​of each block.

[0064] Bidirectionally predicted pictures (B-pictures) may be those that can be coded and decoded using intra- or inter-prediction, using up to two motion vectors and reference indices to predict the sample values ​​of each block. Similarly, multiple predicted pictures may use two or more reference pictures and associated metadata for the reconstruction of a single block.

[0065] A source picture is generally spatially divided into multiple sample blocks (e.g., blocks of 4x4, 8x8, 4x8 or 16x16 samples each) and may be coded block by block. Blocks may be predictively coded with reference to other (already coded) blocks as determined by the coding assignment applied to the respective picture of the block. For example, blocks of I-pictures may be non-predictively coded, or they may be predictively coded with reference to already coded blocks of the same picture (spatial or intra prediction). Pixel blocks of P-pictures may be predictively coded via spatial or temporal prediction with reference to one previously coded reference picture. Blocks of B-pictures may be predictively coded via spatial or temporal prediction with reference to one or two previously coded reference pictures.

[0066] The video encoder (403) may perform encoding operations according to a given video encoding technique or standard, such as ITU-T Rec. H.265. In its operations, the video encoder (403) may perform various compression operations, including predictive encoding operations that exploit temporal and spatial redundancy in the input video sequence. Thus, the encoded video data may conform to a syntax specified by the video encoding technique or standard being used.

[0067] In one embodiment, the transmitter (440) may transmit additional data along with the encoded video. The source coder (430) may include such data as part of the encoded video sequence. The additional data may include temporal / spatial / SNR enhancement layers, other forms of redundant data such as redundant pictures and slices, Supplementary Enhancement Information (SEI) messages, Visual Usability Information (VUI) parameter set fragments, etc.

[0068] A video may be captured as multiple source pictures (video pictures) in a time sequence. Intra-picture prediction (often abbreviated as intra-prediction) uses spatial correlations of a given picture, while inter-picture prediction uses correlations (temporal or other) between pictures. In one example, a particular picture being coded / decoded, called the current picture, is divided into blocks. When a block in the current picture resembles a reference block in a previously coded and still buffered reference picture in the video, the block in the current picture may be coded by a vector called a motion vector. A motion vector points to a reference block in a reference picture and may have a third dimension that identifies the reference picture if multiple reference pictures are used.

[0069] In some embodiments, a bi-prediction technique can be used in inter-picture prediction. According to the bi-prediction technique, two reference pictures, such as a first and a second reference picture, are used that are both ahead in decoding order (but may be past and future, respectively, in display order) relative to a current picture in a video. A block in the current picture can be coded by a first motion vector that points to a first reference block in the first reference picture and a second motion vector that points to a second reference block in the second reference picture. The block can be predicted by a combination of the first and second reference blocks.

[0070] Furthermore, merge mode techniques can be used in inter-picture prediction to improve coding efficiency.

[0071] According to some embodiments of the present disclosure, predictions such as inter-picture prediction and intra-picture prediction are performed in units of blocks. For example, according to the HEVC standard, pictures in a sequence of video pictures are divided into coding tree units (CTUs) for compression, and the CTUs in a picture have the same size, such as 64×64 pixels, 32×32 pixels, or 16×16 pixels. In general, a CTU includes three coding tree blocks (CTBs), one luma CTB and two chrominance CTBs. Each CTU can be recursively quad-tree partitioned into one or more coding units (CUs). For example, a CTU of 64×64 pixels can be partitioned into one coding unit (CU) of 64×64 pixels, four CUs of 32×32 pixels, or 16 CUs of 16×16 pixels. In one example, each CU is analyzed to determine a prediction type of the CU, such as an inter prediction type or an intra prediction type. A CU is divided into one or more prediction units (PUs) depending on temporal and / or spatial predictability. In general, each PU includes one luma prediction block (PB) and two chrominance PBs. In one embodiment, prediction operations in coding (encoding / decoding) are performed in units of prediction blocks. Taking a luma prediction block as an example of a prediction block, the prediction block includes a matrix of values ​​(e.g., luma values) for pixels of 8×8 pixels, 16×16 pixels, 8×16 pixels, 16×8 pixels, etc.

[0072] 5 shows a diagram of a video encoder (503) according to another embodiment of the present disclosure. The video encoder (503) is configured to receive a processed block (e.g., a predictive block) of sample values ​​in a current video picture in a sequence of video pictures and to encode the processed block into an encoded picture that is part of an encoded video sequence. In one embodiment, the video encoder (503) is used in place of the example video encoder (203) of FIG. 2.

[0073] In an HEVC example, the video encoder (503) receives a matrix of sample values ​​for a processing block, such as a predictive block of 8x8 samples. The video encoder (503) determines whether the processing block is best coded using an intra mode, an inter mode, or a bi-predictive mode, e.g., with rate-distortion optimization. When the processing block is coded in an intra mode, the video encoder (503) may code the processing block into a coded picture using an intra prediction technique; when the processing block is coded in an inter mode or a bi-predictive mode, the video encoder (503) may code the processing block into a coded picture using an inter prediction or bi-predictive technique, respectively. In certain video coding techniques, the merge mode may be an inter-picture prediction sub-mode, in which case motion vectors are derived from one or more motion vector predictors without the benefit of coded motion vector components external to the predictors. In certain other video coding techniques, there may be motion vector components applicable to the current block. In one embodiment, the video encoder (503) includes other components, such as a mode decision module (not shown) for determining the mode of the processing blocks.

[0074] In the example of FIG. 5, the video encoder (503) includes an inter-encoder (530), an intra-encoder (522), a residual calculator (523), a switch (526), ​​a residual encoder (524), a general controller (521), and an entropy encoder (525), which are coupled together as shown in FIG. 5.

[0075] The inter-encoder (530) is configured to receive samples of a current block (e.g., a processing block), compare the block to one or more reference blocks in a reference picture (e.g., blocks in previous and future pictures), generate inter-prediction information (e.g., an account of redundant information due to inter-coding techniques, motion vectors, merge mode information), and calculate an inter-prediction result (e.g., a prediction block) based on the inter-prediction information using any suitable technique.

[0076] The intra encoder (522) is configured to receive samples of a current block (e.g., a processing block), optionally compare the block with already encoded blocks in the same picture, generate transformed and quantized coefficients, and optionally also generate intra prediction information (e.g., intra prediction direction information according to one or more intra encoding techniques).

[0077] The general controller (521) is configured to determine general control data and control other components of the video encoder (503) based on the general control data. In one example, the general controller (521) determines the mode of the block and provides a control signal to the switch (526) based on the mode. For example, when the mode is intra, the general controller (521) controls the switch (526) to select an intra mode result for use by the residual calculator (523) and controls the entropy encoder (525) to select intra prediction information and include the intra prediction information in the bitstream, and when the mode is inter mode, the general controller (521) controls the switch (526) to select an inter prediction result for use by the residual calculator (523) and controls the entropy encoder (525) to select inter prediction information and include the inter prediction information in the bitstream.

[0078] The residual calculator (523) is configured to calculate a difference (residual data) between a received block and a prediction result selected from the intra-encoder (522) or the inter-encoder (530). The residual encoder (524) operates on the residual data and is configured to encode the residual data to generate transform coefficients. In one example, the residual encoder (524) is configured to transform the residual data in the frequency domain to generate transform coefficients. The transform coefficients are then subjected to a quantization process to obtain quantized transform coefficients.

[0079] The entropy encoder (525) is configured to format the bitstream to include the encoded block. The entropy encoder (525) is configured to include various information according to an appropriate standard, such as the HEVC standard. In one example, the entropy encoder (525) is configured to include general control data, selected prediction information (e.g., intra-prediction information or inter-prediction information), residual information, and other appropriate information in the bitstream. It should be noted that when encoding a block in a merged sub-mode of either an inter mode or a bi-prediction mode in accordance with the disclosed subject matter, no residual information is present.

[0080] 6 shows a diagram of a video decoder (610) according to another embodiment of the present disclosure. The video decoder (610) is configured to receive encoded pictures that are part of an encoded video sequence and decode the encoded pictures to generate reconstructed pictures. In one example, the video decoder (610) is used in place of the video decoder (210) of the example of FIG. 2.

[0081] In the example of FIG. 6, the video decoder (610) includes an entropy decoder (671), an inter-decoder (680), a residual decoder (673), a reconstruction module (674), and an intra-decoder (672), coupled together as shown in FIG.

[0082] The entropy decoder (671) may be configured to reconstruct from the coded picture certain symbols that represent the syntax elements of which the coded picture is composed. Such symbols may include, for example, prediction information (e.g., intra-prediction information or inter-prediction information) that may identify the mode in which the block is coded (e.g., intra, inter, bi-prediction, merged submode or the latter two in another submode, etc.), certain samples or metadata used for prediction by the intra decoder (672) or inter decoder (680), respectively, residual information in the form of, for example, quantized transform coefficients, etc. In one example, when the prediction mode is an inter or bi-prediction mode, the inter prediction information is provided to the inter decoder (680); when the prediction type is an intra prediction type, the intra prediction information is provided to the intra decoder (672). The residual information may undergo inverse quantization and is provided to the residual decoder (673).

[0083] The inter decoder (680) is configured to receive the inter prediction information and to generate inter prediction results based on the inter prediction information.

[0084] The intra decoder (672) is configured to receive the intra prediction information and to generate a prediction result based on the intra prediction information.

[0085] The residual decoder (673) is configured to perform inverse quantization to extract de-quantized transform coefficients and process the de-quantized transform coefficients to transform the residual from the frequency domain to the spatial domain. The residual decoder (673) may also require certain control information (including a quantizer parameter QP), which may be provided by the entropy decoder (671) (data path not shown as it may be only low volume control information).

[0086] The reconstruction module (674) is configured to combine, in the spatial domain, the residual as output by the residual decoder (673) and the prediction result (possibly as output by an inter- or intra-prediction module) to form a reconstructed block, which may be part of a reconstructed picture, which may be part of a reconstructed video. It should be noted that other suitable operations, such as a deblocking operation, may be performed to improve visual quality.

[0087] It should be noted that the video encoders (203), (403), and (503) and the video decoders (210), (310), and (610) may be implemented using any suitable technology. In one embodiment, the video encoders (203), (403), and (503) and the video decoders (210), (310), and (610) may be implemented using one or more integrated circuits. In another embodiment, the video encoders (203), (403), and (403) and the video decoders (210), (310), and (610) may be implemented using one or more processors executing software instructions.

[0088] Merge candidates may be formed by checking motion information from spatial or temporal neighboring blocks of the current block. With reference to FIG. 7, the current block (701) contains samples found by the encoder / decoder during the motion search process that are predictable from a previous block of the same size but shifted spatially. In some embodiments, instead of directly encoding its motion vector, the motion vector may be derived from metadata associated with one or more reference pictures, from the most recent reference picture (in decoding order), for example using a motion vector associated with any of the five surrounding samples denoted D, A, C, B and E (702-706, respectively). Blocks A, B, C, D and E may be referred to as spatial merge candidates. These candidates may be successively checked into a merge candidate list. A pruning operation may be performed to ensure that duplicate candidates are removed from the list.

[0089] In some embodiments, after putting the spatial candidates into the merge list, the temporal candidates are also checked and listed. For example, find the collocated block of the current block in the specified reference picture. The motion information at C0 position (707) in the reference picture is used as the temporal merge candidate. C0 position may be a block in the reference picture whose top left corner is the bottom right corner of the collocated block in the reference picture of the current block 701. The collocated block in the reference picture may have the same location coordinates (e.g., x and y coordinates) as the current block 701. If the block at C0 position (707) is not coded in inter mode or is not available, the block at C1 position may be used. The block at C1 position may have its top left corner at a central position (e.g., w / 2, h / 2) of the block in the collocated block in the reference picture. In particular, the block at position C1 may be a sub-block of the collocated block of the reference picture. In the above example, w and h are the width and height of the block, respectively. According to some embodiments, the additional merging candidates include a combined bi-predictive candidate and a zero motion vector candidate.

[0090] Skip mode may be used to indicate for a block that motion data is estimated instead of being explicitly signaled and that the prediction residual is zero (i.e., no transform coefficients are transmitted). At the beginning of each CU in an inter-picture predicted slice, a skip flag (e.g., skip_flag) may be signaled that implies one or more of: (i) the CU contains only one PU (e.g., 2Nx2N); (ii) merge mode is used to derive the motion data; or (iii) no residual data is present in the bitstream.

[0091] According to some embodiments, the sub-CU mode is enabled as an additional merge candidate. In some embodiments, no additional syntax elements are used to signal the sub-CU mode. In some embodiments, two additional merge candidates are added to the merge candidate list of each CU to represent an alternative temporal motion vector prediction (ATMVP) mode and a spatial-temporal motion vector prediction (STMVP) mode.

[0092] The sequence parameter set may indicate the number of candidates in the merge list. For example, if the sequence parameter set indicates that ATMVP and STMVP are enabled, up to seven merge candidates may be used in the merge list. The coding logic of the additional merge candidates may be the same as other merge candidates in the merge candidate list, so that two or more rate-distortion (RD) checks are performed for two additional merge candidates per CU in a P slice or B slice. The order of the merge candidates may be A, B, C, D, ATMVP, STMVP, E (when there are less than six merge candidates in the list), temporal candidates, combined bi-predictive candidates, and zero motion vector candidates. The merge candidate list may be referenced by a merge index. In some embodiments, all bins of the merge index are context coded by context-adaptive binary arithmetic coding (CABAC). In other embodiments, only the first bin is context coded, and the remaining bins are context-bypass coded.

[0093] According to some embodiments, candidate motion vectors are searched from previously coded blocks with a step size of 8x8 blocks. Figure 8 shows a current block 800 surrounded by 8x8 blocks. The closest spatial neighbors are category 1 candidates, including the immediately upper row (i.e., the row containing blocks mv0 and mv1), the left column (i.e., the column containing mv2) and the top right corner (i.e., mv2) as category 1. Category 2 candidates may include blocks in the outer region that are far from the current block boundary and juxtaposed in the previously coded frame. Category 2 candidates may include up to three candidates. In Figure 8, category 2 candidates may be selected from the outer upper row (i.e., the row containing blocks mv4 and mv5) and the outer left column (i.e., the column containing blocks mv5 and mv6). Neighboring blocks predicted from different reference frames or intra-coded may be pruned from the list. The remaining reference blocks may each be assigned a weight. The weight may be related to the distance to the current block. 8, the candidate list may include the following category 1 candidates: mv1, mv0, mv2, and mv3. The candidate list may further include the following category 2 candidates: mv5, mv6, and mv4.

[0094] According to some embodiments, the extended merge mode includes additional merge candidates that include blocks that are not immediately adjacent to the current block. These candidates may be in the left, top, bottom-left, top-right and top-left directions. The maximum number of merge candidates may be 10. FIG. 9 shows a current block 900 surrounded on the left, top-left, top and top-right by reference blocks (i.e., blocks with a diagonal pattern). The reference blocks may include neighboring blocks A, B, C, D and E, which correspond to blocks A, B, C, D and E, respectively, in FIG. 7. In FIG. 9, the top-left corner of the reference block may have an offset of (-96, -96) with respect to the current block 900. Each candidate block B(i,j) or C(i,j) may have an offset of 16 in the vertical direction compared to its previous B or C candidate block, respectively. Each candidate block A(i,j) or D(i,j) may have an offset of 16 in the horizontal direction compared to its previous A or D candidate block, respectively. Each E(i,j) block may have an offset of 16 in both horizontal and vertical directions compared to its previous E candidate. The candidates may be checked in the direction from the reference block closest to the current block 900 to the reference block furthest from the current block 900. The order of candidates checked may be A(i,j), B(i,j), C(i,j), D(i,j) and E(i,j).

[0095] In FIG. 9, the extended neighboring positions may be determined for the current block 900 or for the current picture including the current block 900. According to some embodiments, instead of fetching values ​​from these extended neighboring positions, motion information of N previously coded blocks is stored in a history motion vector prediction (HMVP) buffer to provide more motion vector prediction candidates. The HMVP buffer may contain multiple HMVP candidates and may be maintained during the encoding / decoding process. In some embodiments, the HMVP buffer may operate on a first-in-first-out (FIFO) principle so that the latest coded motion information may be considered first when the HMVP buffer is used during a motion vector prediction process, such as in merge mode or AMVP mode.

[0096] The embodiments of the present disclosure disclose several methods of obtaining a motion vector predictor for inter-picture predictive coding. These methods include using an MV predictor from a history-based MV buffer and performing buffer management. These methods may be applied to both merge mode and motion vector prediction with differential coding (AMVP mode). The embodiments of the present disclosure may be extended to any video coding method that uses merge and general MV prediction concepts. The embodiments of the present disclosure may also be applied to skip mode, since skip mode uses merge mode to derive motion information.

[0097] Figures 10A and 10B show the HMVP buffer before and after candidates are inserted, respectively. As shown in Figures 10A and 10B, the HMVP buffer includes five entries with indexes [0] to [4]. In Figure 10B, entry CL_0 is inserted at index [4], which causes the other entries to move left one by one, resulting in the removal of entry HMPV_0 from the buffer. Entry CL_0 may include motion vector predictor information of a previously coded or decoded block.

[0098] According to some embodiments, each entry in the HMVP buffer is motion information from a previously coded block if the block is coded in inter coding mode. The block may be coded in bidirectional prediction mode with two motion vectors or unidirectional mode with one motion vector. For each entry in the HMVP buffer, if coded in bidirectional mode, the entry includes a pair of MVs as MV_L0 (with its reference index) and MV_L1 (with its reference index). According to some embodiments, the two unidirectional motion vectors corresponding to bidirectional mode include (i) MV_L0 for L0 prediction, using the reference index of L0 as the original predictor, and (ii) MV_L1 for L1 prediction, using the reference index of L1 as the original predictor.

[0099] In some embodiments, for each original bidirectional MV predictor in the HMVP buffer, two unidirectional MV predictors derived from the original MV predictor are also considered as new candidates in the merge list when the original bidirectional MV predictor is put into the merge candidate list. In one embodiment, MV_L0 and MV_L1 are put into the list after their corresponding original bidirectional MV predictors whenever the corresponding original bidirectional MV is put into the list. In another embodiment, MV_L0 and MV_L1 are put into the list after N original MV predictors from the HMVP buffer are put into the list, where N is an integer value. N may be the number of MV candidates allowed to be put into the merge list from the HMVP buffer, or N may be a fixed number less than the maximum allowed number copied from the HMVP buffer into the merge list. According to some embodiments, when an entry in the HMVP buffer is used to generate an MV predictor in AMVP mode, for each original bidirectional predictor in the HMVP buffer, a method similar to that described above may be used to generate the unidirectional predictors MV_L0 and MV_L1. These two predictors may be used as additional predictors in the AMVP MV prediction candidate list if the list is not full.

[0100] According to some embodiments, the HMVP buffer is emptied or reset to a zero state when a condition is met, which may be: (i) the current CU is the start of a CTU, (ii) the current CU is the start of a tile, (iii) the current CU is the start of a CTU row, or (iv) the current CU is the start of a slice.

[0101] According to some embodiments, after the first CTU of every CTU row is completed, an HMVP_row buffer with the same size HMVP buffer is used to store the entries of the HMVP buffer. Thus, at the start of a new CTU row, the HMVP buffer can be filled with the information in the HMVP_row buffer. By resetting the HMVP buffer at the end of the CTU row and copying the contents of the HMVP_row buffer to the HMVP buffer, the blocks of the first CTU being decoded can be decoded with information from the CTU directly above the first CTU.

[0102] In some embodiments, for each tile in a picture, the HMVP_row buffer is used to store HMVP information after the first CTU of each tile row is finished. Thus, for the first CTU of a new tile row, the HMVP buffer can be filled using information from the HMVP_row buffer. In some embodiments, the HMVP_row buffer starts in a zero state at the beginning of the first CTU row of a tile or slice.

[0103] 11 shows an example picture 1100 divided into CTU_00 to CTU_23. CTU_00 to CTU_03 are in the first CTU row CTU_Row_[0]. CTU_10 to CTU_13 are in the second CTU row CTU_Row_[1]. CTU_20 to CTU_23 are in the third CTU row. Each CTU in the picture 1100 may be further divided into multiple blocks. The blocks may be CUs or coding blocks (CBs).

[0104] In some embodiments, the HMVP buffer is loaded with an initial value before the first block in the first CTU (e.g., CTU_00) of the picture 1100 is decoded. The initial value may be stored in the memory of the encoder or decoder. In another example, the HMVP buffer may be initialized to a zero state (e.g., no valid entries in the buffer). In addition, the HMVP_row buffer may be initialized to a zero state before the first block in the first CTU (e.g., CTU_00) of the picture 1100 is encoded or decoded. When the last block of CTU_00 (e.g., the first CTU of CTU_Row_[0]) is encoded or decoded, the contents of the HMVP buffer are copied to the HMVP_row buffer. When the last block of CTU_03 is encoded or decoded (e.g., the last CTU of CTU_Row_[0]), and before the first block of CTU_10 is encoded or decoded (e.g., the first CTU of CTU_Row_[1]), the HMVP buffer is emptied and the contents of the HMVP_row buffer are copied to the HMVP buffer. Therefore, by resetting the HMVP buffer and copying the contents of the HMVP_row buffer to the HMVP buffer, encoding or decoding of blocks of CTU_10 can be performed using information from CTU_00 (e.g., blocks above CTU_10), which may be more relevant to encoding or decoding of CTU_10 than information from CTU_03.

[0105] A similar process of copying the contents of the HMVP buffer to HMVP_row may be performed after the last blocks of CTU_10 and CTU_20 are encoded or decoded. Additionally, a similar process of copying the contents of the HMVP buffer to clear and copying the contents of the HMVP_row buffer to the HVMP buffer may be performed after the last block of CTU_13 is encoded or decoded.

[0106] FIG. 12 shows an example of a picture 1100 divided into two tiles Tile_1 and Tile_2. In some embodiments, before the first block of the first CTU (e.g., CTU_00) of Tile_1 is encoded or decoded, the HMVP buffer is loaded with an initial value. The initial value may be stored in the memory of the encoder or decoder. The HMVP buffer may also be initialized to a zero state. In addition, the HMVP_row buffer may be initialized to a zero state before the first block in the first CTU (e.g., CTU_00) of the picture 1100 is encoded or decoded. When the last block of CTU_00 (e.g., the first CTU of CTU_Row_[0] of Tile_1) is encoded or decoded, the contents of the HMVP buffer are copied to the HMVP_row buffer. When the last block of CTU_01 is encoded or decoded (e.g., the last CTU of CTU_Row_[0] of Tile_1), the HMVP buffer is emptied and the contents of the HMVP_row buffer are copied to the HMVP buffer before the first block of CTU_10 is encoded or decoded (e.g., the first CTU of CTU_Row_[1] of Tile_1). Thus, by resetting the HMVP buffer and copying the contents of the HMVP_row buffer to the HMVP buffer, the encoding or decoding of the blocks of CTU_10 can be performed with information from CTU_00, which may be more relevant for encoding or decoding of CTU_10 than information from CTU_01.

[0107] Tile_2 may be encoded or decoded in parallel with Tile_1 and may have separate HMVP and HMVP_row buffers, each of which may be initialized similarly to Tile_1. When the last block of CTU_02 (e.g., the first CTU of CTU_Row_[0] of Tile_2) is encoded or decoded, the contents of the HMVP buffer are copied to the HMVP_row buffer. When the last block of CTU_03 is encoded or decoded (e.g., the last CTU of CTU_Row_[0] of Tile_2), the HMVP buffer is emptied and the contents of the HMVP_row buffer are copied to the HMVP buffer before the first block of CTU_12 is encoded or decoded (e.g., the first CTU of CTU_Row_[1] of Tile_2). Therefore, by resetting the HMVP buffer and copying the contents of the HMVP_row buffer into the HMVP buffer, encoding or decoding of blocks of CTU_12 can be performed with information from CTU_02, which may be more relevant to decoding CTU_12 than information from CTU_03.

[0108] FIG. 13 illustrates an embodiment of a process performed by an encoder, such as encoder 503, or a decoder, such as decoder 610. The process may start at step S1300, where a current picture is retrieved from a video bitstream. For example, picture 1100 (FIG. 11) may be the retrieved picture. The process proceeds to step S1302, where a current block is encoded / decoded using one or more entries from the HMVP buffer. For example, with reference to picture 1100, if a first block of CTU_00 is being encoded / decoded, the HMVP buffer may be initialized to an initial state, and the first block may be encoded / decoded with one or more entries from the HMVP buffer after the HMVP buffer is initialized. The process proceeds to step S1304, where the HMVP buffer is updated with motion vector information of the current block that has been encoded / decoded.

[0109] The process proceeds to step S1306 to determine whether the currently encoded / decoded block is at the end of a CTU row. For example, referring to picture 1100, if the current block to be encoded / decoded is the last block of CTU_03, the next block to be encoded / decoded is the first block of the next row, CTU_10. If the currently encoded / decoded block is at the end of a CTU row, the process proceeds to step S1308, in which the HMVP buffer is reset (e.g., emptied) and the contents of the HMVP_row buffer are copied to the HMVP buffer before the next block is encoded / decoded. The process proceeds from step S1308 to step S1314, which will be described in more detail below.

[0110] If the current coded / decoded block is not the end of the CTU row, the process proceeds to step S1310, where it is determined whether the current coded / decoded block is the last block of the first CTU in the CTU row. If the current coded / decoded block is the last block of the first CTU in the CTU row, the process proceeds to step S1312, where the contents of the HMVP buffer are copied to the HVMP_row buffer. For example, with reference to picture 1100, if the current coded / decoded block is the last block of CTU_00, the contents of the HVMP buffer are copied to the contents of the HMVP_row buffer before the first block of CTU_01 is coded / decoded. The process proceeds from step S1312 to step S1314, which will be described in more detail below.

[0111] If the current coded / decoded block is not the last block of the first CTU in the CTU row, the process proceeds to step S1314 to determine whether the current coded / decoded block is the last block in the obtained picture. If the current coded / decoded block is the last block in the obtained picture, the process of FIG. 13 ends. For example, if the current coded / decoded block is the last block of CTU_23, the process of FIG. 13 is completed. If the current coded / decoded block is not the last block in the obtained picture, the process returns from step S1314 to step S1302.

[0112] The techniques described above may be implemented as computer software using computer readable instructions and may be physically stored on one or more computer readable media. For example, Figure 14 illustrates a computer system (1400) suitable for implementing certain embodiments of the disclosed subject matter.

[0113] Computer software may be coded using any suitable machine code or computer language that may be subject to assembly, compilation, linking, or similar mechanisms to produce code containing instructions that may be executed by one or more computer central processing units (CPUs), graphics processing units (GPUs), etc. directly, or through translation, microcode execution, etc.

[0114] The instructions may be executed on various types of computers or components thereof, including, for example, personal computers, tablet computers, servers, smart phones, gaming devices, Internet of Things devices, etc.

[0115] 14 of the computer system (1400) are exemplary in nature and are not intended to suggest any limitation as to the scope of use or functionality of the computer software implementing the embodiments of the present disclosure, nor should the arrangement of components be interpreted as having any dependency or requirement related to any one or combination of components illustrated in the exemplary embodiment of the computer system (1400).

[0116] The computer system (1400) may include certain human interface input devices. Such human interface input devices may respond to input by one or more human users, for example, through tactile input (e.g., keystrokes, swipes, data glove movements), audio input (e.g., voice, clapping), visual input (e.g., gestures), and olfactory input (not shown). Human interface devices may also be used to capture certain media that are not necessarily directly related to conscious human input, such as audio (e.g., speech, music, ambient sounds), images (e.g., scanned images, photographic images obtained from still image cameras), and video (e.g., two-dimensional video, three-dimensional video including stereoscopic video).

[0117] The input human interface devices may include one or more of the following (only one of which is shown): a keyboard (1401), a mouse (1402), a trackpad (1403), a touch screen (1410), a data glove (not shown), a joystick (1405), a microphone (1406), a scanner (1407), and a camera (1408).

[0118] The computer system (1400) may also include certain human interface output devices. Such human interface output devices may stimulate one or more of the senses of a human user, for example, through haptic output, sound, light, and smell / taste. Such human interface output devices may include haptic output devices (e.g., haptic feedback via a touch screen (1410), data gloves (not shown), or joystick (1405), although there may also be haptic feedback devices that do not function as input devices), audio output devices (e.g., speakers (1409), headphones (not shown)), visual output devices (e.g., screens (1410), including CRT screens, LCD screens, plasma screens, OLED screens, each with or without touch screen input capability, each with or without haptic feedback capability, some of which may have the ability to output two-dimensional visual output or output beyond three-dimensional output through such means as stereoscopic output, virtual reality glasses (not shown), holographic displays, and smoke tanks (not shown)), and printers (not shown).

[0119] The computer system (1400) may also include human accessible storage devices and their associated media, such as a CD / DVD ROM / RW (1420) with CD / DVD or similar media (1421), thumb drives (1422), removable hard drives or solid state drives (1423), legacy magnetic media such as tapes and floppy disks (not shown), and special ROM / ASIC / PLD based devices such as security dongles (not shown).

[0120] Those skilled in the art should also understand that the term "computer-readable medium" as used in connection with the presently disclosed subject matter does not encompass transmission media, carrier waves, or other transitory signals.

[0121] The computer system (1400) may also include interfaces to one or more communication networks. The networks may be, for example, wireless, wired, optical. The networks may further be local, wide area, metropolitan, vehicular and industrial, real-time, delay tolerant, etc. Examples of networks include Ethernet, WLAN, cellular networks including GSM, 3G, 4G, 5G, LTE, etc., TV wired or wireless wide area digital networks including cable TV, satellite TV and terrestrial broadcast TV, vehicular and industrial including CANBus, etc. Certain networks generally require an external network interface adapter attached to a particular general-purpose data port or peripheral bus (1449) (e.g., a USB port on the computer system (1400)), while others are generally integrated into the core of the computer system (1400) by attachment to a system bus described below (e.g., an Ethernet interface to a PC computer system or a cellular network interface to a smartphone computer system). Using any of these networks, the computer system (1400) may communicate with other entities. Such communications may be unidirectional, receive only (e.g., broadcast TV), unidirectional transmit only (e.g., CANBus to a specific CANBus device), or bidirectional, e.g., to other computer systems using local or wide area digital networks. Specific protocols and protocol stacks may be used in each of these networks and network interfaces, as described above.

[0122] The aforementioned human interface devices, human accessible storage devices and network interfaces may be attached to a core (1440) of the computer system (1400).

[0123] The cores (1440) may include one or more central processing units (CPUs) (1441), graphics processing units (GPUs) (1442), special programmable processing units in the form of field programmable gate areas (FPGAs) (1443), hardware accelerators for specific tasks (1444), and the like. These devices may be connected via a system bus (1448), along with internal mass storage such as read only memory (ROM) (1445), random access memory (1446), internal non-user accessible hard drives, SSDs (1447), and the like. In some computer systems, the system bus (1448) may be accessible in the form of one or more physical plugs to allow expansion with additional CPUs, GPUs, and the like. Peripheral devices may be attached directly to the core's system bus (1448) or via a peripheral bus (1449). Peripheral bus architectures include PCI, USB, and the like.

[0124] The CPU (1441), GPU (1442), FPGA (1443) and accelerator (1444) may combine to execute certain instructions that may constitute the computer code described above. The computer code may be stored in ROM (1445) or RAM (1446). Transient data may also be stored in RAM (1446), while permanent data may be stored, for example, in internal mass storage (1447). A cache memory, which may be closely associated with one or more of the CPU (1441), GPU (1442), mass storage (1447), ROM (1445), RAM (1446), etc., may be used to enable fast storage and retrieval in any of the memory devices.

[0125] The computer-readable medium may have computer code thereon for performing various computer-implemented operations. The medium and computer code may be those specially designed and constructed for the purposes of the present disclosure, or they may be of the kind well known and available to those skilled in the computer software arts.

[0126] By way of example and not limitation, a computer system having the architecture (1400), and in particular the core (1440), can provide functionality as a result of a processor (including a CPU, GPU, FPGA, accelerator, etc.) executing software embodied in one or more tangible computer-readable media. Such computer-readable media can be user-accessible mass storage as described above, as well as media associated with specific storage of the core (1440) that is non-transitory in nature, such as the core internal mass storage (1447) or ROM (1445). Software implementing various embodiments of the present disclosure can be stored in such devices and executed by the core (1440). The computer-readable media can include one or more memory devices or chips, depending on the particular needs. The software can cause the core (1440) and in particular the processor therein (including a CPU, GPU, FPGA, etc.) to perform certain processes or certain portions of certain processes described herein, including defining data structures stored in RAM (1446) and modifying such data structures according to processes defined by the software. Additionally or alternatively, the computer system may provide functionality as a result of logic hardwired or otherwise embodied in circuitry (e.g., accelerator (1444)), which may operate in place of or in conjunction with software to perform particular processes or portions of particular processes described herein. References to software include logic, and vice versa, where appropriate. References to computer-readable media may include circuitry (e.g., integrated circuits (ICs)) that store software for execution, circuitry that embodies logic for execution, or both, as appropriate. The present disclosure encompasses any suitable combination of hardware and software. Appendix A: Acronyms MV: Motion Vector HEVC: High Efficiency Video Coding SEI: Supplemental Emphasis Information VUI: Video Usability Information GOP: Group of Pictures TU: conversion unit PU: Prediction Unit CTU: Coding Tree Unit CTB: coding tree block PB: Predicted block HRD: Hypothetical Reference Decoder SNR: Signal to Noise Ratio CPU: Central Processing Unit GPU: Graphics Processing Unit CRT: cathode ray tube LCD: Liquid crystal display OLED: Organic Light Emitting Diode CD:Compact Disc DVD: Digital Video Disc ROM: Read-Only Memory RAM: Random Access Memory ASIC: Application Specific Integrated Circuit PLD: Programmable Logic Device LAN: Local Area Network GSM: Global System for Mobile Communications LTE: Long Term Evolution CANBus: Controller Area Network Bus USB: Universal Serial Bus PCI: Peripheral Component Interconnect FPGA: Field Programmable Gate Area SSD: Solid State Drive IC: Integrated Circuit CU: Coding Unit

[0127] While this disclosure has described several exemplary embodiments, there are alterations, permutations, and various substitute equivalents that are within the scope of this disclosure. It will thus be appreciated that those skilled in the art will be able to devise many systems and methods that, although not explicitly shown or described herein, embody the principles of the present disclosure and are therefore within the spirit and scope of the present disclosure.

[0128] (1) A method of video decoding performed by a decoder, the method including the steps of: obtaining a current picture from an encoded video bitstream, the current picture being divided into a plurality of units, each unit being divided into a plurality of blocks, the plurality of blocks in each unit forming a grid; for one of the plurality of units, decoding a current block from the plurality of blocks using an entry from a history motion vector (HMVP) buffer; updating the HMVP buffer with a motion vector of the decoded current block; determining whether the current block is at the beginning of a row in the grid of one of the plurality of units; and resetting the HMVP buffer in response to determining that the current block is at the beginning of the row.

[0129] (2) the method of feature (1), further comprising the steps of: (a) determining whether the current block is the last block of a first unit of a row; and (b) copying the contents of the HMVP buffer to a row buffer in response to determining that the current block is the last block of the first unit of the row.

[0130] (3) The method according to feature (2), further comprising the step of copying the contents of the row buffer to the HMVP buffer after each entry of the HMVP buffer is reset in response to determining that the current block is the beginning of a row.

[0131] (4) A method according to any one of features (1) to (3), wherein the HMVP buffer is a first-in, first-out (FIFO) buffer, and the step of updating the HMVP buffer with the motion vector includes the steps of storing the motion vector in a last entry of the HMVP buffer and deleting a first entry of the HMVP buffer.

[0132] (5) A method according to any one of features (1) to (4), wherein the unit is a coding tree unit (CTU).

[0133] (6) A method according to any one of features (2) to (5), wherein the units are tiles, one of the units being decoded is a first tile, and the first tile and the second tile from the multiple units are decoded in parallel.

[0134] (7) A video decoder for video decoding, comprising: a processing circuit configured to: obtain a current picture from an encoded video bitstream, the current picture being divided into a plurality of units, each unit being divided into a plurality of blocks, the plurality of blocks in each unit forming a grid; for one of the plurality of units, decode a current block from the plurality of blocks using an entry from a history motion vector (HMVP) buffer; update the HMVP buffer with a motion vector of the decoded current block; determine whether the current block is at the beginning of a row included in the grid of one of the plurality of units; and reset the HMVP buffer in response to determining that the current block is at the beginning of the row.

[0135] (8) The video decoder of feature (7), wherein the processing circuitry is further configured to determine whether the current block is the last block of a first unit of a row, and in response to determining that the current block is the last block of the first unit of the row, copy contents of the HMVP buffer to the row buffer.

[0136] (9) The video decoder of feature (8), wherein the processing circuitry is further configured to copy contents of the row buffer to the HMVP buffer after each entry of the HMVP buffer is reset in response to determining that the current block is the beginning of a row.

[0137] (10) A video decoder according to any one of features (7) to (9), wherein the HMVP buffer is a first-in, first-out (FIFO) buffer, and updating the HMVP buffer with the motion vector includes storing the motion vector in a last entry of the HMVP buffer and deleting a first entry of the HMVP buffer.

[0138] (11) The video decoder according to any one of features (7) to (10), wherein the unit is a coding tree unit (CTU).

[0139] (12) A video decoder according to any one of features (8) to (11), wherein the units are tiles, one of the units decoded is a first tile, and the first tile and second tile from the multiple units are decoded in parallel.

[0140] (13) A non-transitory computer-readable medium having instructions that, when executed by a processor in a video decoder, cause the processor to perform a method comprising: obtaining a current picture from an encoded video bitstream, the current picture being divided into a plurality of units, each unit being divided into a plurality of blocks, the plurality of blocks in each unit forming a grid; for one of the plurality of units, decoding a current block from the plurality of blocks using an entry from a historical motion vector (HMVP) buffer; updating the HMVP buffer with a motion vector of the decoded current block; determining whether the current block is at the beginning of a row in the grid of one of the plurality of units; and resetting the HMVP buffer in response to determining that the current block is at the beginning of a row.

[0141] (14) The non-transitory computer-readable medium of feature (13), further comprising: determining whether the current block is the last block of a first unit of a row; and copying contents of the HMVP buffer to a row buffer in response to determining that the current block is the last block of the first unit of the row.

[0142] (15) The non-transitory computer-readable medium of feature (14), wherein the method further includes a step of copying contents of the row buffer to the HMVP buffer after each entry of the HMVP buffer is reset in response to determining that the current block is the beginning of a row.

[0143] (16) A non-transitory computer-readable medium according to any one of features (13) to (15), wherein the HMVP buffer is a first-in, first-out (FIFO) buffer, and the step of updating the HMVP buffer with the motion vector includes the steps of storing the motion vector in a last entry of the HMVP buffer and deleting a first entry of the HMVP buffer.

[0144] (17) The non-transitory computer-readable medium according to any one of features (13) to (16), wherein the unit is a coding tree unit (CTU).

[0145] (18) The non-transitory computer-readable medium of any one of features (14) to (17), wherein the units are tiles, one of the units being decoded is a first tile, and the first tile and second tile from the multiple units are decoded in parallel.

Claims

1. 1. A method for generating an encoded video bitstream performed by an encoder, comprising: generating an encoded video bitstream by encoding the input video bitstream; and transmitting the encoded video bitstream; Encoding the input video bitstream includes: obtaining a current picture from the input video bitstream, dividing the current picture into a plurality of units, and dividing the current picture into a plurality of tiles, each tile including at least one unit; encoding a first current unit in a first tile of the plurality of tiles; updating a first HMVP buffer with the motion vector of the first current unit that has been coded; determining a position of the first current unit in the first tile of the plurality of tiles; resetting the first HMVP buffer in response to determining that the first current unit is located in a first column of the first tile; A method for generating an encoded video bitstream, comprising:

2. 1. A method for generating an encoded video bitstream performed by an encoder, comprising: generating an encoded video bitstream by encoding the input video bitstream; and storing the encoded video bitstream; Encoding the input video bitstream includes: obtaining a current picture from the input video bitstream, dividing the current picture into a plurality of units, and dividing the current picture into a plurality of tiles, each tile including at least one unit; encoding a first current unit in a first tile of the plurality of tiles; updating a first HMVP buffer with the motion vector of the first current unit that has been coded; determining a position of the first current unit in the first tile of the plurality of tiles; resetting the first HMVP buffer in response to determining that the first current unit is located in a first column of the first tile; A method for generating an encoded video bitstream, comprising:

3. In response to determining that the first current unit is located in a first column of the first tile, copying contents of a first row buffer to the first HMVP buffer. A method for generating an encoded video bitstream according to claim 1 or 2.

4. In response to determining that the first current unit is located in the last column of the first tile, copying the contents of the first HMVP buffer to the first row buffer.

4. The method of claim 3 for generating an encoded video bitstream.

5. encoding a second current unit in a second tile of the plurality of tiles; updating a second HMVP buffer with the motion vector of the second current unit that has been coded; determining a position of the second current unit in the second tile of the plurality of tiles; resetting the second HMVP buffer in response to determining that the second current unit is located in a first column of the second tile; Further comprising: A method for generating an encoded video bitstream according to any one of claims 1 to 4.

6. In response to determining that the second current unit is located in a first column of the second tile, copying contents of a second row buffer to the second HMVP buffer.

6. A method for generating an encoded video bitstream according to claim 5.

7. In response to determining that the second current unit is located in the last column of the second tile, copying the contents of the second HMVP buffer to the second row buffer.

7. The method of claim 6 for generating an encoded video bitstream.

8. the encoding of the first current unit is performed in parallel with the encoding of the second current unit. A method for generating an encoded video bitstream according to any one of claims 5 to 7.

9. the first HMVP buffer and the second HMVP buffer are first-in, first-out (FIFO) buffers; updating the first HMVP buffer with the motion vector of the first current encoded unit includes storing the motion vector in a last entry of the first HMVP buffer and deleting a first entry of the first HMVP buffer; updating the second HMVP buffer with the motion vector of the second current encoded unit includes storing the motion vector in the last entry of the second HMVP buffer and deleting the first entry of the second HMVP buffer; A method for generating an encoded video bitstream according to any one of claims 5 to 8.

10. A video encoder including processing circuitry, A video encoder, wherein the processing circuitry is configured to perform a method for generating an encoded video bitstream according to any one of claims 1 to 9.

Images