Distance-based hmvp

Abstract

An apparatus includes at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to: determine that at least one distance from a current coding unit to at least one coding unit associated with at least one entry of a table is closer than other one or more distances from the current coding unit to other coding units associated with one or more entries of the table; wherein the entries of the table correspond to motion information of respective inter coded coding units; and code the current coding unit using the motion information corresponding to the at least one entry of the table.

Description

DISTANCE-BASED HMVPTECHNICAL FIELD

[0001] The examples and non-limiting embodiments relate generally to multimedia transport and, more particularly, to distance-based HMVP.BACKGROUND

[0002] It is known to perform data compression and data decompression in a multimedia system.BRIEF DESCRIPTION OF THE DRAWINGS

[0003] The foregoing embodiments and other features are explained in the following description, taken in connection with the accompanying drawings, wherein:

[0004] FIG. 1 shows a current CU (X) in a current CTU.

[0005] FIG. 2 shows motion information in an HMVP table.

[0006] FIG. 3 A shows an HMVP table before sorting.

[0007] FIG. 3B shows an HMVP table after sorting.

[0008] FIG. 4 shows schematically a user equipment suitable for employing embodiments of the examples described herein.

[0009] FIG. 5 is a block diagram illustrating a system in accordance with an example.

[0010] FIG. 6 is an example apparatus configured to implement the examples described herein.

[0011] FIG. 7 shows a representation of an example of non-volatile memory media used to store instructions that implement the examples described herein.

[0012] FIG. 8 shows an encoder according to an embodiment.

[0013] FIG. 9 shows a decoder according to an embodiment.

[0014] FIG. 10 is an example method, based on the examples described herein.DETAILED DESCRIPTION OF EXAMPLE EMBODIMENTS

[0015] Versatile Video Coding (VVC) is a new international video coding standard, and Enhanced Compression Model (ECM), built on top of VVC, is potentially a future video coding standard currently under development sponsored by JVET. Both VVC and ECM are block-based video coding standards, where an input picture is divided into Coding Tree Units (CTUs), and each CTU may be further split into Coding Units (CUs). A CU (or block) is coded in either inter-coding mode or intra-coding mode. If the block is in inter-coding mode, the encoder searches for a temporal prediction block in reference picture(s) and signals the decoder on how to find the same prediction block in reference picture(s) at the decoder end. If the block is in intra-coding mode, the encoder constructs a spatial prediction block from the current picture and signals the decoder on how to form the same spatial prediction block from the current picture at the decoder end.

[0016] For a current inter-CU in a current picture, the associated temporal prediction block in reference pictures is represented by motion information (e.g., motion vectors, reference pictures, and reference picture lists) with respect to the current CU in the current picture. The encoder signals the motion information to the decoder, and the decoder uses the motion information to form the temporal prediction block from reference pictures. The motion information is either coded directly or derived from the motion information of the past coded CUs either in the current picture or in reference pictures.

[0017] VVC and ECM supports many new and refined coding tools for deriving the motion information prediction for a current CU. One of the coding tools is merge prediction, in which for a current CU, both encoder and decoder construct a same list of merge candidates. The merge candidates hold the motion information of the past inter coded CUs around the current CU both spatially and temporally. Encoder selects a merge candidate (motion information) from the merge candidate list for the current CU, and signals decoder which merge candidate in the merge candidate list to be used for the current CU.

[0018] For a current CU in a current picture, a merge candidate list is constructed by including the following types of candidates (1-7): 1) Spatial merge candidates, 2) Temporal merge candidates, 3) Non-adjacent merge candidates, 4) History -based merge candidates (HMVP), 5) Pairwise average merge candidates, 6) History-based merge candidates from Affine HMVP, 7) Zero MV merge candidates.

[0019] HMVP merge candidates are derived from HMVP table. In the current design of VVC and ECM, the HMVP table has 5 entries. The HMVP table can hold motion information of up to 5 past inter-coded CUs. The HMVP table is updated after a nonsubblock inter-CU is coded, with the associated motion information (MI) being added to the HMVP table following the First-In-First-Out (FIFO) rule. Before adding new motion information into the HMVP table, an identical entry (motion information), if existing, or the oldest entry in the HMVP table, is removed from the HMVP table, and all the HMVP entries afterward are moved forward. The HMVP table is reset (emptied) at the beginning of each CTU row. When one or more HMVP entries are needed for coding a current CU, the one or more newest entries in the HMVP table are selected.

[0020] In VVC and ECM, for a current CU, HMVP merge candidates are derived from the HMVP table without taking the distances of the past inter-coded CUs to the current CU into consideration.

[0021] Described herein is a method, wherein when one or more HMVP entries are needed for coding a current CU, the one or more entries in the HMVP table with the associated CUs that have the closest distances to the current CU are selected. The herein described selection process likely aims to improve prediction accuracy by utilizing motion information from nearby coding units.

[0022] In VVC and ECM, for a current CU in a current picture, HMVP table contains the motion information of the past inter coded CUs in the current picture. In the current design of VVC and ECM, HMVP table is of size 5.

[0023] FIG. 1 shows an example, where a current CU (X) is in a CTU 100. For the current CU (X), the past inter coded CUs are CU (nO), CU (nl), CU (n2), CU (n3) and CU (n4). The motion information associated with CU (nO), CU (nl), CU (n2), CU (n3) and CU (n4)are, respectively, mO, ml, m2, m3, and m4. Thus, FIG. 1 shows a current CU (X) in a current CTU 100.

[0024] With the coding path specified in VVC and ECM, CU (nO) is coded first followed by CU (nl), CU (n2), CU (n3) and CU (n4). Hence, in the current design with VVC and ECM, mO enters HMVP table 200 first, followed by ml, m2, m3 and m4, as shown in FIG. 2. Thus, FIG. 2 shows motion information in an HMVP table.

[0025] When one or more HMVP entries are needed for coding a current CU, encoder and decoder selects the one or more newest entries in the HMVP table.

[0026] One practical implementation is to assign an index to each position in HMVP table.

[0027] In one embodiment, index 0 is assigned to the last position in the HMVP table, index 1 to the second last position of the HMVP table, and so on. FIG. 2 shows the index assignment for the positions of the HMVP table, where index 0 is assigned the most left position, index 1 to the second most left position, and so on. The HMVP table is filled from the most right position to the most left position. After the HMVP table is full, a new entry (MI) will enter the most left position, and pushes all the other entries forward from left to right and the one in the most right position is out, if no identical entry exists.

[0028] With such position indexing, when one or more HMVP entries are needed for coding a current CU, encoder and decoder can simply select the one or more entries with the smallest indices in the HMVP table.

[0029] In the example of FIG. 2, if one HMVP entry is needed for coding a current CU, m4 is selected because m4 is in the position with index 0.

[0030] In the example of FIG. 2, if two HMVP entries are needed for coding a current CU, m4 and m3 are selected because m4 and m3 are in the positions with indices 0 and 1.

[0031] In the example of FIG. 2, if three HMVP entries are needed for coding a current CU, m4, m3, and m2 are selected because m4, m3, and m2 are in the positions with indices 0, 1 and 2.

[0032] In the example of FIG. 2, if four HMVP entries are needed for coding a currentCU, m4, m3, m2 and ml are selected because m4, m3, m2 and ml are in the positions with indices 0, 1, 2 and 3.

[0033] In the example of FIG. 2, if five HMVP entries are needed, all the five entries of m4, m3, m2, ml and mO are selected.

[0034] From FIG. 1 and FIG. 2, it can be observed that the CU (n4) is far away from the current CU (X) as compared to CU (nO) and CU (nl), but its motion information is the first one to be selected in the current design. In natural video, two close blocks, however, are likely statistically more correlated with each other than two far distance blocks.

[0035] When one or more HMVP entries are needed for coding a current CU, the one or more entries in the HMVP table with the associated CUs that have the closest distances to the current CU are selected.

[0036] In one embodiment, the distance between two CUs is defined as the Euclidean distance between the top-left corner pixels of the two CUs. For example, assume that (x10,y10)andarethe coordinators of the top-left pixels of two CUs. Their distance is d = (x10- x20)2+ (yio “ y2o)2-

[0037] In one embodiment, the distance between two CUs is defined as the Euclidean distance between the center pixels of the two CUs. For example, assume that (xlc,ylcand (x2c,y2c)arethe coordinators of the center pixels of two CUs. Their distance is d =

[0038] In one embodiment, the distance between two CUs is defined as the minimum Euclidean distance between any pair of four corner pixels of the two CUs. For example, assume that (x1£,y1£) and (x2£,y2£) ■> where i = 0,1, 2, 3, are the coordinators of the four comer pixels of two CUs. Their distance is d = min ( (x1£—0,1, 2, 3).

[0039] In one embodiment, if, at an instant, one entry (MI) in HMVP table is needed for coding a current CU, the entry in the HMVP table with its associated CU that has the closest distance to the current CU is selected.

[0040] In one embodiment, if, at an instant, two entries (MI) in HMVP table are needed for coding a current CU, the two entries in the HMVP table with their associated CUs that have the closest distances to the current CU are selected.

[0041] In one embodiment, if, at an instant, N entries (MI) in HMVP table are needed, the N entries in the HMVP table with their associated CUs that have the closest distances to the current CU are selected.

[0042] In the example of FIG. 2, if one HMVP entry is needed for a current CU, mO is selected because its associated CU (nO) has the distance closest to the current CU (X).

[0043] In the example of FIG. 2, if two HMVP entries are needed for a current CU, mO and ml are selected because their associated CU (nO) and CU (nl) have the closest distances to the current CU (X).

[0044] In the example of FIG. 2, if three HMVP entries are needed for a current CU, mO, ml and m4 are selected because their associated CU (nO), CU (nl) and CU (n4) have the closest distances to the current CU (X).

[0045] In the example of FIG. 2, if four HMVP entries are needed for a current CU, mO, ml, m4 and m2 are selected because their associated CU (nO), CU (nl), CU (n4) and CU (n2) have the closest distances to the current CU (X).

[0046] In the example of FIG. 2, if five HMVP entries are needed for a current CU, all the five entries of mO, ml, m4, m2 and m3 are selected.

[0047] One possible implementation of the examples described herein is that for a current CU in a current picture, entries (MI) in the HMVP table are sorted according to the distances of their associated inter coded CUs to the current CU.

[0048] If position indexing of the HMVP table remains unchanged, the entry with the associated CU that has the closest distance to the current CU is moved to the last position with index 0, the entry with the associated CU that has second closest distance to the current CU to the second last position with index 1, and so on.

[0049] For the example of FIG. 1, FIG. 3A shows entries in HMVP table 200 beforesorting and FIG. 3B shows the HMVP table 200 after sorting. As seen, the entries with the associated CUs that have the closest distances to the current CU are moved to the positions with the smallest indices after the sorting process.

[0050] mO is moved to the last position with index 0 because its associated CU (nO) has the closest distance to the current CU (X).

[0051] ml is moved to the second last position with index 1 because its associated CU (n2) has the second closest distance to the current CU (X).

[0052] m4 is moved to the third last position with index 2 because its associated CU (n4) has the third closest distance to the current CU (X).

[0053] m2 is moved to the fourth last position with index 0 because its associated CU (n2) has the fourth closest CU to the current CU (X).

[0054] m3 is moved to the first position with index 4 because its associated CU (n3) has the farthest distance to the current CU (X).

[0055] Note that the position with index 0 holds the newest entry before storing, and after sorting, it holds the entry with the associated CU that has the closest distance to the current CU.

[0056] With the HMVP table sorting, when one or more HMVP entries are needed for coding a current CU, the one or more entries in the HMVP table with the smallest position indices are selected because those entries are associated with the past inter coded CUs that have the closest distances to the current CU.

[0057] In the example of FIG. 3, if one HMVP entry is needed for coding a current CU, mO is selected because mO is in the position with index 0.

[0058] In the example of FIG. 3, if two HMVP entries are needed, mO and ml are selected because mO and ml are in the positions with indices 0 and 1.

[0059] In the example of FIG. 3, if three HMVP entries are needed, mO, ml, and m4 are selected because mO, ml, and m4 are in the positions with indices 0, 1, and 2.

[0060] In the example of FIG. 3, if four HMVP entries are needed, mO, ml, m4 and m2 are selected because mO, ml, m4 and m2 are in the positions with indices 0, 1, 2 and 3.

[0061] In the example of FIG. 3, if five HMVP entries are needed, all the five entries of mO, ml, m4, m2 and m3 are selected.

[0062] FIG. 4 shows a layout of an apparatus 50 according to an example embodiment. The electronic device 50 may for example be a mobile terminal or user equipment of a wireless communication system, a sensor device, a tag, or other lower power device. However, the embodiments of the examples described herein may be implemented within any electronic device or apparatus which may encode or decode multimedia content.

[0063] The apparatus 50 may comprise a housing 30 for incorporating and protecting the device. The apparatus 50 further may comprise a display 32 in the form of a liquid crystal display. In other embodiments of the examples described herein the display may be any suitable display technology suitable to display an image or video. The apparatus 50 may further comprise a keypad 34. In other embodiments of the examples described herein any suitable data or user interface mechanism may be employed. For example the user interface may be implemented as a virtual keyboard or data entry system as part of a touch-sensitive display.

[0064] The apparatus may comprise a microphone 36 or any suitable audio input which may be a digital or analog signal input. The apparatus 50 may further comprise an audio output device which in embodiments of the examples described herein may be any one of: an earpiece 38, speaker, or an analog audio or digital audio output connection. The apparatus 50 may also comprise a battery (or in other embodiments of the examples described herein the device may be powered by any suitable mobile energy device such as solar cell, fuel cell or clockwork generator). The apparatus may further comprise a camera capable of recording or capturing images and / or video. The apparatus 50 may further comprise an infrared port for short range line of sight communication to other devices. In other embodiments the apparatus 50 may further comprise any suitable short range communication solution such as for example a Bluetooth wireless connection or a USB / firewire wired connection.

[0065] As shown in FIG. 4, distance-based HMVP 60 may implement the examplesdescribed herein related to distance-based HMVP.

[0066] FIG. 5 is a block diagram illustrating a system 500 in accordance with several examples. In an example, the encoder 530 is used to encode an image or video from the scene 515, and the encoder 530 is implemented in a transmitting apparatus 580. The encoder 530 produces a bitstream 510 comprising signaling that is received by the receiving apparatus 582, which implements a decoder 540. The encoder 530 sends the bitstream 510 that comprises the herein described signaling. The decoder 540 forms the image or video for the scene 515-1, and the receiving apparatus 582 would present this to the user, e.g., via a smartphone, television, or projector among many other options.

[0067] In some examples, the transmitting apparatus 580 and the receiving apparatus 582 are at least partially within a common apparatus, and for example are located within a common housing 550. In other examples the transmitting apparatus 580 and the receiving apparatus 582 are at least partially not within a common apparatus and have at least partially different housings. Therefore in some examples, the encoder 530 and the decoder 540 are at least partially within a common apparatus, and for example are located within a common housing 550. For example the common apparatus comprising the encoder 530 and decoder 540 implements a codec. In other examples the encoder 530 and the decoder 540 are at least partially not within a common apparatus and have at least partially different housings, but when together still implement a codec.

[0068] In some examples, 3D media from the capture (e.g., volumetric capture) at a viewpoint 512 of the scene 515, which includes a person 513) is converted via projection to a series of 2D representations with occupancy, geometry, attributes and / or displacements. Additional atlas information is also included in the bitstream to enable inverse reconstruction. For decoding, the received bitstream 510 is separated into its components with atlas information; occupancy, geometry, displacement, and attribute 2D representations. A 3D reconstruction is performed to reconstruct the scene 515-1 created looking at the viewpoint 512-1 with a “reconstructed” person 513-1. The “-1” are used to indicate that these are reconstructions of the original. As indicated at 520, the decoder 540 performs an action or actions based on the received signaling.

[0069] Encoding 590 performs distance-based HMVP coding based on the examplesdescribed herein. Decoding 592 performs distance-based HMVP decoding, based on the examples described herein.

[0070] FIG. 6 is an example apparatus 600, which may be implemented in hardware, configured to implement the examples described herein. The apparatus 600 comprises at least one processor 602 (e.g., an FPGA and / or CPU), one or more memories 604 including computer program code 605, the computer program code 605 having instructions to carry out the methods described herein, wherein the at least one memory 604 and the computer program code 605 are configured to, with the at least one processor 602, cause the apparatus 600 to implement circuitry, a process, component, module, or function (implemented with control module 606) to implement the examples described herein.

[0071] Apparatus 600 may be a smartphone, personal digital device or assistant, smart television, laptop, tablet, head-mounted display (HMD) or other user device or terminal device. The memory 604 may be a non-transitory memory, a transitory memory, a volatile memory (e.g. RAM), or a non-volatile memory (e.g., ROM).

[0072] Distance-based HMVP 630 implements the examples described herein related to distance-based HMVP encoding and decoding.

[0073] The apparatus 600 includes a display and / or VO interface 608, which includes user interface (UI) circuitry and elements, that may be used to display features or a status of the methods described herein (e.g., as one of the methods is being performed or at a subsequent time), or to receive input from a user such as with using a keypad, camera, touchscreen, touch area, microphone, biometric recognition, one or more sensors, etc. The apparatus 600 includes one or more communication e.g. network (N / W) interfaces (I / F(s)) 610. The communication I / F(s) 610 may be wired and / or wireless and communicate over the Internet / other network(s) via any communication technique including via one or more links 624. The communication I / F(s) 610 may comprise one or more transmitters or one or more receivers.

[0074] The transceiver 616 comprises one or more transmitters 618 and one or more receivers 620. The transceiver 616 and / or communication I / F(s) 610 may comprise standard well-known components such as an amplifier, filter, frequency-converter, (de)modulator,and encoder / decoder circuitries and one or more antennas, such as antennas 614 used for communication over wireless link 626.

[0075] The control module 606 of the apparatus 600 comprises one of or both parts 606- 1 and / or 606-2, which may be implemented in a number of ways. The control module 606 may be implemented in hardware as control module 606-1, such as being implemented as part of the one or more processors 602. The control module 606-1 may be implemented also as an integrated circuit or through other hardware such as a programmable gate array. In another example, the control module 606 may be implemented as control module 606-2, which is implemented as computer program code (having corresponding instructions) 605 and is executed by the one or more processors 602. For instance, the one or more memories 604 store instructions that, when executed by the one or more processors 602, cause the apparatus 600 to perform one or more of the operations as described herein. Furthermore, the one or more processors 602, one or more memories 604, and example algorithms (e.g., as flowcharts and / or signaling diagrams), encoded as instructions, programs, or code, are means for causing performance of the operations described herein.

[0076] The apparatus 600 to implement the functionality of control 606 may correspond to any of the apparatuses depicted herein. Alternatively, apparatus 600 and its elements may not correspond to any of the other apparatuses depicted herein, as apparatus 600 may be part of a self-organizing / optimizing network (SON) node or other node, such as a node in a cloud.

[0077] The apparatus 600 may also be distributed throughout the network including within and between apparatus 600 and any network element (such as a base station and / or terminal device and / or user equipment).

[0078] Interface 612 enables data communication and signaling between the various items of apparatus 600, as shown in FIG. 6. For example, the interface 612 may be one or more buses such as address, data, or control buses, and may include any interconnection mechanism, such as a series of lines on a motherboard or integrated circuit, fiber optics or other optical communication equipment, and the like. Computer program code (e.g. instructions) 605, including control 606 may comprise object-oriented software configured to pass data or messages between objects within computer program code 605. Computerprogram code (e.g. instructions) 605, including control 606 may comprise procedural, functional, or scripting code. The apparatus 600 need not comprise each of the features mentioned, or may comprise other features as well. The various components of apparatus 600 may at least partially reside in a common housing 628, or a subset of the various components of apparatus 600 may at least partially be located in different housings, which different housings may include housing 628.

[0079] FIG. 7 shows a schematic representation of non-volatile memory media 700a (e.g. computer / compact disc (CD) or digital versatile disc (DVD)) and 700b (e.g. universal serial bus (USB) memory stick) and 700c (e.g. cloud storage for downloading instructions and / or parameters 702 or receiving emailed instructions and / or parameters 702) storing instructions and / or parameters 702 which when executed by a processor allows the processor to perform one or more of the operations of the methods described herein. Instructions and / or parameters 702 may represent or correspond to a non-transitory computer readable medium.

[0080] FIG. 8 shows an encoder 800 according to an embodiment. FIG. 8 illustrates an image to be encoded (In), a predicted representation of an image block (P'n), a prediction error signal (Dn), a reconstructed prediction error signal (D'n), a preliminary reconstructed image (I'n), a final reconstructed image (R'n), a transform (T) and inverse transform (T4), a quantization (Q) and inverse quantization (Q4), entropy encoding (E), a reference frame memory (RFM), inter prediction (pinter), intra prediction (Pmtra), mode selection (MS) and filtering (F). Distance-based HMVP 830 implements the examples described herein related to distance-based HMVP encoding.

[0081] FIG. 9 shows a decoder 900 according to an embodiment. FIG. 9 illustrates a predicted representation of an image block (P'n), a reconstructed prediction error signal (D'n), a preliminary reconstructed image (I'11), a final reconstructed image (R'n), an inverse transform (T ’), an inverse quantization (Q- 1), an entropy decoding (E1), a reference frame memory (RFM), a prediction (either inter or intra) (P), and filtering (F). Distance-based HMVP 930 implements the examples described herein related to distance-based HMVP decoding.

[0082] FIG. 10 is an example method 1000, based on the examples described herein. At 1010, the method includes determining that at least one distance from a current coding unitto at least one coding unit associated with at least one entry of a table is closer than other one or more distances from the current coding unit to other coding units associated with one or more entries of the table. At 1020, the method includes wherein the entries of the table correspond to motion information of respective inter coded coding units. At 1030, the method includes coding the current coding unit using the motion information corresponding to the at least one entry of the table. Method 1000 may be performed with apparatus 50, transmitting apparatus 580 with encoder 530, receiving apparatus 582 with decoder 540, apparatus 600, encoder 800 with distance-based HMVP 830, or decoder 900 with distancebased HMVP 930.

[0083] The following examples are provided and described herein.

[0084] Example 1. An apparatus including: at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to: determine that at least one distance from a current coding unit to at least one coding unit associated with at least one entry of a table is closer than other one or more distances from the current coding unit to other coding units associated with one or more entries of the table; wherein the entries of the table correspond to motion information of respective inter coded coding units; and code the current coding unit using the motion information corresponding to the at least one entry of the table.

[0085] Example 2. The apparatus of example 1, wherein the at least one distance from the current coding unit to the at least one coding unit associated with the at least one entry of the table comprises at least one closest distance among distances from the current coding unit to the coding units associated with the entries of the table.

[0086] Example 3. The apparatus of any of examples 1 to 2, wherein a distance between two coding units is defined as the Euclidean distance between top-left corner pixels of the two coding units.

[0087] Example 4. The apparatus of any of examples 1 to 3, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to: determine a distance between a first coding unit and a second coding unit to be a square root of: a square of a difference between a horizontal position of a top-left corner pixel of a first coding unitand a horizontal position of a top-left corner pixel of the second coding unit added to a square of a difference between a vertical position of the top-left corner pixel of the first coding unit and a vertical position of the top-left corner pixel of the second coding unit.

[0088] Example 5. The apparatus of any of examples 1 to 2, wherein a distance between two coding units is defined as the Euclidean distance between center pixels of the two coding units.

[0089] Example 6. The apparatus of any of examples 1 to 2 or 5, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to: determine a distance between a first coding unit and a second coding unit to be a square root of: a square of a difference between a horizontal position of a center pixel of a first coding unit and a horizontal position of a center pixel of the second coding unit added to a square of a difference between a vertical position of the center pixel of the first coding unit and a vertical position of the center pixel of the second coding unit.

[0090] Example 7. The apparatus of any of examples 1 to 2, wherein a distance between two coding units is defined as a minimum Euclidean distance between any pair of four corner pixels of the two coding units.

[0091] Example 8. The apparatus of any of examples 1 to 2 or 7, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to: determine a distance between a first coding unit and a second coding unit to be a minimum, over pairs of corner pixels of the first coding unit and the second coding unit, of a square root of: a square of a difference between a horizontal position of a corner pixel of a first coding unit and a horizontal position of a corner pixel of the second coding unit added to a square of a difference between a vertical position of the corner pixel of the first coding unit and a vertical position of the comer pixel of the second coding unit; wherein the corner pixels of the first coding unit and the second coding unit comprise: a top-left comer pixel of the first coding unit and the second coding unit, a top-right corner pixel of the first coding unit and the second coding unit, a bottom-left corner pixel of the first coding unit and the second coding unit, and a bottom-right corner pixel of the first coding unit and the second coding unit.

[0092] Example 9. The apparatus of any of examples 1 to 8, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to: determine that a number of the at least one entry of the table is to be used for coding the current coding unit; and select the number of the at least one entry of the table for coding the current coding unit.

[0093] Example 10. The apparatus of any of examples 1 to 9, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to: sort the entries of the table in order from closest distance from the current coding unit to one coding unit associated with an entry of the table to largest distance from the current coding unit to another coding unit associated with another entry of the table, such that the entries of the table correspond respectively to an index of indices of the table; wherein the indices are sorted from smallest to largest to correspond respectively to the entries of the table in order from closest distance from the current coding unit to the one coding unit associated with the entry of the table to largest distance from the current coding unit to the another coding unit associated with the another entry of the table.

[0094] Example 11. The apparatus of example 10, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to: select the at least one entry of the table that corresponds to the at least one smallest index of the table to code the current coding unit among the one or more indices of the table.

[0095] Example 12. The apparatus of any of examples 1 to 11, wherein the current coding unit and the coding units are rectangular.

[0096] Example 13. The apparatus of any of examples 1 to 12, wherein the current coding unit and the other coding units form part of a coding tree unit.

[0097] Example 14. The apparatus of any of examples 1 to 13, wherein the current coding unit forms part of a coding tree unit, and at least one coding unit of the other coding units forms part of another coding tree unit different from the coding tree unit comprising the current coding unit.

[0098] Example 15. The apparatus of any of examples 1 to 14, wherein the table comprises a history-based motion vector prediction table.

[0099] Example 16. The apparatus of any of examples 1 to 15, wherein the other one or more distances are other than the at least one distance, and the other one or more entries are other than the at least one entry.

[0100] Example 17. The apparatus of any of examples 1 to 16, wherein the motion information of the respective coding units comprise at least one or more of: a motion vector, or a reference picture, or a reference picture list.

[0101] Example 18. The apparatus of any of examples 1 to 17, wherein an encoder comprises the apparatus, or the apparatus comprises an encoder.

[0102] Example 19. The apparatus of any of examples 1 to 18, wherein a decoder comprises the apparatus, or the apparatus comprises a decoder.

[0103] Example 20. A method including: determining that at least one distance from a current coding unit to at least one coding unit associated with at least one entry of a table is closer than other one or more distances from the current coding unit to other coding units associated with one or more entries of the table; wherein the entries of the table correspond to motion information of respective inter coded coding units; and coding the current coding unit using the motion information corresponding to the at least one entry of the table.

[0104] Example 21. An apparatus including: means for determining that at least one distance from a current coding unit to at least one coding unit associated with at least one entry of a table is closer than other one or more distances from the current coding unit to other coding units associated with one or more entries of the table; wherein the entries of the table correspond to motion information of respective inter coded coding units; and means for coding the current coding unit using the motion information corresponding to the at least one entry of the table.

[0105] Example 22. A computer readable medium including instructions stored thereon for performing at least the following: determining that at least one distance from a current coding unit to at least one coding unit associated with at least one entry of a table is closer than other one or more distances from the current coding unit to other coding units associated with one or more entries of the table; wherein the entries of the table correspond to motion information of respective inter coded coding units; and coding the current codingunit using the motion information corresponding to the at least one entry of the table.

[0106] References to a ‘computer’, ‘processor’, etc. should be understood to encompass not only computers having different architectures such as single / multi-processor architectures and sequential / parallel architectures but also specialized circuits such as field- programmable gate arrays (FPGAs), application specific circuits (ASICs), signal processing devices and other processing circuitry. References to computer program, instructions, code etc. should be understood to encompass software for a programmable processor or firmware such as, for example, the programmable content of a hardware device such as instructions for a processor, or configuration settings for a fixed-function device, gate array or programmable logic device, etc.

[0107] The term “non-transitory,” as used herein, is a limitation of the medium itself (i.e., tangible, not a signal ) as opposed to a limitation on data storage persistency (e.g., RAM vs. ROM).

[0108] As used herein, the term ‘circuitry’, ‘circuit’ and variants may refer to any of the following: (a) hardware circuit implementations, such as implementations in analog and / or digital circuitry, and (b) combinations of circuits and software (and / or firmware), such as (as applicable): (i) a combination of processor(s) or (ii) portions of processor(s) / software including digital signal processor(s), software, and one or more memories that work together to cause an apparatus to perform various functions, and (c) circuits, such as a microprocessor s) or a portion of a microprocessor s), that require software or firmware for operation, even when the software or firmware is not physically present. As a further example, as used herein, the term ‘circuitry’ would also cover an implementation of merely a processor (or multiple processors) or a portion of a processor and its (or their) accompanying software and / or firmware. The term ‘circuitry’ would also cover, for example and when applicable to the particular element, a baseband integrated circuit or applications processor integrated circuit for a mobile phone or a similar integrated circuit in a server, a cellular network device, or another network device. Circuitry or circuit may also be used to mean a function or a process used to execute a method.

[0109] It should be understood that the foregoing description is only illustrative. Various alternatives and modifications may be devised by those skilled in the art. For example,features recited in the various dependent claims could be combined with each other in any suitable combination(s). In addition, features from different embodiments described above could be selectively combined into a new embodiment. Accordingly, the description is intended to embrace all such alternatives, modifications and variances which fall within the scope of the appended claims.

[0110] The following acronyms and abbreviations that may be found in the specification and / or the drawing figures are defined as follows (the abbreviations may be appended with each other or with other characters using e.g. a hyphen, dash (-), or number (or abbreviations having a character may be the same with a character removed), and may be case insensitive):2D two-dimensional3D three-dimensionalASIC application specific integrated circuitCPU central processing unitCTU coding tree unitCU coding unitECM enhanced compression modelFIFO first in first outFPGA field programmable gate arrayHMD head-mounted displayHMVP history-based motion vector predictionI / F interfaceI / O input / outputJVET joint video experts teamMI motion informationMVP motion vector predictionN / W networkRAM random access memoryRFM reference frame memoryROM read only memorySON self-organizing / optimizing networkUI user interfaceUSB universal serial busVVC versatile video coding

Claims

CLAIMSWhat is claimed is:

1. An apparatus comprising: at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to: determine that at least one distance from a current coding unit to at least one coding unit associated with at least one entry of a table is closer than other one or more distances from the current coding unit to other coding units associated with one or more entries of the table; wherein the entries of the table correspond to motion information of respective inter coded coding units; and code the current coding unit using the motion information corresponding to the at least one entry of the table.

2. The apparatus of claim 1, wherein the at least one distance from the current coding unit to the at least one coding unit associated with the at least one entry of the table comprises at least one closest distance among distances from the current coding unit to the coding units associated with the entries of the table.

3. The apparatus of any of claims 1 to 2, wherein a distance between two coding units is defined as the Euclidean distance between top-left corner pixels of the two coding units.

4. The apparatus of any of claims 1 to 3, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to:determine a distance between a first coding unit and a second coding unit to be a square root of: a square of a difference between a horizontal position of a top-left corner pixel of a first coding unit and a horizontal position of a top-left corner pixel of the second coding unit added to a square of a difference between a vertical position of the top-left comer pixel of the first coding unit and a vertical position of the top-left corner pixel of the second coding unit.

5. The apparatus of any of claims 1 to 2, wherein a distance between two coding units is defined as the Euclidean distance between center pixels of the two coding units.

6. The apparatus of any of claims 1 to 2 or 5, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to: determine a distance between a first coding unit and a second coding unit to be a square root of: a square of a difference between a horizontal position of a center pixel of a first coding unit and a horizontal position of a center pixel of the second coding unit added to a square of a difference between a vertical position of the center pixel of the first coding unit and a vertical position of the center pixel of the second coding unit.

7. The apparatus of any of claims 1 to 2, wherein a distance between two coding units is defined as a minimum Euclidean distance between any pair of four corner pixels of the two coding units.

8. The apparatus of any of claims 1 to 2 or 7, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to: determine a distance between a first coding unit and a second coding unit to be a minimum, over pairs of corner pixels of the first coding unit and the second coding unit, of a square root of: a square of a difference between a horizontal position of a corner pixel of a first coding unit and a horizontal position of a corner pixel of the second coding unit added to a square of a difference between a vertical position of the corner pixel of the first coding unit and a vertical position of the corner pixel of the second coding unit;wherein the corner pixels of the first coding unit and the second coding unit comprise: a top-left corner pixel of the first coding unit and the second coding unit, a top-right corner pixel of the first coding unit and the second coding unit, a bottom-left corner pixel of the first coding unit and the second coding unit, and a bottom-right corner pixel of the first coding unit and the second coding unit.

9. The apparatus of any of claims 1 to 8, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to: determine that a number of the at least one entry of the table is to be used for coding the current coding unit; and select the number of the at least one entry of the table for coding the current coding unit.

10. The apparatus of any of claims 1 to 9, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to: sort the entries of the table in order from closest distance from the current coding unit to one coding unit associated with an entry of the table to largest distance from the current coding unit to another coding unit associated with another entry of the table, such that the entries of the table correspond respectively to an index of indices of the table; wherein the indices are sorted from smallest to largest to correspond respectively to the entries of the table in order from closest distance from the current coding unit to the one coding unit associated with the entry of the table to largest distance from the current coding unit to the another coding unit associated with the another entry of the table.

11. The apparatus of claim 10, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to: select the at least one entry of the table that corresponds to the at least one smallest index of the table to code the current coding unit among the one or more indicesof the table.

12. The apparatus of any of claims 1 to 11, wherein the current coding unit and the coding units are rectangular.

13. The apparatus of any of claims 1 to 12, wherein the current coding unit and the other coding units form part of a coding tree unit.

14. The apparatus of any of claims 1 to 13, wherein the current coding unit forms part of a coding tree unit, and at least one coding unit of the other coding units forms part of another coding tree unit different from the coding tree unit comprising the current coding unit.

15. The apparatus of any of claims 1 to 14, wherein the table comprises a history-based motion vector prediction table.

16. The apparatus of any of claims 1 to 15, wherein the other one or more distances are different from the at least one distance, and the other one or more entries are different from the at least one entry.

17. The apparatus of any of claims 1 to 16, wherein the motion information of the respective coding units comprise at least one or more of: a motion vector, or a reference picture, or a reference picture list.

18. The apparatus of any of claims 1 to 17, wherein an encoder comprises the apparatus, or the apparatus comprises an encoder.

19. The apparatus of any of claims 1 to 18, wherein a decoder comprises the apparatus, or the apparatus comprises a decoder.

20. A method comprising: determining that at least one distance from a current coding unit to at least one coding unit associated with at least one entry of a table is closer than other one or more distances from the current coding unit to other coding units associated with one or moreentries of the table; wherein the entries of the table correspond to motion information of respective inter coded coding units; and coding the current coding unit using the motion information corresponding to the at least one entry of the table.

21. An apparatus comprising: means for determining that at least one distance from a current coding unit to at least one coding unit associated with at least one entry of a table is closer than other one or more distances from the current coding unit to other coding units associated with one or more entries of the table; wherein the entries of the table correspond to motion information of respective inter coded coding units; and means for coding the current coding unit using the motion information corresponding to the at least one entry of the table.

22. A computer readable medium comprising instructions stored thereon for performing at least the following: determining that at least one distance from a current coding unit to at least one coding unit associated with at least one entry of a table is closer than other one or more distances from the current coding unit to other coding units associated with one or more entries of the table; wherein the entries of the table correspond to motion information of respective inter coded coding units; and coding the current coding unit using the motion information corresponding to the at least one entry of the table.

Images