Simplification of hash-based motion search

Abstract

Methods, systems, and devices for hash-based motion estimation in video coding are described. An exemplary method of video processing includes: determining motion information associated with a current block of video using a hash-based motion search for conversion between the current block and a bitstream representation of the video, the current block being of size M×N, M and N being positive integers, and M not equal to N; performing a prediction on the current block based on the motion information and a video picture including the current block; and performing conversion based on the prediction.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application timely claims the priority and benefit of International Patent Application No. PCT / CN2019 / 070049 filed on January 2, 2019 and International Patent Application No. PCT / CN2019 / 087969 filed on May 22, 2019, in accordance with applicable patent laws and / or the rules applicable to the Paris Convention. For all legal purposes, the entire disclosure of the above application is incorporated by reference as part of the disclosure of this application. Technical Field

[0003] This document deals with video and image coding techniques. Background Art

[0004] Despite advances in video compression, digital video still accounts for the largest use of bandwidth on the Internet and other digital communications networks. As the number of connected user devices capable of receiving and displaying video increases, the bandwidth required for digital video usage is expected to continue to grow. Summary of the invention

[0005] Devices, systems and methods related to digital video coding are described, including hash-based motion estimation. The described methods can be applied to existing video coding standards (e.g., High Efficiency Video Coding (HEVC) and / or Versatile Video Coding (VVC)) and future video coding standards or video codecs.

[0006] In one representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes: for conversion between a current block of video and a bitstream representation of the video, using a hash-based motion search to determine motion information associated with the current block, wherein the size of the current block is M×N, wherein M and N are positive integers and M is not equal to N; applying prediction to the current block based on the motion information and a video picture including the current block; and performing conversion based on the prediction.

[0007] In another representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes: for conversion between a current block of video and a bitstream representation of the video, determining motion information associated with the current block by using a hash-based motion search on a non-rectangular and non-square region of the current block; applying prediction to the current block based on the motion information and a video picture including the current block; and performing the conversion based on the prediction.

[0008] In yet another representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes: for conversion between a current block of video and a bitstream representation of the video, determining motion information associated with the current block by using a hash-based motion search on a fixed subset of samples of the current block; applying prediction to the current block based on the motion information and a video picture including the current block; and performing the conversion based on the prediction.

[0009] In yet another representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes: performing a hash-based motion search as part of a conversion between a current block of video and a bitstream representation of the video; determining a rate-distortion cost of a skip mode for each of one or more coding modes of the current block for the conversion based on the hash-based motion search finding a hash match and a quantization parameter (QP) of a reference frame not greater than the QP of the current block; applying prediction to the current block based on the rate-distortion cost and a video picture including the current block; and performing the conversion based on the prediction.

[0010] In yet another representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes: for conversion between a current block of video and a bitstream representation of the video, determining motion information associated with the current block using a hash-based motion search, the hash-based motion search being based on hash values ​​of square sub-blocks of the current block, wherein the current block has a size of M×N, and wherein M and N are positive integers; applying prediction to the current block based on the motion information and a video picture including the current block; and performing conversion based on the prediction.

[0011] In yet another representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes: for conversion between a current block of a video and a bitstream representation of the video, using a hash-based motion search to determine motion information associated with the current block, the hash-based motion search including performing a K-pixel integer motion vector (MV) accuracy check for the hash-based motion search, where K is a positive integer; applying a prediction to the current block based on the motion information and a video picture including the current block; and performing the conversion based on the prediction.

[0012] In yet another representative aspect, the above method is embodied in the form of processor executable code and stored in a computer readable program medium.

[0013] In yet another representative aspect, a device configured or operable to perform the above method is disclosed. The device may include a processor programmed to implement the method.

[0014] In yet another representative aspect, a video decoder device may implement a method as described herein.

[0015] The above and other aspects and features of the disclosed technology are described in more detail in the drawings, the description and the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 An example of bottom-up hash calculation is shown.

[0017] Figure 2A and Figure 2B An example of a 2N×2N block row / column identical information check based on the results of six N×N blocks is shown.

[0018] Figure 3 An example of a current picture reference is shown.

[0019] Figure 4-9 is a flow chart of an example of a video processing method.

[0020] Fig.10 is a block diagram of an example of a hardware platform for implementing the visual media decoding or visual media encoding techniques described herein.

[0021] Fig.11 is a block diagram of an example video processing system in which the disclosed techniques may be implemented. DETAILED DESCRIPTION

[0022] This document provides various techniques that can be used by a decoder of an image or video bitstream to improve the quality of a decompressed or decoded digital video or image. For brevity, the term "video" is used herein to include both a sequence of pictures (traditionally referred to as video) and a single image. In addition, a video encoder may also implement these techniques during the encoding process in order to reconstruct a decoded frame for further encoding.

[0023] The section headings used in this document are for ease of understanding and do not limit the embodiments and techniques to the corresponding sections. Thus, the embodiments of one section can be combined with the embodiments of other sections.

[0024] 1. Overview

[0025] The invention relates to video coding techniques. In particular, it relates to motion estimation in video coding. It can be applied to existing video coding standards (such as HEVC), or to a finalized standard (Universal Video Coding). It may also be applicable to future video coding standards or video codecs.

[0026] 2. Background

[0027] Video coding standards have evolved primarily through the development of the well-known ITU-T and ISO / IEC standards. ITU-T produced H.261 and H.263, ISO / IEC produced MPEG-1 and MPEG-4 Visual, and the two organizations worked together to produce the H.262 / MPEG-2 Video and H.264 / PEG-4 Enhanced Video Coding (AVC) and H.265 / HEVC[1] standards. Starting with H.262, video coding standards are based on a hybrid video coding structure that utilizes temporal prediction plus transform coding. In order to explore future video coding technologies beyond HEVC, VCEG and MPEG jointly established the Joint Video Exploration Team (JVET) in 2015. Since then, JVET has adopted many new methods and introduced them into a reference software called the "Joint Exploration Model" (JEM)[2][3]. In April 2018, the Joint Video Experts Group (JVET) between VCEG (Q6 / 16) and ISO / IEC JTC1 SC29 / WG11 (MPEG) was established to work on the VVC standard with the goal of reducing bit rate by 50% compared to HEVC.

[0028] The latest version of the VVC draft, General Video Coding (Draft 2), can be found at:

[0029] http: / / phenix.it-sudparis.eu / jvet / doc_end_user / documents / 11_ Ljubljana / wg11 / JVET-K1001-v7.zip

[0030] The latest reference software for VVC, called VTM, can be found at:

[0031] https: / / vcgit.hhi.fraunhofer.de / jvet / VVCSoftware_VTM / taffs / VTM-2.1

[0032] Figure 5 is a block diagram of an example implementation of a video encoder. Figure 5 The encoder implementation is shown to have a built-in feedback path within which the video encoder also performs a video decoding function (reconstructing a compressed representation of the video data for use in the encoding of the next video data).

[0033] 2.1 Example of hash-based motion search

[0034] A hash-based search is applied to 2Nx2N blocks. An 18-bit hash based on the original pixels is used. The first 2 bits are determined by the block size, e.g. 00 for 8x8, 01 for 16x16, 10 for 32x32, 11 for 64x64. The next 16 bits are determined by the original pixels.

[0035] For a block, we use a similar approach but different CRC truncation polynomials to calculate two hash values. The first hash value is used for retrieval, and the second hash value is used to exclude certain hash collisions.

[0036] The hash value is calculated as follows:

[0037] For each row, a 16-bit CRC value is calculated for all pixels Hash[i].

[0038] Group the row hash values ​​together (Hash[0] Hash[1] ...), and then calculate the 24-bit CRC value H.

[0039] The lower 16 bits of H will be used as the lower 16 bits of the hash value of the current block.

[0040] To avoid having too many entries for one hash value, we avoid adding blocks to the hash table that satisfy the following conditions:

[0041] There is only one pixel value per row; or

[0042] Each column has only one pixel value.

[0043] Because for such blocks, blocks with a shift of one pixel can have the same content. Therefore, we exclude these blocks from the hash table. And such blocks can be perfectly predicted by horizontal prediction or vertical prediction, so excluding them from the hash table may not cause a noticeable performance degradation.

[0044] After building the hash table, the motion search is performed as follows.

[0045] For a 2Nx2N block,

[0046] A hash-based search is performed first.

[0047] If a hash match is found, the regular integer pixel motion search and fractional pixel motion search are skipped. Otherwise, the regular motion search is performed.

[0048] Save the integer MV found for the 2Nx2N block.

[0049] For non-2Nx2N blocks, the cost of using MVP as the ME starting point is compared with the cost of using the stored MV of the 2Nx2N block of the same reference picture as the ME starting point, and then the one with the lower cost is selected as the motion search starting point.

[0050] We also develop an early termination algorithm based on hash search, which terminates the RDO process if all the following conditions are met without checking other modes and CU partitions.

[0051] A hash match is found.

[0052] The quality of the reference block is not worse than the expected quality of the current block (the QP of the reference block is not greater than the QP of the current block).

[0053] The current CU depth is 0.

[0054] 2.2 Additional Examples of Hash-Based Motion Search

[0055] 1. Bottom-up hash value calculation

[0056] To speed up the computation of hash values, we propose to compute the block hash values ​​in a bottom-up manner. The same method is used as in [1], but different CRC truncation polynomials are used to compute the two hash values. And the CRC truncation polynomial used is the same as in the current SCM.

[0057] For a 2Nx2N (N=2, 4, 8, 16 or 32) block, the hash value is calculated based on the hash results of its four sub-NxN blocks instead of pixels, so that the intermediate data (the hash value results of the smaller blocks) can be used as much as possible to reduce the complexity. For N=1, that is, 2×2 blocks, the block hash value is calculated directly from the pixels. The whole process is as follows Figure 1 shown.

[0058] 2. Bottom-up validity check of the process of adding blocks to the hash table,

[0059] In the proposed scheme, the conditions for judging whether a block is valid to add to the hash table have not changed. The only change is the step of checking whether a block is valid. Similar to the block hash value calculation, the validity check can also be performed in a bottom-up manner to reuse the intermediate results and reduce the complexity.

[0060] For all 2x2 blocks in the picture, row / column identical checks are performed on a pixel basis. The row / column identical decision results are stored for validity checks of 4x4 blocks. For a 2Nx2N (N=2, 4, 8, 16, or 32) block, validity can be simply determined by the row / column identical information of the 6 sub-NxN blocks in each direction, respectively.

[0061] Take the same check as an example. Figure 2A In the example, the four sub-NxN blocks are marked as . In addition, in order to check whether the boundary pixels between blocks 0 and 1 and blocks 2 and 3 are the same, the row identical information should be used for the middle NxN blocks 4 and 5 for the upper half block and the lower half block, respectively. In other words, the 2N×2N block row identical information can be determined by the row identical information of the 6 sub-N×N blocks that have been recorded in the previous step. In this way, repeated pixel checking for different sizes of blocks can be avoided.

[0062] 2.3 Current Image Reference

[0063] In HEVC Screen Content Coding Extension (HEVC-SCC) [1] and the current VVC Test Model (VTM-3.0) [2], Current Picture Reference (CPR) (or once called Intra Block Copy (IBC)) has been adopted. IBC extends the concept of motion compensation from inter-frame coding to intra-frame coding. Figure 3 As shown, when CPR is applied, the current block is predicted by a reference block in the same picture. Before the current block is encoded or decoded, the samples in the reference block must have been reconstructed. Although CPR is not efficient for most camera-captured sequences, it shows significant coding gains for screen content. The reason is that there are many repeated patterns in screen content pictures, such as icons and text characters. CPR can effectively eliminate the redundancy between these repeated patterns. In HEVC-SCC, if the inter-coded codec unit (CU) selects the current picture as its reference picture, the inter-coded codec unit (CU) can apply CPR. In this case, MV is renamed as block vector (BV), and BV always has integer pixel precision. In order to be compatible with the main profile HEVC, the current picture is marked as a "long-term" reference picture in the decoded picture buffer (DPB). It should be noted that similarly, in the multi-view / 3D video coding standard, the inter-view reference picture is also marked as a "long-term" reference picture.

[0064] After the BV finds its reference block, a prediction can be generated by copying the reference block. The residual can be obtained by subtracting the reference pixels from the original signal. Transforms and quantization can then be applied as in other coding modes.

[0065] However, when the reference block is outside the picture, or overlaps with the current block, or is outside the reconstructed area, or is outside the valid area subject to some constraints, some or all pixel values ​​are undefined. Basically, there are two ways to solve this problem. One is to prohibit this situation, for example in terms of bitstream consistency. The other is to apply padding to those undefined pixel values. The following subsections describe the solutions in detail.

[0066] 2.4 Hash-based search for intra block copy / current picture reference

[0067] Hash-based search is applied to 4x4, 8x8, and 16x16 blocks. For a block, we use a similar approach but with different CRC truncation polynomials to calculate two hash values. The first hash value is used for retrieval and the second hash value is used to exclude certain hash collisions. The hash values ​​are calculated as follows:

[0068] For each row, a 16-bit CRC value is calculated for all pixels Hash[i].

[0069] Group the row hash values ​​together (Hash[0] Hash[1] ...), and then calculate the 24-bit CRC value H.

[0070] The lower 16 bits of H will be used as the lower 16 bits of the hash value of the current block.

[0071] To avoid one hash value corresponding to too many entries, we avoid adding blocks that satisfy the following conditions to the hash table.

[0072] There is only one pixel value per row, or

[0073] Each column has only one pixel value.

[0074] Because for such blocks, blocks with a shift of one pixel can have the same content. Therefore, we exclude these blocks from the hash table. And such blocks can be perfectly predicted by horizontal prediction or vertical prediction.

[0075] 3. Disadvantages of existing implementations

[0076] Current hash-based searches are limited to NxN (square) blocks, which limits their performance.

[0077] 4. Examples of Embodiments

[0078] The following detailed techniques should be considered as examples to explain the general concept. These techniques should not be interpreted narrowly. In addition, these techniques can be combined in any way.

[0079] 1. It is proposed that hash-based motion search can be based on blocks of size MxN, where M is not equal to N.

[0080] a. In one example, M=2N, 4N, 8N, 16N or 32N.

[0081] b. In one example, N=2M, 4M, 8M, 16M or 32M.

[0082] c. The hash value of a non-square block can be derived from the hash value of a square block.

[0083] d. In one example, if M is greater than N, the hash value of the MxN block can be derived from the hash values ​​of M / N(N,N) blocks.

[0084] e. Alternatively, if M is less than N, the hash value of the MxN block can be derived from the hash values ​​of N / M(M,M) blocks.

[0085] 2. It is proposed that hash-based motion search can be based on triangular areas.

[0086] a. In one example, the region is defined as x<ky; x, y=0, 1, 2, ...M-1, where x, y are coordinates relative to the start of the block and k is a predefined value.

[0087] b. Alternatively, define the region as x>ky, x, y=0, 1, 2, ...M-1

[0088] c. Alternatively, define the region as x<M-ky, x, y=0, 1, 2, ...M-1

[0089] d. Alternatively, define the region as x>M-ky, x, y=0, 1, 2, ...M-1

[0090] 3. It is proposed that hash-based motion search can be based on a fixed subset of blocks.

[0091] a. In one example, a subset is defined as a set of predefined coordinates relative to the start point of the block.

[0092] 4. Hash-based motion vectors can be added as additional starting points in the MV estimation process.

[0093] a. In integer motion estimation (eg xTZSearch or xTZSearchSelective), hash-based motion vectors will be checked for best starting point initialization.

[0094] b. If hash-based motion vectors exist, fractional motion estimation can be skipped.

[0095] 5. Early Termination

[0096] a. When a hash match is found and the quantization parameter of the reference frame is not greater than the quantization parameter of the current block, the rate-distortion cost of the skip mode is checked only for ETM_MERGE_SKIP, ETM_AFFINE, ETM_MERGE_TRIANGLE modes.

[0097] i. In addition, alternatively, skip checking the rate-distortion cost of other modes (such as AMVP (advanced motion vector prediction)).

[0098] ii. Additionally, alternatively, a finer-grained block partition check is terminated.

[0099] iii. Furthermore, alternatively, if the best mode is ETM_HASH_INTER after checking those skip modes, the finer-grained block partition check may be terminated. Otherwise, the finer-grained block partition check may not be terminated.

[0100] b. In one example, when a hash match is found and the quantization parameter of the reference frame is not greater than the quantization parameter of the current block, the rate-distortion cost of the skip mode is checked only for ETM_MERGE_SKIP, ETM_AFFINE, ETM_MERGE_TRIANGLE modes and ETM_INTER_ME (e.g., AMVR (adaptive motion vector resolution)) modes.

[0101] i. Additionally, alternatively, checking the rate-distortion penalty for other modes (eg, ETM_INTRA) is skipped.

[0102] ii. Additionally, alternatively, a finer-grained block partition check is terminated.

[0103] c. Whether to adopt the above method may depend on the size of the block.

[0104] i. In one example, it can be called only when the block size is greater than or equal to M x N.

[0105] ii. In one example, it can be called only when the block size is equal to 64x64.

[0106] 6. It is proposed to check the hash pattern of a rectangular block by its sub-square hash values.

[0107] a. In one example, this method can also be applied to square blocks.

[0108] 7. Proposed to add K-pixel integer MV accuracy check for hash MV

[0109] a. In one example, K is set to 4.

[0110] 8. The above method can be applied to a portion of the allowed block sizes.

[0111] a. In one example, for square blocks, a hash-based approach may be applied to N×N blocks where N is less than or equal to a threshold (eg, 64).

[0112] b. In one example, for non-square (rectangular) blocks, a hash-based method may be applied to M×N blocks where M*N is less than or equal to a threshold value (eg, 64).

[0113] c. In one example, for non-square (rectangular) blocks, the hashing based approach can be applied to 4x8, 8x4 only.

[0114] d. The above method may be applied only to certain color components, such as the luminance color component / base color component (eg, G).

[0115] Figure 4 4 is a flow chart of a method 400 for video processing. The method 400 includes, at operation 410, determining motion information associated with a current block of video using a hash-based motion search for conversion between a current block of video and a bitstream representation of the video, wherein the current block has a size of M×N, wherein M and N are positive integers and M is not equal to N.

[0116] The method 400 includes, at operation 420 , applying prediction to the current block based on the motion information and the video picture including the current block.

[0117] The method 400 includes, at operation 430 , performing a conversion based on the prediction.

[0118] Figure 5 is a flow chart of a method 500 for video processing. The method 500 includes, at operation 510, determining motion information associated with a current block of video by using a hash-based motion search on a non-rectangular and non-square region of the current block for conversion between the current block of video and a bitstream representation of the video.

[0119] The method 500 includes, at operation 520 , applying prediction to the current block based on the motion information and the video picture including the current block.

[0120] The method 500 includes, at operation 530 , performing a conversion based on the prediction.

[0121] Figure 6 is a flow chart of a method 600 for video processing. The method 600 includes, at operation 610, for conversion between a current block of video and a bitstream representation of the video, determining motion information associated with the current block by using a hash-based motion search on a fixed subset of samples of the current block.

[0122] The method 600 includes, at operation 620 , applying prediction to the current block based on the motion information and the video picture including the current block.

[0123] The method 600 includes, at operation 630 , performing a conversion based on the prediction.

[0124] Figure 7 is a flow chart of a method 700 for video processing. The method 700 includes, at operation 710, performing a hash-based motion search as part of a conversion between a current block of video and a bitstream representation of the video.

[0125] The method 700 includes, at operation 720, determining a rate-distortion cost of skip mode for each of one or more coding modes of the current block for conversion based on the hash-based motion search finding a hash match and the quantization parameter (QP) of the reference frame being no greater than the QP of the current block.

[0126] The method 700 includes, at operation 730 , applying prediction to the current block based on the rate-distortion cost and a video picture including the current block.

[0127] The method 700 includes, at operation 740 , performing a conversion based on the prediction.

[0128] Figure 8 8 is a flow chart of a method 800 for video processing. The method 800 includes, at operation 810, determining motion information associated with a current block of video using a hash-based motion search for conversion between a current block of video and a bitstream representation of the video, the hash-based motion search being based on hash values ​​of square sub-blocks of the current block, wherein the current block has a size of M×N, and wherein M and N are positive integers.

[0129] The method 800 includes, at operation 820 , applying prediction to the current block based on the motion information and the video picture including the current block.

[0130] The method 800 includes, at operation 830 , performing a conversion based on the prediction.

[0131] Fig. 9 is a flow chart of a method 900 for video processing. The method 900 includes, at operation 910, determining motion information associated with a current block of video using a hash-based motion search for conversion between a current block of video and a bitstream representation of the video, the hash-based motion search including performing a K-pixel integer motion vector (MV) accuracy check for the hash-based motion search, where K is a positive integer.

[0132] The method 900 includes, at operation 920 , applying prediction to the current block based on the motion information and the video picture including the current block.

[0133] The method 900 includes, at operation 930 , performing a conversion based on the prediction.

[0134] In some embodiments, the following technical solutions may be implemented:

[0135] A1. A video processing method, comprising: using a hash-based motion search to determine motion information associated with a current block of a video for conversion between a current block of the video and a bitstream representation of the video, wherein the size of the current block is M×N, wherein M and N are positive integers and M is not equal to N; applying prediction to the current block based on the motion information and a video picture including the current block; and performing conversion based on the prediction.

[0136] A2. A method according to solution A1, wherein M=K×N, and wherein K is a positive integer.

[0137] A3. A method according to solution A1, wherein N=K×M, and wherein K is a positive integer.

[0138] A4. A method according to solution A2 or A3, wherein K=2, 4, 8, 16 or 32.

[0139] A5. Method according to solution A1, wherein the hash value of the current block is derived based on the hash values ​​of square blocks of the video.

[0140] A6. The method according to solution A5, wherein, since it is determined that M is greater than N, the hash value of the current block is derived based on the hash values ​​of M / N blocks of size N×N forming the current block.

[0141] A7. The method according to solution A5, wherein, since it is determined that N is greater than M, the hash value of the current block is derived based on the hash values ​​of N / M blocks of size M×M forming the current block.

[0142] A8. A method of video processing, comprising: determining motion information associated with a current block of a video by using a hash-based motion search on a non-rectangular and non-square region of the current block for conversion between a current block of the video and a bitstream representation of the video; applying prediction to the current block based on the motion information and a video picture including the current block; and performing conversion based on the prediction.

[0143] A9. A method according to solution A8, wherein the area includes a triangular area, the triangular area includes sample points with coordinates (x, y), where x and y are relative to the starting point of the current block, where the size of the current block is M×N, where M and N are positive integers.

[0144] A10. A method according to solution A9, wherein x<k×y, k is a predefined value, and y=0, 1, . . . , M-1.

[0145] A11. A method according to solution A9, wherein x>k×y, k is a predefined value, and y=0, 1, ..., M-1.

[0146] A12. A method according to solution A9, wherein x<(Mk×y), k is a predefined value, and y=0, 1, . . . , M-1.

[0147] A13. A method according to solution A9, wherein x>(Mk×y), k is a predefined value, and y=0, 1, . . . , M-1.

[0148] A14. A video processing method, comprising: for a conversion between a current block of a video and a bitstream representation of the video, determining motion information associated with the current block by using a hash-based motion search on a fixed subset of samples of the current block; applying a prediction to the current block based on the motion information and a video picture including the current block; and performing the conversion based on the prediction.

[0149] A15. Method according to solution A14, wherein the fixed subset of samples comprises a set of predefined coordinates relative to the start point of the current block.

[0150] A16. The method of any of solutions A1 to A15, wherein the hash-based motion search comprises adding hash-based motion vectors (MVs) as a starting point during the MV estimation technique.

[0151] A17. The method according to solution A16, wherein a hash-based MV is checked for optimal starting point initialization in integer motion estimation.

[0152] A18. The method according to solution A16 or A17, wherein the fractional motion estimation operation is skipped due to determining that a hash-based MV exists.

[0153] A19. A method according to any of solutions A1 to A18, wherein the conversion generates the current block from a bitstream representation.

[0154] A20. A method according to any of solutions A1 to A18, wherein the conversion generates a bitstream representation from the current block.

[0155] A21. An apparatus in a video system, comprising a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to implement the method of any one of Solutions A1 to A20.

[0156] A22. A computer program product stored on a non-transitory computer readable medium, the computer program product comprising a program code for executing the method of any one of Solutions A1 to A20.

[0157] In some embodiments, the following technical solutions may be implemented:

[0158] B1. A video processing method, comprising: performing a hash-based motion search as part of a conversion between a current block of a video and a bitstream representation of the video; determining a rate-distortion cost of a skip mode for each of one or more coding modes of the current block for the conversion based on finding a hash match and a quantization parameter (QP) of a reference frame not greater than the QP of the current block according to the hash-based motion search; applying prediction to the current block based on the rate-distortion cost and a video picture including the current block; and performing the conversion based on the prediction.

[0159] B2. The method according to solution B1, wherein the one or more encoding modes include ETM_MERGE_SKIP mode, ETM_AFFINE mode, ETM_MERGE_TRIANGLE mode or ETM_INTER_ME mode.

[0160] B3. A method according to solution B1 or B2, wherein the one or more coding modes do not include Advanced Motion Vector Prediction (AMVP) mode or ETM_INTRA mode.

[0161] B4. The method according to solution B1 further includes: after the determination, terminating the finer-grained block partition check.

[0162] B5. The method according to solution B4, wherein the rate-distortion cost of the skip mode for the ETM_HASH_INTER mode is determined to be lower than the rate-distortion cost of the skip mode for any other coding mode.

[0163] B6. The method according to any of solutions B1 to B5, wherein the hash-based motion search and determination is performed based on the height or width of the current block.

[0164] B7. A method according to solution B6, wherein the size of the current block is M×N or larger.

[0165] B8. A method according to solution B6, wherein the size of the current block is 64×64.

[0166] B9. A method according to any of solutions B1 to B8, wherein the conversion generates the current block from a bitstream representation.

[0167] B10. A method according to any of solutions B1 to B8, wherein the conversion generates a bitstream representation from the current block.

[0168] B11. An apparatus in a video system, comprising a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to implement the method of any one of solutions B1 to B10.

[0169] B12. A computer program product stored on a non-transitory computer-readable medium, the computer program product comprising a program code for executing the method of any one of solutions B1 to B10.

[0170] In some embodiments, the following technical solutions may be implemented:

[0171] C1. A video processing method, comprising: for conversion between a current block of a video and a bitstream representation of the video, using a hash-based motion search to determine motion information associated with the current block, the hash-based motion search being based on hash values ​​of square sub-blocks of the current block, wherein the current block has a size of M×N, and wherein M and N are positive integers; applying prediction to the current block based on the motion information and a video picture including the current block; and performing conversion based on the prediction.

[0172] C2. The method according to solution C1, wherein M is not equal to N.

[0173] C3. The method according to solution C1, wherein M equals N.

[0174] C4. A video processing method, comprising: for conversion between a current block of a video and a bitstream representation of the video, using a hash-based motion search to determine motion information associated with the current block, the hash-based motion search comprising performing a K-pixel integer motion vector (MV) accuracy check for the hash-based motion search, where K is a positive integer; applying a prediction to the current block based on the motion information and a video picture including the current block; and performing conversion based on the prediction.

[0175] C5. A method according to solution C4, where K=4.

[0176] C6. A method according to any one of solutions C1 to C5, wherein the size of the current block is N×N, wherein N is a positive integer less than or equal to a threshold.

[0177] C7. A method according to any one of solutions C1 to C5, wherein the size of the current block is M×N, wherein M and N are positive integers and M is not equal to N, and wherein M or N is less than or equal to a threshold.

[0178] C8. A method according to solution C6 or C7, wherein the threshold is 64.

[0179] C9. A method according to any one of solutions C1 to C5, wherein the size of the current block is 8×4 or 4×8.

[0180] C10. The method according to any of solutions C1 to C5, wherein the hash-based motion search is applied to the luma component of the video.

[0181] C11. The method according to any of the solutions C1 to C5, wherein the hash-based motion search is applied to the primary color components of the video.

[0182] C12. The method according to any of the solutions C1 to C11, wherein the conversion generates the current block from a bitstream representation.

[0183] C13. A method according to any of solutions C1 to C11, wherein the conversion generates a bitstream representation from the current block.

[0184] C14. An apparatus in a video system, comprising a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to implement the method of any one of Solutions C1 to C13.

[0185] C15. A computer program product stored on a non-transitory computer-readable medium, the computer program product comprising a program code for executing the method of any one of solutions C1 to C13.

[0186] In some embodiments, the following technical solutions may be implemented:

[0187] D1. A method for processing a video, comprising: determining, by a processor, motion information about a first video block using a hash-based motion search, the first video block having a size of M×N, where M is not equal to N; and performing further processing on the first video block using the motion information.

[0188] D2. The method according to solution D1, wherein M is 2N, 4N, 8N, 16N or 32N.

[0189] D3. The method according to solution D1, wherein N is 2M, 4M, 8M, 16M or 32M.

[0190] D4. The method of solution D1, wherein determining the motion information using a hash-based motion search comprises deriving a hash value for the non-square video block using a hash value for a square video block that is a sub-block of the non-square video block.

[0191] D5. The method of solution D1, further comprising: determining that M is greater than N, and wherein determining the motion information using the hash-based motion search comprises deriving a hash value for the first video block from the M / N(N,N) video block hash values.

[0192] D6. The method of solution D1, further comprising: determining that M is less than N, and wherein determining the motion information using the hash-based motion search comprises deriving a hash value for the first video block from the M / N(N,N) video block hash values.

[0193] D7. A method for processing a video, comprising: determining, by a processor, motion information about a first video block using a hash-based motion search, the first video block having a non-rectangular and non-square shape; and performing further processing on the first video block using the motion information.

[0194] D8. The method of solution D7, wherein the shape is a triangle.

[0195] D9. A method according to solution D7, wherein the shape is an area defined by x<ky; x, y=0, 1, 2, ...M-1, where x and y are relative coordinates relative to the start point of the first video block, and where k is a predefined value.

[0196] D10. A method according to solution D7, wherein the shape is an area defined by x>ky, x, y=0, 1, 2, ...M-1, wherein x and y are relative coordinates relative to the start point of the first video block, and wherein k is a predefined value.

[0197] D11. A method according to solution D7, wherein the shape is defined as x<M-ky, x, y=0, 1, 2, ...M-1, wherein x and y are relative coordinates relative to the start point of the first video block, and wherein k is a predefined value.

[0198] D12. A method according to solution D7, wherein the shape is defined as x>M-ky, x, y=0, 1, 2, ...M-1, wherein x and y are relative coordinates relative to the start point of the first video block, and wherein k is a predefined value.

[0199] D13. The method of solutions D1-D12, wherein the hash-based motion search is based on a fixed subset of the first video blocks.

[0200] D14. The method according to solution D13, wherein the fixed subset is defined by a set of predefined coordinates relative to the start point of the first video block.

[0201] D15. The method of solutions D1-D14, wherein the hash-based motion search comprises adding hash-based motion vectors (MVs) as a starting point during the MV estimation technique.

[0202] D16. A method according to solution D15, wherein a hash-based MV is checked for optimal starting point initialization in integer motion estimation.

[0203] D17. The method according to solutions D1-D16 also includes: determining a hash-based motion search that results in a hash match; determining that a quantization parameter (QP) of the reference frame is greater than the QP of the first video block; and determining a rate-distortion cost of the skip mode based on the determination that the hash-based motion search results in a hash match and the QP of the reference frame is greater than the QP of the first video block, wherein further processing of the first video block is performed based on the rate-distortion cost.

[0204] D18. The method according to solution D17, wherein determining the rate-distortion cost includes skipping Advanced Motion Vector Prediction (AMVP).

[0205] D19. The method according to solution D17, wherein determining the rate-distortion penalty comprises stopping finer-grained block partition checking.

[0206] D20. The method of solution D17, wherein determining the rate-distortion cost is based on a size of the first video block.

[0207] D21. The method of solution D20, wherein a size of the first video block is greater than or equal to M×N.

[0208] D22. The method of solution D20, wherein the size of the first video block is 64×64 pixels.

[0209] D23. The method of solution D1, wherein the hash pattern of the first video block is based on a sub-square hash value, the first video block having a rectangular shape.

[0210] D24. The method of solution D1, wherein the hash pattern of the first video block is based on a sub-square hash value, the first video block having a square shape.

[0211] D25. A method according to solution D1 or D7, wherein the motion information comprises a motion vector (MV).

[0212] D26. The method according to solution D1 or D7, further comprising: performing a K-pixel integer motion vector (MV) accuracy check on the hash-based motion search.

[0213] D27. The method according to solution D26, wherein K is 4.

[0214] D28. The method of solutions D1-D27, wherein the first video block is a fractional size of an allowed block size.

[0215] D29. The method of solutions D1-D27, wherein the first video block has a square shape, and hash-based motion search is applied to the N×N video block, wherein N is less than or equal to a threshold.

[0216] D30. A method according to solution D29, wherein the threshold is 64.

[0217] D31. The method according to solutions D1-D27, wherein the first video block has a rectangular shape and the hash-based motion search is applied to the M×N video block, wherein M*N is less than or equal to a threshold.

[0218] D32. A method according to solution D31, wherein the threshold is 64.

[0219] D33. The method of solutions D1-D27, wherein the first video block is 4×8 or 8×4.

[0220] D34. A method according to solutions D1-D27, wherein a hash-based motion search is applied to color components.

[0221] D35. The method according to solutions D1-D16 also includes: determining a hash-based motion search that results in a hash match; determining that a quantization parameter (QP) of a reference frame is not greater than a QP of a first video block; determining a rate-distortion cost of a skip mode based on the hash-based motion search that results in a hash match and the QP of the reference frame is greater than the QP of the first video block, wherein further processing of the first video block is performed based on the rate-distortion cost.

[0222] D36. A method according to solution D35, wherein the skip mode includes a skip mode for affine mode, triangle mode or adaptive motion vector resolution (AMVR) mode.

[0223] D37. The method according to solution D35, wherein determining the rate-distortion penalty comprises stopping finer-grained block partition checking.

[0224] D38. A video decoding device comprising a processor configured to implement the method described in one or more of solutions D1 to D37.

[0225] D39. According to a video encoding device, comprising a processor configured to implement the method described in one or more of solutions D1 to D37.

[0226] D40. A computer program product having computer code stored thereon, which, when executed by a processor, causes the processor to implement the method described in any one of solutions D1 to D37.

[0227] Fig.101 is a block diagram of a video processing device 1000. Device 1000 may be used to implement one or more methods described herein. Device 1000 may be embodied in a smartphone, a tablet computer, a computer, an Internet of Things (IoT) receiver, etc. Device 1000 may include one or more processors 1002, one or more memories 1004, and video processing hardware 1006. Processor(s) 1002 may be configured to implement one or more methods described in this document. Memory(s) 1004 may be used to store data and code for implementing the methods and techniques described herein. Video processing hardware 1006 may be used to implement some of the techniques described in this document in hardware circuits.

[0228] In some embodiments, the video encoding method may use Fig.10 The described device is implemented on a hardware platform.

[0229] Some embodiments of the disclosed technology include making a decision or determining to enable a video processing tool or mode. In an example, when a video processing tool or mode is enabled, the encoder will use or implement the tool or mode in the processing of a video block, but will not necessarily modify the generated bitstream based on the use of the tool or mode. That is, the conversion from a video block to a bitstream representation of a video will use the video processing tool or mode when enabling the video processing tool or mode based on a decision or determination. In another example, when a video processing tool or mode is enabled, the decoder processes the bitstream knowing that the bitstream has been modified based on the video processing tool or mode. That is, the conversion from the bitstream representation of a video to a video block will be performed using the video processing tool or mode enabled based on a decision or determination.

[0230] Some embodiments of the disclosed technology include making a decision or determination to disable a video processing tool or mode. In an example, when the video processing tool or mode is disabled, the encoder will not use the tool or mode in the conversion of video blocks to a bitstream representation of the video. In another example, when the video processing tool or mode is disabled, the decoder processes the bitstream knowing that the bitstream is not modified using the video processing tool or mode enabled based on the decision or determination.

[0231] Fig.111 is a block diagram illustrating an example video processing system 1100 in which various techniques disclosed herein may be implemented. Various implementations may include some or all of the components of system 1100. System 1100 may include an input 1102 for receiving video content. The video content may be received in a raw or uncompressed format (e.g., 8 or 10-bit multi-component pixel values), or may be received in a compressed or encoded format. Input 1102 may represent a network interface, a peripheral bus interface, or a storage interface. Examples of network interfaces include wired interfaces (such as Ethernet, passive optical networks (PONs), etc.) and wireless interfaces (such as Wi-Fi or cellular interfaces).

[0232] System 1100 may include a coding component 1104, which may implement various encodings or encoding methods described in this document. Coding component 1104 may reduce the average bit rate of a video from input 1102 to the output of coding component 1104 to generate a coded representation of the video. Therefore, coding techniques are sometimes referred to as video compression or video transcoding techniques. As represented by component 1106, the output of coding component 1104 may be stored or sent via a connected communication. The bitstream (or encoding) representation of the storage or transmission of the video received at input 1102 may be used by component 1108 to generate pixel values ​​or displayable video, which may be sent to display interface 1110. The process of generating user-visible video from a bitstream representation is sometimes referred to as video decompression. In addition, although some video processing operations are referred to as "coding" operations or tools, it should be understood that the coding tools or operations are used at the encoder, and the corresponding decoding tools or operations that reverse the coding results will be performed by the decoder.

[0233] Examples of peripheral bus interfaces or display interfaces may include Universal Serial Bus (USB) or High Definition Multimedia Interface (HDMI) or Displayport, etc. Examples of storage interfaces include SATA (Serial Advanced Technology Attachment), PCI, IDE interfaces, etc. The technology described in this document may be embodied in various electronic devices, such as mobile phones, laptop computers, smart phones, or other devices capable of performing digital data processing and / or video display.

[0234] The disclosed and other solutions, examples, embodiments, modules and functional operations described in this document can be implemented in digital electronic circuits, or computer software, firmware or hardware, including the structures disclosed in this document and their structural equivalents, or a combination of one or more thereof. The implementation of the subject matter described in this specification can be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a tangible and non-volatile computer-readable medium for execution by a data processing device or to control the operation of the data processing device. The computer-readable medium can be a machine-readable storage device, a machine-readable storage substrate, a storage device, a material composition that affects a machine-readable propagation signal, or a combination of one or more thereof. The term "data processing device" includes all devices, equipment and machines for processing data, including, for example, a programmable processor, a computer or a multiprocessor or a computer group. In addition to hardware, the device may also include code that creates an execution environment for a computer program, for example, code constituting a processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more thereof. A propagated signal is an artificially generated signal, for example, a machine-generated electrical, optical or electromagnetic signal, which is generated to encode information for transmission to a suitable receiver device.

[0235] A computer program (also referred to as a program, software, software application, script, or code) may be written in any form of programming language (including compiled or interpreted languages) and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program may be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program, or in multiple coordinated files (e.g., files that store one or more modules, subroutines, or portions of code). A computer program may be deployed for execution on one or more computers that are located at one site or distributed across multiple sites and interconnected by a communications network.

[0236] The processes and logic flows described in this specification may be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows may also be performed by special purpose logic circuits, and the apparatus may also be implemented as special purpose logic circuits, such as FPGAs (field programmable gate arrays) or ASICs (application specific integrated circuits).

[0237] For example, processors suitable for executing computer programs include general and special purpose microprocessors, and any one or more of any type of digital computer. Typically, the processor will receive instructions and data from a read-only memory or a random access memory or both. The basic elements of a computer are a processor that executes instructions and one or more storage devices that store instructions and data. Typically, a computer will also include one or more mass storage devices for storing data, such as magnetic disks, magneto-optical disks, or optical disks, or be operatively coupled to one or more mass storage devices to receive data from them or transfer data to one or more mass storage devices, or both. However, a computer does not necessarily have such a device. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices, such as EPROM, EEPROM, and flash memory devices; magnetic disks, such as internal hard disks or removable hard disks; magneto-optical disks; and CD ROM and DVD ROM disks. The processor and memory may be supplemented by, or incorporated into, dedicated logic circuits.

[0238] Although this patent document contains many details, they should not be interpreted as limitations on any subject matter or the scope of the claims, but rather as descriptions of features of specific embodiments of specific technologies. Certain features described in this patent document in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various functions described in the context of a single embodiment can also be implemented separately in multiple embodiments, or in any suitable subcombination. In addition, although the above-mentioned features may be described as working in certain combinations, or even initially required to be so, in some cases, one or more features in the claim combination can be removed from the combination, and the combination of claims can be directed to subcombinations or variations of subcombinations.

[0239] Likewise, although operations are described in a particular order in the drawings, this should not be understood as requiring that such operations be performed in the particular order or order shown, or that all illustrated operations be performed, in order to achieve the desired results. In addition, the separation of various system components in the embodiments described in this patent document should not be understood as requiring such separation in all embodiments.

[0240] Only a few implementations and examples are described, and other implementations, enhancements, and variations can be made based on what is described and illustrated in this patent document.

Claims

1. A method for processing video data, comprising: Determining motion information associated with a current block of a video using a hash-based motion search for conversion between the current block and a bitstream of the video, the hash-based motion search being based on hash values ​​of square sub-blocks of the current block, wherein the current block has a size of M×N, and wherein M and N are positive integers; Applying prediction to the current block based on the motion information and a video picture including the current block; and performing the transforming based on the prediction, Wherein, the method further comprises: determining that the hash-based motion search results in a hash match, Determining that a quantization parameter QP of the reference frame is greater than the QP of the video block, and determining a rate-distortion cost for the prediction based on (a) the hash-based motion search results in a hash match and (b) the QP of the reference frame is greater than the QP of the video block, wherein further processing of the current block is based on the rate-distortion cost, and Wherein, determining the rate-distortion cost (a) is based on the size of the current block and (b) includes skipping Advanced Motion Vector Prediction (AMVP) or stopping finer-grained block partition checking.

2. The method according to claim 1, wherein: M is not equal to N.

3. The method according to claim 1, wherein: M equals N.

4. The method according to claim 1, wherein: The hash-based motion search includes performing a K-pixel integer motion vector MV accuracy check for the hash-based motion search, where K is a positive integer.

5. The method according to claim 4, wherein: K＝4。 6. The method according to any one of claims 1 to 5, wherein: The size of the current block is N×N, where N is a positive integer less than or equal to a threshold.

7. The method according to any one of claims 1 to 5, wherein: The size of the current block is M×N, wherein M and N are positive integers, and M is not equal to N, and wherein M or N is less than or equal to a threshold.

8. The method according to claim 6, wherein: The threshold is 64.

9. The method according to claim 7, wherein: The threshold is 64.

10. The method according to any one of claims 1 to 5, wherein: The size of the current block is 8×4 or 4×8.

11. The method according to any one of claims 1 to 5, wherein: The hash-based motion search is applied to the luma component of the video.

12. The method according to any one of claims 1 to 5, wherein: The hash-based motion search is applied to the primary color component of the video.

13. The method according to any one of claims 1 to 5, wherein: The converting includes decoding the current block from the bitstream.

14. The method according to any one of claims 1 to 5, wherein: The converting includes encoding the current block into the bitstream.

15. An apparatus in a video system, comprising a processor and a non-transitory memory having instructions thereon, wherein: The instructions, when executed by the processor, cause the processor to implement the method of any one of claims 1 to 14 .

16. A computer program product stored on a non-transitory computer readable medium, the computer program product comprising program code for executing the method of any one of claims 1 to 14.

Images